How to update piecewise a blob in SQLite?

Question

I have large blobs in an SQLite app, and need to update small segments of these blobs in a piecewise fashion. Something like saying "update bytes X through Y of blob B with data D", this can be done in other databases with Blob manipulating functions, but I can't find anything like this for SQLite, am I stuck? Or does SQLite have ways of manipulating Blobs?

Thanks.

Answer 1

SQLite 3.x supports this through the sqlite3_blob_write function.

int sqlite3_blob_write(sqlite3_blob *, const void *z, int n, int iOffset)

Note that this function is provided as part of the SQLite 3 C/C++ API. You'll need to program against this directly to use it.

If you're using some other higher level wrapper such as the System.Data.SQLite, last time I looked, you will not have access to this function.

Answer 2

This is not directly an answer to your question, but I have some experience using random access for (big) blobs in SQLite, and I advise you against using it, if you can. Here's why:

Blobs break the SQL query format entirely. If your blob data needs any kind of processing, it will certainly at some point need filtering. Whatever mechanism you have in place to deal with filtering in SQL will be useless in this regard.

Dealing with binary blobs wrapped in databases opposed to binary data in raw files limits your options. You will not be able to randomly read and write to data at the same time from multiple processes, which is possible with files. You can't use any tool that deals with this data and provides only a file I/O interface. You can't truncate or resize the blob. Files are simply a lot more versatile.

It may seem convenient to have everything contained within one file, as it simplifies backup and transfer, but the pain of working with blobs is simply not worth it.

So as your attorney, I advise you to write your blobs as raw files to the file system, and merely store a reference to the filename in your database. If your blobs are rather small and guaranteed not to grow, forget my advice.

Answer 3

I strongly agree with everything said by paniq. Using BLOBs limits your options considerably.

If you are using System.Data.SQLite you won't have real support for BLOBs. That's why I wrote my own class to handle them. You can find the code here: Sample BLOB code

Note that SQLite has several potential pitfalls for those who dare to work with BLOBs. One of the major problems is that SQLite loads the entire BLOB field to memory before allowing you to even check their length.. So expect a lot of memory thrashing if you have large BLOB fields...