SQLCipher for Android getReadableDatabase() Overherad

Question

I've modified my DatabaseHelper class to use the SQLCipher library.

To do that, I:

• Copied the assets into my assets folder and the libraries (armeabi, x86, commons-codec, guava-r09, sqlcipher) into my libs folder.

• Changed the imports in my DatabaseHelper class so that they point to import net.sqlcipher.database.* instead.

• Call SQLiteDatabase.loadLibs(getApplicationContext()); when the app starts up.

• Modified the lines where I call getReadableDatabase() and getWriteableDatabase() so that they include a passphrase as a parameter;

Everything seems to work fine as data is read/written properly. My issue is related to performance, as my app may execute DB operations with some frequency, causing it to become slow (after migrating to SQLCipher).

For my DatabaseHelper methods, I believe I'm following the standard approach, e.g.:

/*
 * Getting all MyObjects
 */
public List<MyObject> getMyObjects() {

    List<MyObject> objects = new ArrayList<MyObject>();

    String selectQuery = "SELECT * FROM " + TABLE_NAME;
    Log.v(LOG, selectQuery);

    // Open
    SQLiteDatabase db = this.getReadableDatabase("...the password...");  
    // I know this passphrase can be figured out by decompiling.

    // Cursor with query
    Cursor c = db.rawQuery(selectQuery, null);

    // looping through all rows and adding to list
    if (c.moveToFirst()) {
        do {
            MyObject object = createMyObjectFromCursor(c); // Method that builds MyObject from Cursor data
            // adding to list
            objects.add(object);
        } while (c.moveToNext());
    }

    c.close();
    db.close();
    return objects;
}

I'm not entirely familiar with the internal mechanics of SQLCipher (e.g. does it decrypt the whole DB file when I call getReadableDatabase()?) but, while debugging, it seems that the overhead is in getReadableDatabase(password) and getWritableDatabase(password), which makes sense if my supposition above is true.

Would moving those calls to a DatabaseHelper.open() and DatabaseHelper.close() method which would be called by the Activities whenever they instantiate a DatabaseHelper, instead of calling them on each individual method, be a bad practice? Please share your knowledge on how to address this issue.

EDIT:

I've used DDMS to trace one of the methods and I can see that the overhead is indeed at the SQLiteOpenHelper.getReadableDatabase() (taking ~4 sec. each time). The queries seem to work fast and I don't think I need to worry about them.

If I drill down the calls, following the one with the longest duration every time, I end up with:

SQLiteDatabase.OpenOrCreateDatabase --> SqLiteDatabase.openDatabase --> SQLiteDatabase.openDatabase --> SQLiteDatabase.setLocale

So the SQLiteDatabase.setLocale(java.util.Locale) seems to be the culprit, as it is taking ~4 seconds everytime getReadableDatabase() is called. I've looked into the source for SQLiteDatabase and it just locks the DB, calls native_setLocale(locale.toString(), mFlags) (the 4 sec. overhead takes place here) and unlocks the DB.

Any idea on why this happens?

Answer 1

The performance issue you are seeing is most likely due to SQLCipher key derivation. SQLCipher's performance for opening a database is deliberately slow, using PBKDF2 to perform key derivation (i.e. thousands of SHA1 operations) to defend against brute force and dictionary attacks (you can read more about this at http://sqlcipher.net/design). This activity is deferred until the first use of the database, which happens to occur in setLocale, which is why you are seeing the performance issue there when profiling.

The best option is to cache the database connection so that it can be used multiple times without having to open and key the database repeatedly. If this is possible, opening the database once during startup is the preferred course of action. Subsequent access on the same database handle will not trigger key derivation, so performance will be much faster.

If this is not possible the other option is to disable or weaken key derivation. This will cause SQLCipher to use fewer rounds of PBKDF2 when deriving the key. While this will make the database open faster, it is significantly weaker from a security perspective. Thus it is not recommended except in exceptional cases. That said, here is the information on how to reduce the KDF iterations:

http://sqlcipher.net/sqlcipher-api/#kdf_iter

Answer 2

does it decrypt the whole DB file when I call getReadableDatabase()?

No. It decrypts pages (4KB??) as needed, on the fly.

it seems that the overhead is in getReadableDatabase(password) and getWritableDatabase(password), which makes sense if my supposition above is true

Only call those once for the lifetime of your process. Anything else is insecure, as it requires you to keep the password around, above and beyond any overhead issues.

Of course, you seem to be hard-coding a password, in which case all this encryption is pointless and a waste of time.

Please share your knowledge on how to address this issue.

Use Traceview to determine exactly where your time is being spent.

In one benchmark that I performed -- converting a SQLite benchmark to SQLCipher -- I could not detect any material overhead. Disk I/O swamped the encryption overhead, near as I can tell.

To the extent that a well-written SQLCipher for Android app adds overhead, it will make bad operations worse. So, for example, a query that needs to do a table scan sucks already; SQLCipher will make it suck incrementally harder. The solution there is to add the appropriate indexes (or FTS3) as needed to avoid the table scan.