Repairing a corrupted sqlite database retrieved from disk data recovery

Question

So I pulled a sqlite database from a formatted disk using testdisk. I have the file but it has a bunch of weird stuff in the middle of it that looked like some log from something. The point is that it is severely corrupted.

I do have an earlier version of the sqlite file though....

The earlier version of the file contains all the data that has weird stuff in it in the other file. The newer entries that the corrupted files has that the uncorrupted file does not have appear to be fine.

I have tried to repair the corrupted file using the various echo .dump methods and all that, but I get an empty file.

Baxters-MacBook-Pro:desktop bax$ sqlite3 corrupted.sqlite3
SQLite version 3.7.13 2012-06-11 02:05:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> pragma integrity_check;
Error: database disk image is malformed

dumping gives me:

PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
/****** CORRUPTION ERROR *******/
/****** database disk image is malformed ******/
/****** ERROR: database disk image is malformed ******/
/****** CORRUPTION ERROR *******/
/****** database disk image is malformed ******/
/****** ERROR: database disk image is malformed ******/
/**** ERROR: (11) database disk image is malformed *****/
ROLLBACK; -- due to errors

I tried joining them dumbly in HEX. I pasted the contents of the old (not corrupted database) over the overlapping information in the corrupted file. That is, the last bit of data in the uncorrupted file was a timestamp, 15:15:14.419734 (the database is from an old rails app). I found that in the corrupted file and pasted the uncorrupted file hex over everything before that point in the corrupted file. So now it opens, but not surprisingly I can only get to the data from the uncorrupted file (about 122 of 160-something entries).

Where the two join looks like this:

15:15:14.419734√Ö√£)    �√ÖYAA{1B, 8B

1B, 8B is the beginning of an entry.

In hex:

31353A 31353A31 342E3431 39373334 C3851FC3 A3290900 
01C38501 59014141 7B31422C 203842

Is there any way for me to join these two databases, or am I boned?

Answer 1

In SQLite files, table data is stored in b-trees; it is likely that your joined file does not contain the correct pointers in the upper tree levels.

The FAQ says:

Depending how badly your database is corrupted, you may be able to recover some of the data by using the CLI to dump the schema and contents to a file and then recreate. Unfortunately, once humpty-dumpty falls off the wall, it is generally not possible to put him back together again.

But you could either

• read the file format documentation and try to extract as much data as posssible from the file by hand or with some ad-hoc tool; or
• pay somebody to do this for you.