iPhone: sqlite3 blob field not unserialising properly

StackOverflow https://stackoverflow.com/questions/12975031

Question

I've got an iPhone/iPad app that has some data that's stored in a property list. I'm expanding the app to handle multiple occurrences of the same data, unfortunately its too tied to the properly list to just convert the whole lot to sqlite3 or core data so I need to store the property list inside an sqlite3 table in a blob field.

I've serialised the data using [NSPropertySerialisation dataFromPropertyList:format:errorDescription] and can successfully unserialise it back to an NSDictionary in testing. I can store the data fine but when I retrieve it I cannot get it back into [NSPropertySerialisation propertyListFromData] correctly.

Here is the NSDictionary object:

{
    companydetails =     {
        address1 = "Company Address";
        "company_name" = "New Survey Company";
    };
}

Converted to serialised data with:

NSData *data = [NSPropertyListSerialization dataFromPropertyList:[Survey data] format:NSPropertyListXMLFormat_v1_0 errorDescription:&errStr];

Which yields:

<3c3f786d 6c207665 7273696f 6e3d2231 2e302220 656e636f 64696e67 3d225554 462d3822 3f3e0a3c 21444f43 54595045 20706c69 73742050 55424c49 4320222d 2f2f4170 706c652f 2f445444 20504c49 53542031 2e302f2f 454e2220 22687474 703a2f2f 7777772e 6170706c 652e636f 6d2f4454 44732f50 726f7065 7274794c 6973742d 312e302e 64746422 3e0a3c70 6c697374 20766572 73696f6e 3d22312e 30223e0a 3c646963 743e0a09 3c6b6579 3e636f6d 70616e79 64657461 696c733c 2f6b6579 3e0a093c 64696374 3e0a0909 3c6b6579 3e616464 72657373 313c2f6b 65793e0a 09093c73 7472696e 673e436f 6d70616e 79204164 64726573 733c2f73 7472696e 673e0a09 093c6b65 793e636f 6d70616e 795f6e61 6d653c2f 6b65793e 0a09093c 73747269 6e673e4e 65772053 75727665 7920436f 6d70616e 793c2f73 7472696e 673e0a09 3c2f6469 63743e0a 3c2f6469 63743e0a 3c2f706c 6973743e 0a>

I can then unserialise this with:

NSDictionary *revData = [NSPropertyListSerialization propertyListFromData:data mutabilityOption:NSPropertyListImmutable format:NULL errorDescription:&errStr];

Which returns me back to:

{
    companydetails =     {
        address1 = "Company Address";
        "company_name" = "New Survey Company";
    };
}

This is my schema for the table:

CREATE TABLE surveys (  id integer primary key autoincrement,   company_name text,   address text,   surveydata blob );

Here is my insert statement for the database:

NSString *insSQL = [NSString stringWithFormat:@"INSERT INTO surveys (company_name, address, surveydata) VALUES (\"%@\", \"%@\", \"%@\")", SURVEY_DEFAULT_COMPANY, SURVEY_DEFAULT_ADDRESS, data];
const char *sql = [insSQL UTF8String];

Which yields:

INSERT INTO surveys (company_name, address, surveydata) VALUES ("New Survey Company", "Company Address", "<3c3f786d 6c207665 7273696f 6e3d2231 2e302220 656e636f 64696e67 3d225554 462d3822 3f3e0a3c 21444f43 54595045 20706c69 73742050 55424c49 4320222d 2f2f4170 706c652f 2f445444 20504c49 53542031 2e302f2f 454e2220 22687474 703a2f2f 7777772e 6170706c 652e636f 6d2f4454 44732f50 726f7065 7274794c 6973742d 312e302e 64746422 3e0a3c70 6c697374 20766572 73696f6e 3d22312e 30223e0a 3c646963 743e0a09 3c6b6579 3e636f6d 70616e79 64657461 696c733c 2f6b6579 3e0a093c 64696374 3e0a0909 3c6b6579 3e616464 72657373 313c2f6b 65793e0a 09093c73 7472696e 673e436f 6d70616e 79204164 64726573 733c2f73 7472696e 673e0a09 093c6b65 793e636f 6d70616e 795f6e61 6d653c2f 6b65793e 0a09093c 73747269 6e673e4e 65772053 75727665 7920436f 6d70616e 793c2f73 7472696e 673e0a09 3c2f6469 63743e0a 3c2f6469 63743e0a 3c2f706c 6973743e 0a>")

When I retrieve the data from the table in another function, I get the correct blob data back:

<3c3f786d 6c207665 7273696f 6e3d2231 2e302220 656e636f 64696e67 3d225554 462d3822 3f3e0a3c 21444f43 54595045 20706c69 73742050 55424c49 4320222d 2f2f4170 706c652f 2f445444 20504c49 53542031 2e302f2f 454e2220 22687474 703a2f2f 7777772e 6170706c 652e636f 6d2f4454 44732f50 726f7065 7274794c 6973742d 312e302e 64746422 3e0a3c70 6c697374 20766572 73696f6e 3d22312e 30223e0a 3c646963 743e0a09 3c6b6579 3e636f6d 70616e79 64657461 696c733c 2f6b6579 3e0a093c 64696374 3e0a0909 3c6b6579 3e616464 72657373 313c2f6b 65793e0a 09093c73 7472696e 673e436f 6d70616e 79204164 64726573 733c2f73 7472696e 673e0a09 093c6b65 793e636f 6d70616e 795f6e61 6d653c2f 6b65793e 0a09093c 73747269 6e673e4e 65772053 75727665 7920436f 6d70616e 793c2f73 7472696e 673e0a09 3c2f6469 63743e0a 3c2f6469 63743e0a 3c2f706c 6973743e 0a>

However, it's when I come to convert this back to a property list that i'm getting problems:

So, I select my data using:

SELECT surveydata FROM surveys WHERE id = 2

The raw data in sqlite3_column_blob(stmt,0) is:

NSString *blobRaw = [NSString stringWithFormat:@"%s", sqlite3_column_blob(stmt, 0)];
NSLog(@"Raw BLOB = %@", blobRaw);

Raw BLOB = <3c3f786d 6c207665 7273696f 6e3d2231 2e302220 656e636f 64696e67 3d225554 462d3822 3f3e0a3c 21444f43 54595045 20706c69 73742050 55424c49 4320222d 2f2f4170 706c652f 2f445444 20504c49 53542031 2e302f2f 454e2220 22687474 703a2f2f 7777772e 6170706c 652e636f 6d2f4454 44732f50 726f7065 7274794c 6973742d 312e302e 64746422 3e0a3c70 6c697374 20766572 73696f6e 3d22312e 30223e0a 3c646963 743e0a09 3c6b6579 3e636f6d 70616e79 64657461 696c733c 2f6b6579 3e0a093c 64696374 3e0a0909 3c6b6579 3e616464 72657373 313c2f6b 65793e0a 09093c73 7472696e 673e436f 6d70616e 79204164 64726573 733c2f73 7472696e 673e0a09 093c6b65 793e636f 6d70616e 795f6e61 6d653c2f 6b65793e 0a09093c 73747269 6e673e4e 65772053 75727665 7920436f 6d70616e 793c2f73 7472696e 673e0a09 3c2f6469 63743e0a 3c2f6469 63743e0a 3c2f706c 6973743e 0a>

(NB: i'm just selecting the one field so it's at column 0)

[NSPropertyListSerialisation propertyListFromData] expects NSData as the input so i'm trying to cast the blob to NSData with:

NSData *blobData = [NSData dataWithBytes:sqlite3_column_blob(stmt, 0) length:sqlite3_column_bytes(stmt, 0)];

But if I output the NSData I get:

<3c336333 66373836 64203663 32303736 36352037 32373336 39366620 36653364 32323331 20326533 30323232 30203635 36653633 36662036 34363936 65363720 33643232 35353534 20343632 64333832 32203366 33653061 33632032 31343434 66343320 35343539 35303435 20323037 30366336 39203733 37343230 35302035 35343234 63343920 34333230 32323264 20326632 66343137 30203730 36633635 32662032 66343435 34343420 32303530 34633439 20353335 34323033 31203265 33303266 32662034 35346532 32323020 32323638 37343734 20373033 61326632 66203737 37373737 32652036 31373037 30366320 36353265 36333666 20366432 66343435 34203434 37333266 35302037 32366637 30363520 37323734 37393463 20363937 33373432 64203331 32653330 32652036 34373436 34323220 33653061 33633730 20366336 39373337 34203230 37363635 37322037 33363936 66366520 33643232 33313265 20333032 32336530 61203363 36343639 36332037 34336530 61303920 33633662 36353739 20336536 33366636 64203730 36313665 37392036 34363537 34363120 36393663 37333363 20326636 62363537 39203365 30613039 33632036 34363936 33373420 33653061 30393039 20336336 62363537 39203365 36313634 36342037 32363537 33373320 33313363 32663662 20363537 39336530 61203039 30393363 37332037 34373236 39366520 36373365 34333666 20366437 30363136 65203739 32303431 36342036 34373236 35373320 37333363 32663733 20373437 32363936 65203637 33653061 30392030 39336336 62363520 37393365 36333666 20366437 30363136 65203739 35663665 36312036 64363533 63326620 36623635 37393365 20306130 39303933 63203733 37343732 36392036 65363733 65346520 36353737 32303533 20373537 32373636 35203739 32303433 36662036 64373036 31366520 37393363 32663733 20373437 32363936 65203637 33653061 30392033 63326636 34363920 36333734 33653061 20336332 66363436 39203633 37343365 30612033 63326637 30366320 36393733 37343365 2030613e>

Oddly enough if I then pass this NSData through [NSPropertySerialization propertyListFromData] I get:

<3c3f786d 6c207665 7273696f 6e3d2231 2e302220 656e636f 64696e67 3d225554 462d3822 3f3e0a3c 21444f43 54595045 20706c69 73742050 55424c49 4320222d 2f2f4170 706c652f 2f445444 20504c49 53542031 2e302f2f 454e2220 22687474 703a2f2f 7777772e 6170706c 652e636f 6d2f4454 44732f50 726f7065 7274794c 6973742d 312e302e 64746422 3e0a3c70 6c697374 20766572 73696f6e 3d22312e 30223e0a 3c646963 743e0a09 3c6b6579 3e636f6d 70616e79 64657461 696c733c 2f6b6579 3e0a093c 64696374 3e0a0909 3c6b6579 3e616464 72657373 313c2f6b 65793e0a 09093c73 7472696e 673e436f 6d70616e 79204164 64726573 733c2f73 7472696e 673e0a09 093c6b65 793e636f 6d70616e 795f6e61 6d653c2f 6b65793e 0a09093c 73747269 6e673e4e 65772053 75727665 7920436f 6d70616e 793c2f73 7472696e 673e0a09 3c2f6469 63743e0a 3c2f6469 63743e0a 3c2f706c 6973743e 0a>

Which is identical to the serialised input and the blob data.

Please can anyone help me with how to store the serialised property list and recover it from the sqlite table - BLOB seemed to be the right way to store the data but I can't reconstitute it.

Was it helpful?

Solution

In general, blob data types are not suitable to just insert into a SQL statement within quotes. So, instead of your:

NSString *insSQL = [NSString stringWithFormat:@"INSERT INTO surveys (company_name, address, surveydata) VALUES (\"%@\", \"%@\", \"%@\")", SURVEY_DEFAULT_COMPANY, SURVEY_DEFAULT_ADDRESS, data];

You might want to consider: 

NSString *insSQL = @"INSERT INTO surveys (company_name, address, surveydata) VALUES (?, ?, ?)";
if (sqlite3_bind_text(database, 1, [SURVEY_DEFAULT_COMPANY UTF8String]) != SQLITE_OK)
    NSLog("Bind error on #%d: '%s'", 1, sqlite3_errmsg(database));
if (sqlite3_bind_text(database, 2, [SURVEY_DEFAULT_ADDRESS UTF8String]) != SQLITE_OK)
    NSLog("Bind error on #%d: '%s'", 2, sqlite3_errmsg(database));
if (sqlite3_bind_blob(database, 3, [data bytes], [data length], SQLITE_STATIC) != SQLITE_OK)
    NSLog("Bind error on #%d: '%s'", 3, sqlite3_errmsg(database));

I haven't tested the above code, but hopefully you get the idea. Don't build your SQL statement yourself, but rather use the sqlite3_bind functions.

Here's a more complete example of how to serialize a NSDictionary, store in database, and then to retrieve and unserialize. Here's how you serialize and save the data in your SQLite database:

- (IBAction)saveButtonClicked:(id)sender
{
    sqlite3 *database;

    NSString *documentsDirectory = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES)[0];
    NSString *databaseName = [documentsDirectory stringByAppendingPathComponent:@"test.db"];

    if (sqlite3_open([databaseName UTF8String], &database) != SQLITE_OK)
    {
        NSLog(@"open failed");
        return;
    }

    NSString *sql = @"CREATE TABLE IF NOT EXISTS company (company_id INTEGER PRIMARY KEY AUTOINCREMENT, company_details BLOB);";

    if (sqlite3_exec(database, [sql UTF8String], NULL, NULL, NULL) != SQLITE_OK)
    {
        NSLog(@"create failed %s", sqlite3_errmsg(database));
        sqlite3_close(database);
        return;
    }

    NSDictionary *dictionary = @{@"name"   : @"IBM",
                                 @"street" : @"1 New Orchard Road",
                                 @"city"   : @"Armonk",
                                 @"state"  : @"New York",
                                 @"postal" : @"10504-1722"};

    NSData *data= [NSPropertyListSerialization dataFromPropertyList:dictionary
                                                             format:NSPropertyListXMLFormat_v1_0
                                                   errorDescription:nil];

    sql = @"INSERT INTO company (company_details) VALUES (?);";
    sqlite3_stmt *statement;

    if (sqlite3_prepare_v2(database, [sql UTF8String], -1, &statement, NULL) != SQLITE_OK)
    {
        NSLog(@"prepare failed %s", sqlite3_errmsg(database));
        sqlite3_close(database);
        return;
    }

    if (sqlite3_bind_blob(statement, 1, [data bytes], [data length], SQLITE_STATIC) != SQLITE_OK)
    {
        NSLog(@"bind failed %s", sqlite3_errmsg(database));
        sqlite3_finalize(statement);
        sqlite3_close(database);
        return;
    }

    if (sqlite3_step(statement) != SQLITE_DONE)
    {
        NSLog(@"step failed %s", sqlite3_errmsg(database));
    }
    else
    {
        NSLog(@"success");
    }

    sqlite3_finalize(statement);

    sqlite3_close(database);
}

And to load and unserialize the data:

- (IBAction)loadButtonClicked:(id)sender
{
    sqlite3 *database;

    NSString *documentsDirectory = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES)[0];
    NSString *databaseName = [documentsDirectory stringByAppendingPathComponent:@"test.db"];

    if (sqlite3_open([databaseName UTF8String], &database) != SQLITE_OK)
    {
        NSLog(@"open failed");
        return;
    }

    sqlite3_stmt *statement;
    NSString *sql = @"SELECT * FROM company;";

    if (sqlite3_prepare_v2(database, [sql UTF8String], -1, &statement, NULL) != SQLITE_OK)
    {
        NSLog(@"prepare failed %s", sqlite3_errmsg(database));
        return;
    }

    while (sqlite3_step(statement) == SQLITE_ROW)
    {
        NSInteger companyId = sqlite3_column_int(statement, 0);
        const void *blob = sqlite3_column_blob(statement, 1);
        NSInteger bytes = sqlite3_column_bytes(statement, 1);
        NSData *data = [NSData dataWithBytes:blob length:bytes];

        NSDictionary *dictionary = [NSPropertyListSerialization propertyListFromData:data
                                                                    mutabilityOption:NSPropertyListImmutable
                                                                              format:NULL
                                                                    errorDescription:nil];

        NSLog(@"%@", @{@"company_id"      : @(companyId),
                       @"company_details" : dictionary});
    }

    sqlite3_finalize(statement);

    sqlite3_close(database);
}

OTHER TIPS

You are putting a string containing an ASCII representation of the blob into the database:

INSERT INTO surveys(...) VALUES(..., "<3c3f786d...")

However, in SQLite, a blob literal must be formatted as a single-quoted string with an x prefix, and must contain any other characters like <>:

INSERT INTO surveys(...) VALUES(..., x'3c3f786d...')

Anyway, to avoid formatting problems like this, using parameters like in Rob's answer is preferred.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top