How to change the Maximum Depth Of An Expression Tree of SQLite with FMDB SDK?

Question

I am using FMDB to implement a SQLite based apps, which I need to insert a large amount of data into SQLite, say 1000 rows of data, but I dont want to insert them one by one, I just want to do this with a single query, this make the insert progress much faster, like this:

NSString *query = @"insert into table (c1,c2,c3,c4) values " ;
NSString *middle = @"" ;
for ( int i = 0 ; i < 1000 ; i++ ) {

    NSDictionary *tmpItem = [data objectAtIndex:i] ;

    if ( i == 999 ) {

        middle = [NSString stringWithFormat:@"%@('%@','%@','%@','%@')", middle,[tmpItem objectForKey:@"c1"],[tmpItem objectForKey:@"c2"],[tmpItem objectForKey:@"c3"],[tmpItem objectForKey:@"c4"]] ;
    }

    else {

        middle = [NSString stringWithFormat:@"%@('%@','%@','%@','%@') ,", middle,[tmpItem objectForKey:@"c1"],[tmpItem objectForKey:@"c2"],[tmpItem objectForKey:@"c3"],[tmpItem objectForKey:@"c4"]] ;
    }
}

query = [NSString stringWithFormat:@"%@%@", query, middle] ;
[db executeUpdate:query];

Above code cannot insert any data on "table", but when I change the for-loop to run only 10 times, data inserted correctly, that means there is no syntax error.

I tried by error that, the data is inserted correctly with the for-loop set up to i<500, while setting i<501, no data is inserted.

Then I search http://www.sqlite.org/limits.html , there is a Maximum Depth Of An Expression Tree on this page, and it should be what I want to set, but how can I set this property while using FMDB?

Or the only method I can insert a large amount of data is just insert them one by one?

Answer 1

Your options aren't limited to one or all at a time.

You could insert the data in batches which will still give you the speed gains but also be more predictable on success. Changing limits is still just that, still a limit (and there's others you may hit depending on data). Batching 100 at a time seems reasonable.

Also ensure that you're not doing large batching work on the main UI thread.

In addition to that, the sqlite link you provided has preprocessor compilations settings if you compile your own sqlite binary. FMDB is just a wrapper over sqlite and by default on iOS you'll get the sqlite packaged with iOS. Of course, you could compile your own with altered limits and link into your app.

So, you would need to recompile and link with a limit of 0. Note that the runtime limit call only allows you to lower the compiled limit if it's set above 0 (default is 100)

The maximum depth of an expression tree can be lowered at run-time using the sqlite3_limit(db,SQLITE_LIMIT_EXPR_DEPTH,size) interface if the SQLITE_MAX_EXPR_DEPTH is initially positive. In other words, the maximum expression depth can be lowered at run-time if there is already a compile-time limit on the expression depth. If SQLITE_MAX_EXPR_DEPTH is set to 0 at compile time (if the depth of expressions is unlimited) then the sqlite3_limit(db,SQLITE_LIMIT_EXPR_DEPTH,size) is a no-op.