How to bulk insert / update with Linq2indexedDB

Question

I'm experiencing very bad performance and crashes with Linq2indexedDB when bulk inserting this way:

for(var i=0;i<clients.length;i++) {
    db.from(config.objectStoreName).insert(clients[i]).then(function(args){
        deferred.resolve(clients.length);
    }, function(args){
        deferred.reject("Client item couldn't be added!");
    }); 
}

When doing something very similar with the native indexedDB, it's working fine:

var store = db.transaction(["client"], "readwrite").objectStore("client");
for(var i=0;i<clients.length;i++) {
    var request = store.put(clients[i]);  
}

request.onsuccess = function() {
    deferred.resolve(clients.length);
}

request.onerror = function(e) {
    deferred.reject("Client item couldn't be added!");
}

When the array of "clients" doesn't go above a few 1000, it's ok, but by 50000 it's hanging and then the tab crashes. On the native implementation, it only takes a few seconds to upsert the 50000 (on Chrome).

Am I missing anything, i.e. is there another method to batch insert records with Linq2indexedDB or is Linq2indexedDB simply not working with batch insert/update?

Answer 1

Thank you for trying the Linq2indexedDB library. This is still a work in progress, so feedback like this is mutch appriciated. So if you have other feedback, just let me know and I will see what I can do for you.

You are correct the DbContext doesn't provide a bulk insert for now. I will look into providing this functionality.

I think I know why you have the performance issue. The way it currently works now is that a the connection is created and closed for every insert you do. This is a choice I made in the lib. I wanted to be sure to work on the latest db everytime I connect. I'm thinking about changing this and make it possible to cache the connection, this will definitly improve the performance.

Other things that can affect the performance: - Having debugging enabled (writes log information to the console) - The viewer inside the library. For every insert an update is sent to this object.

As you mentioned for now you can use the native solution, or you can take advantage of my wrapper inside the library.

var dbpromise = linq2indexedDB.core.db("name", 1);
var transactionPromise = linq2indexedDB.core.transaction(dbpromise, ["objectstore"]);
var objectStorePromise = linq2indexedDB.core.objectStore(transactionPromise, "objectstore");

linq2indexedDB.core.insert(objectStorePromise, {}).then(success, error);
linq2indexedDB.core.insert(objectStorePromise, {}).then(success, error);
linq2indexedDB.core.insert(objectStorePromise, {}).then(success, error);
...
linq2indexedDB.core.insert(objectStorePromise, {}).then(success, error);

function success(args){
    var data = args[0]; 
    var primaryKey= args[1]; 
    var transaction= args[2];
    var orignalevent = args[3];
}

function error(args){
   var error= args;
}

transactionPromise.then(function () { 
    // Transaction completed
    // Bulk insert done.
});