NodeJS - Is Mongoose faster than MongoJS?

Question

I've got an application that runs on NodeJS, and uses MongoDB as a database for information.

Currently I hook into MongoDB via the MongoJS module, which aims to "emulate the official mongodb API as much as possible".

The application gets about 20,000 objects and saves each one to MongoDB. First, it looks up the database to see whether the object already exists, then it either updates the existing entry or adds a new entry.

This can be quite slow. I'm not sure if it's because MongoJS is synchronous/single-stream (if it even is - I'm not sure!), or it's just a reality of writing a lot of entries to a DB, but it takes 45min-1hr to do all of this and I'd obviously like to cut this down as much as possible.

I'm wondering whether Mongoose would be a better/faster option for this. Apparently it's asynchronous, I don't know whether that would have an affect at all on performance.

Answer 1

My guess is the bottle neck is your application getting the objects. If you were to synthesize fake objects in RAM and slam then into mongo, you would be getting probably more like several hundred objects into the DB per second (rough guess, but at least 2 orders of magnitude more than the 1/s you claim). In either case FYI both MongoJS and Mongoose are asynchronous, as is pretty much every node database API. In fact, AFAIK there is simply no synchronous networking API in node, so while it's possible to wrestle node into synchronous filesystem I/O, all networking in node might be inherently asynchronous.

In either case, mongoose is going to add a tiny amount of overhead so it's likely to be technically slower than mongojs, but not by any meaningful amount (like it takes 5.1 seconds to insert 20K records vs 5.0 seconds).

Answer 2

First, it looks up the database to see whether the object already exists, then it either updates the existing entry or adds a new entry.

You can let mongodb do this for you with a single update command and the 'upsert' (update/insert) option set:

http://docs.mongodb.org/manual/reference/method/db.collection.update/#db.collection.update

Try that first and see if things speed up for you.