node-mongodb-native creates sequential object ids

Question

Somehow mongod-native creates sequential object ids for inserted objects. I would prefer it if the database could do this job, or mongodb-native could at least use the same generation strategy as the db does.

Inserting with mongodb-native yields:

"_id" : ObjectId("520cc99c00bd49c20180aad0")
"_id" : ObjectId("520cc99c00bd49c20180aad1")
"_id" : ObjectId("520cc99c00bd49c20180aad2")

Inserting in the same collection using the shell, yields a real id:

"_id" : ObjectId("520cc9f25aea0256082427e9")

What is causing this? Isn't this a big problem for replication/sharding?

Btw. I'm using

db: {
    forceServerObjectId: true
}

already with the only effect, that the _id is not set on the object in node...

Answer 1

First, look at the bytes of an ObjectId:

ObjectId is a 12-byte BSON type, constructed using:

a 4-byte value representing the seconds since the Unix epoch,
a 3-byte machine identifier,
a 2-byte process id, and
a 3-byte counter, starting with a random value.

So, if you create a series of ObjectIds on a single machine rapidly, you'll end up creating essentially the same _id as it will contain nearly the same bytes, with the exception of a 3-byte counter (as the timestamp, process id, and machine identifier will all be the same).

Most MongoDB drivers/clients create _ids locally by default, rather than on the Database server (and as they are all open source, you can take a look at each of the implementations to see the specifics of how the _id is generated. Sometimes, admittedly, it does require a bit of digging. Here's one for NodeJS for example).

The _id that was generated from the shell is no more "real" than the _ids generated from the client. They just have different seeding values (the machine and process Ids would of course be different).

By using an ObjectId for sharding, as of 2.4+ you've got two choices: range and hash. Each has it's pros and cons and ultimately, either work very well, depending on the nature of the writes, reads, queries, etc. that are needed. You can read more about that here.

Answer 2

If you have issue with sequential Object IDs , you can also create custom and random object ids

Since _id field is always indexed, and primary key, you need to make sure that different objectid is generated for each object. There are some guidelines to optimize user defined object ids :

http://docs.mongodb.org/manual/core/document/#record-documents

http://docs.mongodb.org/manual/reference/object-id/