How do I fill a collection with millions of dummy documents without a million inserts?

Question

I wish to fill a collection with user accounts, full of dummy data ofc. Now I don't expect calling 1 mil inserts will be smart, so I require how to do it database-side.

I sort of remember something about doing javascript in the database and perhaps that would be the way, but more specifically?

Answer 1

While you could just write code to insert documents using your favorite programming language (as many of the drivers offer techniques to insert documents in batches via an array structure), I'd suggest you consider creating either a JSON file or a CSV file containing the structure of your documents (maybe in multiple files if necessary for the import to work), and then using mongoimport, import all of the data.

http://docs.mongodb.org/manual/reference/program/mongoimport/

This way, you can create the file(s) once, and run the import directly from the database server without installing extra software/platforms/node/etc.

If you wanted to use Node.JS, you can use insert (documentation) and simply pass in an array of documents. I'd suggest doing it batches of some size far less than one million, and since it's going to be write heavy anyway, just do one batch at a time (don't rely on async behavior of JavaScript to get many connections going).