Problems with memory usage of mongo db server

Question

I have a mongod process running in a webfaction server with 512MB RAM. I have a MongoDB database with two collections sizing 10MB and 2GB. Whenever I run a query on my collections through mongo shell, the RAM usage grows exceptionally high and the server is killed automatically by the hosting provider.

Is there any way to control the memory usage by mongod server? What am I doing wrong? PS: please guide me, as I am new to MongoDB and experimenting a little bit.

Answer 1

This is a problematic setup, and there isn't much you can do about it from the code side. The problem is that your application gets killed because it requires RAM. MongoDB is very greedy with RAM and will try to get what it can. By carefully avoiding larger result sets and large amounts of data you can probably delay this problem a bit, but why would you use a web-scale database in the first place, if that's impossible?

Use a provider that gives you a fixed amount of RAM so MongoDB knows how much RAM it can allocate. That might not be fast, but at least it won't simply kill the service out of nowhere.

Even if you have very good indexes, at least the indexes should fit into RAM. If the OS needs say 100MB (which is very conservative) you only have 400MB left for a 10GB data store. Realistically, there's probably only 200MB left for actual data storage in RAM, and that seems on the low end for a 10GB database.

Even then, do not code your software against unnecessarily strong limitations. A virtual machine with 1-2GB of RAM isn't that expensive, but more importantly, make sure the RAM is guaranteed and fixed (no 'boost' RAM).

Answer 2

The other way of avoiding mongodb crashes because of memory is add swap to your OS, which however would reduce the performance when your db is large.

I have followed the steps in How To Add Swap on Ubuntu 14.04, since which my mongodb instance never crashes. I am running mongodb with a 1.5GB collection and multiple 512MB collections on a 512MB AWS instance.