WiredTiger internal cache

Question

It is stated in the MongoDB documentation FAQ (https://docs.mongodb.com/manual/faq/storage/#wiredtiger-storage-engine) :

Avoid increasing the WiredTiger internal cache size above its default value.

which should be 50 % ou 60 % of RAM by default

and

The storage.wiredTiger.engineConfig.cacheSizeGB limits the size of the WiredTiger internal cache. The operating system will use the available free memory for filesystem cache, which allows the compressed MongoDB data files to stay in memory.

I don't understand this statement. Does the WT cache contains the working set in terms of compressed data and index ? If so, do we need to keep extra memory for some kind of memory mapped files for MongoDB data files ?

My db.serverStatus().mem shows this :

{
    "bits" : 64,
    "resident" : 10518,
    "virtual" : 12131,
    "supported" : true,
    "mapped" : 0,
    "mappedWithJournal" : 0
}

So I would guess I've no memory mapped bytes, then I can increase de WT cache ?

Thanks !

Answer 1

Do we need to keep extra memory for some kind of memory mapped files for MongoDB data files

The mem.mapped and mem.mappedWithJournal metrics in serverStatus output are only applicable to the MMAPv1 storage engine, so are expected to be 0 with WiredTiger. These metrics predate the storage engine API (and WiredTiger) so will likely be moved to a storage-engine specific section of serverStatus at some point.

Does the WT cache contains the working set in terms of compressed data and index ?

The WiredTiger internal cache is used for uncompressed data which is a different representation from the compressed data format on disk. Memory outside the WiredTiger cache is available for filesystem cache (which caches the on-disk representation, including compression) as well as other temporary RAM requirements such as per-connection overhead (1MB per connection), in-memory sorts, aggregations, and JavaScript contexts.

Your working set is the total set of data and indexes that your application frequently accesses. For best performance this should ideally fit into RAM. A larger WiredTiger cache will allow for more uncompressed data in RAM at the expense of memory available for other purposes. If your data compresses significantly, more free memory for the filesystem cache will allow for a larger total working set in RAM (uncompressed + compressed data).

So I would guess I've no memory mapped bytes, then I can increase de WT cache ?

As per the MongoDB documentation, you should generally leave the WiredTiger cache at the default size (or possibly even decrease this). You can always try to experiment with tuning the cache size for your workload and deployment, but setting this too large may adversely affect performance.

Answer 2

Answer is no! First, because you are watching wrong information. Because you are using wiredTiger, you should look db.serverStatus().wiredTiger.cache

Lot's of information but

"bytes currently in the cache" : 88404

Tell you current situation

"maximum bytes configured" : 19327352832

Maximum cache size (50% of memory)

"bytes read into cache" : 43999,
"bytes written from cache" : 88126

How much data has been read from disk (and wrote back) since startup

MongoDB also needs additional memory outside of this cache for aggregations, sorting, connection management, and the like, so it is important to make sure you leave MongoDB with enough memory to do its work. If not, there is a chance MongoDB can get killed by the OS Out of memory (OOM) killer.