.NET4.0 GC does not collect unused space => OutOfMemoryException

Question

We have WinForm application that processes huge amount of map data. We have a cache (a simple Hashtable), where we store frequently accessed map data, to speed up operation. While we do limit the size of the cache, Task Manager shows an ever-growing memory consumption, and finally we get an OutOfMemoryException. The strange thing is that both the memory profiler and GC.GetTotalMemory and our own object size calculations say the same thing: our cache never uses more memory than our limit set. If we disable the cache, our system just works fine: during processing the map data, it peaks for a few seconds but after that it drops back to normal.

The memory profiler shows nicely that the used space (indicated with green) corresponds exactly to our cache size, but the majority of the reserved memory is not in use.
During the profiling session, we took 4 memory snapshots:

• Memory snapshot1: cache on (max cache size 300MB)
• Memory snapshot2: cache off
• Memory snapshot3: cache on again (max cache size 300MB)
• Memory snapshot4: cache off again

Watch the color-coding below. Why those huge unused spaces (blue) are not claimed back? Calling GC.Collect makes no difference.

Answer 1

The root problem was memory fragmentation, as Hans indicated in his comment. During the creation of the our cached object (rather big ones), we had a many mid sized transient objects creating gaps among the big permanent objects. As soon as we reduced the amount of transient objects by reuse, the fragmentation dropped radically.

As you can see on the graphs, the ratio of used : unused space was 1:2. After the modification the ratio is 8:1.