Memory-mapped files remain in physical memory?

Question

I have a process that uses a lot of memory mapped files.
Problem is that those files are kept in physical memory, even when the machine is low on memory, and other processes require this memory.

I've tried using SetProcessWorkingSetSize to limit the process working set, but it doesn't help, the process' working set keeps growing over the max value.

Is there a better way to limit the process' working set?
Can I change Windows' heuristcs for paging memory mapped files?

Answer 1

Ended up using brute force VirtualUnlock.

PROCESS_MEMORY_COUNTERS pmc;
if (GetProcessMemoryInfo( hProcess, &pmc, sizeof(pmc)) )
{
        if (pmc.WorkingSetSize > MaxWorkingSetSize)
        {
                VirtualUnlock(FilePtr.pData, MaxWorkingSetSize);
                UnmapViewOfFile(FilePtr.pData);
                CloseHandle(FilePtr.hFileMap);
                CloseHandle(FilePtr.hFile);
        }
}

Answer 2

If you find that your process with the memory mapped files is holding on to a lot of those pages, then that means the OS has not needed to discard any of your memory mapped regions to give to other processes. So, how do you know other processes actually need the memory currently used for the mapped files? Just because the OS is low on physical RAM means nothing. Other processes have to demand memory to cause the OS to drop your mapped pages and give RAM to them.

So, it appears that your mmap-I/O process is starving your other process which uses RAM less frequently. One approach is to judiciously lock memory in the process that is being starved. Look at VirtualLock for win32.

Answer 3

Check my answer here; with VirtualUnlock() you can manually uncommit portions of the MMFs; e.g., sections you don't think you'll be accessing again soon.

Answer 4

I think this behaviour is due to how MMF (Memory Mapped Files) work. Have a look at this blog article. It explains that MMF files skip the Windows paging process, and thus are not backed by the page file. Instead, the MMF becomes the data backup in itself, which means, ultimately, it takes up more RAM since it's not being pages out (Er, I'm not sure I get that myself - best to read the blog!)

Here's the MSDN documentation for memory mapped files and here's another SO question relating to MMF.