C++ cannot rewrite shared memory in my windows app. It allocates new memory

Question

Shared memory in C++ is new thing for me, still learning, and now I found that it doesn't work how I expected. I want to implement class called CharBuffer which has char array mapped to shared memory by CreateFileMapping and MapViewOfFile functions:

HANDLE hBuffer = CreateFileMapping(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE, 0, size, shmBufferName); 
char * buffer = (char *) MapViewOfFile(hBuffer, FILE_MAP_ALL_ACCESS, 0, 0, size);

Adding bytes to buffer is implemented by calling CharBuffer function "add":

void CharBuffer::add(char * data, size_t size) {
    memcpy(&buffer[0], &data[0], size);
}

There is no check for data size but this is no problem, it is just my test app for learning with shared memory. This is OK, I can add bytes to buffer or read from buffer successfully - no error, no problem. This I am doing with first instance of CharBuffer.

But now I'd like to open shared memory with second instance of CharBuffer and read or rewrite the data in shared memory. For open and map memory I use:

HANDLE hBuffer = OpenFileMapping(FILE_MAP_ALL_ACCESS, FALSE, shmBufferName);
char * buffer = (char *) MapViewOfFile(hBuffer, FILE_MAP_ALL_ACCESS, 0, 0, size);

There is no error during opening. Now I am getting to my problem. In first instance I copied 64 MB to shared memory. When I try read them byte-by-byte with second instance which has correctly mapped shared memory (it prints data OK) but in Resource monitor I can see I have two times more memory - 128 MB. After first adding data the memory took 64 MB but during printing data via second instance the memory is increased by next 64 memory.

The same problem is writing data to memory with second instance.

My goal is to have limited shared memory and work with it (read/write) by two or more instances of CharBuffer. Can you please explain me where could be the problem or how I can implement what I want?

Thank you a lot.

Answer 1

I have never used memory mapped files to lower memory consumption. I think that your problem is that MapViewOfFile always returns pointer to unique address space, which will modify the same pages in you page file. I tried to find prove of that in MSDN, all I found is this article:

http://msdn.microsoft.com/en-us/library/ms810613.aspx

which I suppose explains it:

As mentioned above, you can have multiple views of the same memory-mapped file, and they can overlap. But what about mapping two identical views of the same memory-mapped file? After learning how to unmap a view of a file, you could come to the conclusion that it would not be possible to have two identical views in a single process because their base address would be the same, and you wouldn't be able to distinguish between them. This is not true. Remember that the base address returned by either the MapViewOfFile or the MapViewOfFileEx function is not the base address of the file view. Rather, it is the base address in your process where the view begins. So mapping two identical views of the same memory-mapped file will produce two views having different base addresses, but nonetheless identical views of the same portion of the memory-mapped file.

The point of this little exercise is to emphasize that every view of a single memory-mapped file object is always mapped to a unique range of addresses in the process. The base address will be different for each view. For that reason the base address of a mapped view is all that is required to unmap the view.

so MapViewOfFile returns addresses to memory that once committed will have assigned its own physical memory, which actually increases your process memory consumption - as you have described.

[edit]

actually, I am starting to see that even thought both MapViewOffFile returns addresses to unique virtual address spaces, they are both backed by the same physical RAM pages. You can read on this in Windows Via c/c++ in Memory-Mapped Files and Coherence:

If multiple processes are mapping views of a single data file, the data is still coherent because there is still only one instance of each page of RAM within the data file—it's just that the pages of RAM are mapped into multiple process address spaces.

this is also proved by the trick in following blog:

http://blogs.msdn.com/b/oldnewthing/archive/2003/10/07/55194.aspx

the fact that resource monitor shows you memory consumption too high each time you access your view of file, is that it shows working set size which to my understanding might show shared memory twice.