Is there really no mremap in Darwin?

Question

I'm trying to find out how to remap memory-mapped files on a Mac (when I want to expand the available space).

I see our friends in the Linux world have mremap but I can find no such function in the headers on my Mac. /Developer/SDKs/MacOSX10.6.sdk/usr/include/sys/mman.h has the following:

• mmap
• mprotect
• msync
• munlock
• munmap
• but no mremap

man mremap confirms my fears.

I'm currently having to munmap and mmmap if I want to resize the size of the mapped file, which involves invalidating all the loaded pages. There must be a better way. Surely?

I'm trying to write code that will work on Mac OS X and Linux. I could settle for a macro to use the best function in each case if I had to but I'd rather do it properly.

Answer 1

You can ftruncate the file to a large size (creating a hole) and mmap all of it. If the file is persistent I recommend filling the hole with write calls rather than by writing in the mapping, as otherwise the file's blocks may get unnecessarily fragmented on the disk.

Answer 2

If you need to shrink the map, just munmap the part at the end you want to remove.

If you need to enlarge the map, you can mmap the proper offset with MAP_FIXED to the addresses just above the old map, but you need to be careful that you don't map over something else that's already there...

The above text under strikeout is an awful idea; MAP_FIXED is fundamentally wrong unless you already know what's at the target address and want to atomically replace it. If you are trying to opportunistically map something new if the address range is free, you need to use mmap with a requested address but without MAP_FIXED and see if it succeeds and gives you the requested address; if it succeeds but with a different address you'll want to unmap the new mapping you just created and assume that allocation at the requested address was not possible.

Answer 3

If you expand in large enough chunks (say, 64 MB, but it depends on how fast it grows) then the cost of invalidating the old map is negligible. As always, benchmark before assuming a problem.

Answer 4

I have no experience with memory mapping, but it looks like you can temporarily map the same file twice as a means to expand the mapping without losing anything.

int main() {
    int fd;
    char *fp, *fp2, *pen;

      /* create 1K file */
    fd = open( "mmap_data.txt", O_RDWR | O_CREAT, 0777 );
    lseek( fd, 1000, SEEK_SET );
    write( fd, "a", 1 );

      /* map and populate it */
    fp = mmap( NULL, 1000, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0 );
    pen = memset( fp, 'x', 1000 );

      /* expand to 8K and establish overlapping mapping */
    lseek( fd, 8000, SEEK_SET );
    write( fd, "b", 1 );
    fp2 = mmap( NULL, 7000, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0 );

      /* demonstrate that mappings alias */
    *fp = 'z';
    printf( "%c ", *fp2 );

      /* eliminate first mapping */
    munmap( fp, 1000 );

      /* populate second mapping */
    pen = memset( fp2+10, 'y', 7000 );

      /* wrap up */
    munmap( fp2, 7000 );
    close( fd );
    printf( "%d\n", errno );
}

The output is zxxxxxxxxxyyyyyy.....

I suppose, if you pound on this, it may be possible to run out of address space faster than with mremap. But nothing is guaranteed either way and it might on the other hand be just as safe.