Zero a large memory mapping with `madvise`

Question

I have the following problem:

I allocate a large chunk of memory (multiple GiB) via mmap with MAP_ANONYMOUS. That chunk holds a large hash map which needs to be zeroed every now and then. Not the entire mapping may be used in each round (not every page is faulted in), so memset is not a good idea - takes too long.

What is the best strategy to do this quickly?

Will

madvise(ptr, length, MADV_DONTNEED);

guarantee me that any subsequent accesses provide new empty pages?

From the Linux man madvise page:

This call does not influence the semantics of the application (except in the case of MADV_DONTNEED), but may influence its performance. The kernel is free to ignore the advice.

...

MADV_DONTNEED

Subsequent accesses of pages in this range will succeed, but will result either in reloading of the memory contents from the underlying mapped file (see mmap(2)) or zero-fill-on-demand pages for mappings without an underlying file.

...

The current Linux implementation (2.4.0) views this system call more as a command than as advice ...

Or do I have to munmap and remap the region anew?

It has to work on Linux and ideally have the same behaviour on OS X.

Answer 1

There is a much easier solution to your problem that is fairly portable:

mmap(ptr, length, PROT_READ|PROT_WRITE, MAP_FIXED|MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);

Since MAP_FIXED is permitted to fail for fairly arbitrary implementation-specific reasons, falling back to memset if it returns MAP_FAILED would be advisable.

Answer 2

On Linux, you can rely on MADV_DONTNEED on an anonymous mapping zeroing the mapping. This isn't portable, though - madvise() itself isn't standardised. posix_madvise() is standardised, but the POSIX_MADV_DONTNEED does not have the same behaviour as the Linux MADV_DONTNEED flag - posix_madvise() is always advisory, and does not affect the semantics of the application.

Answer 3

This madvise behavior is certainly not standard, so this wouldn't be portable.

If the part that you want to zero out happens to be at the end of your mapping you could get away with ftruncate. You'd have to introduce one step more:

1. shm_open to have a "persistent" file descriptor to your data
2. ftruncate to the needed size
3. mmap of that FD

Then you could always

1. munmap
2. ftruncate to something short
3. ftruncate to the real length you need
4. mmap again

and then the part that you "remapped" would be zero initialized.

But have also in mind that the system has to do the zeroing of the pages. This might be a bit more efficient than the inline stuff that your compiler produces for memset, but that is not sure.