C++ Memory mapped file data prefetching

Question

I'm dealing with very large memory mapped files (200+ GB) which can not be loaded fully into memory, and are accessed randomly. The mapped files are stored on a solid state drive array, but I still find that accessing the needed parts one at a time is very slow unless the same search has been done previously, and the pages used have already been read into memory.

Adding multiple threads to read the variables in the mmap simultaneously improves the speed dramatically, and I was unable to reach an upper bound to the improvement in my testing, but having more than 1000 threads causes openmp to throw resource unavailable errors.

I have also tried madvise to advise the kernel of the specific parts which will be needed (MADV_WILLNEED) but the kernel does not seem to act on the advise quick enough to make a difference.

I'm looking for a way to simultaneously prefetch the parts of the data needed immediately prior to them actually being used. What would be the least resource intensive way to read a variable (or the memory page sized piece of the mapped file containing it) forcing it into memory without blocking on the read.

If blocking can not be avoided, a way to run a much larger number of very light weight threads to do the reading would also work.

Answer 1

You seem to have answered your own question. Your only solution besides threading is to loop through however many accesses you can doing madvise for each. Then after some x number of madvises (say 10,000) you come back and access the memory. It should be noted however that the O/S does NOT guarantee that the I/O will be done in the order that madvise is called. Therefore, the O/S might process the first madvise, then jump to the end of the madvises, or the one with lowest address, basically whatever it pleases. There simply is no way to considerably speed up I/O to the extent it sounds like you want.

Example:

for(i=0; i < accesses + 10000; ++i)
{
    madvise(access[i].addr, access[i].length, MADV_WILLNEED);
    if(i >= 10000)
    {
        // Access location access[i-10000].addr
    }
}

You should however ask yourself if memory mapping this file is really what you want to do if you're using random access. It would seem that asynchronous I/O would make better sense.