Long arrays caching issues

Question

I noticed lately that scanning long arrays with a loop makes the performance of caching decrease. Can you guys please explain why this happens and what are some work arounds of this problem. I'm working on a linux platform with C/C++

Answer 1

This is a classic caching issue: if you loop over a large enough array, every memory reference is to an item that has already been flushed, and therefore requires that the item be fetched from RAM to cache. In the simplest scenario, you need only access an array that is one byte larger than your cache to see this happen.

Even sophisticated, predictive fetching schemes are subject to this issue. No matter what the caching scheme, it is always possible to design a fetching pattern that will result in a cache-validation fault on every memory reference.

The good news is that modern caching systems work hard to reduce the impact of this. For example, the system you are using probably does some forward fetching in your large arrays, so it isn't faulting on every memory access. In the end, taking time to understand caching systems, maybe reviewing your notes from the architecture class you slept through in undergrad :-) will help you work with the caching system.