Reducing Instruction Cache misses (in C++)

Question

Let's say I have a C++ class whose implementation looks something like this:

// ...

MyClass::iterativeFunction() {
     for (int i = 0; i < 1000000; i++) {
          performAction(i);
     }
}

MyClass::performAction(int index) {
     // Block of code (non-inline-able)
}

// ...

At the C++ level, do I have any control over the spacial locality of these methods, or do I just have to hope the compiler will notice the related methods and optimise its assembly accordingly? Ideally I would like them to be right next to each other so they will be loaded into the instruction cache together, but I have no idea how to let the compiler know I'd really like this to happen.

Answer 1

In either case, the code can't run until it gets into the cache. In either case, it will be equally obvious to the CPU where the code flow goes because the flow is unconditional. So it won't make any difference. A modern code cache doesn't fetch ahead in address space, it fetches ahead in the instruction flow, following unconditional branches and predicting conditional branches as necessary.

So there is no reason to care about this. It won't make any difference.

Answer 2

Technically speaking, no. However, on modern-ish processors you don't generally need to worry about the instruction cache nearly as much as you do the data cache, unless you have a very large executable or really horrific branches everywhere.

The reasons is that the cache lines are only around 64 bytes long, which means that if your methods are larger than 64 bytes (and they are), they will need to be loaded into multiple cache entries even if they are directly next to each other in physical memory.

Answer 3

If you need that level of control and optimization then C++ is the wrong language for you.

But the actual answer to your question is "No".

Answer 4

No, as far as I know there is no way for you to specify the location of your methods. If C++ allowed nested procedures that would be one way to ensure that the called procedure was local.