How does cache acess time limit the maximum frequency of a core?

Question

Imagine we have a single core chip with a L1 cache. How does the access time of the L1 cache limit the maximum frequency that can be achieved with the core?

Is there some kind of formula that can be used to calculate the frequency of a core, taking the access time in account? So for example if the access time is twice is high, would the frequency be twice as low?

So far I've read that the access time is 1/frequency, is this correct?

Thanks in advance.

Answer 1

Well, that depends on how many stages of the pipeline are allocated for memory access. In the older 5-stage MIPS architectures fetch was 1 cycle; on modern Intel 14-stage pipelines it's 3 cycles; on some ARMs it's 2 cycles, etc. Basically for N-cycle fetch the maximal frequency would be N/(L1 latency)

Answer 2

My (non-formula) answer would be the following:

• cpu -> L1 cache (data and instruction) -> Memory

If CPU is capable of operating at a much faster speed than the speed of L1 cache getting instructions and/or data from memory, then the total pipeline speed will be limited by the lowest speed i.e. cpu will operate at the lower speed of the L1 cache.

To prevent this problem, we have many technologies like cache pre-fetching, memory banks for parallel memory access, etc etc. to keep the cpu operating at peak speed.

Answer 3

What kind of chip, what is the micro architecture?

I don't see any direct connection - if you take longer time to access the cache than a single cycle, you could just introduce a bubble after a load/store if you're in-order, or do nothing if you're an out-of-order CPU (since the consumers of the load/store can wait for the data to return while the rest of the instructions are progressing).

You could say that on an in-order cpu it's a better trade-off to stretch the cycle duration (lower freq), but that's a design question. What if your average code hardly has any memory ops? (i.e. you're optimizing for compute intensive code)