Reducing Cache Misses

Reducing Cache Misses

When performance requirements are high, you become concerned, not with the loss of milliseconds to a page fault, but with the loss of microseconds to a cache miss. When your program accesses instructions or data that are not in cache memory, the CPU requests a load of a "cache line" of 128 bytes from main memory. Possibly hundreds of CPU clock cycles pass while the cache is being loaded. Due to the pipeline architecture of the CPU, it can often continue to work during this delay. However, multiple successive cache misses can bring effective work to a halt for tens of microseconds.

Locality of Reference
Cache Mapping in Challenge and Onyx Systems
Multiprocessor Cache Conflicts
Detecting Cache Problems