What is a CPU cache, and how do you write cache-friendly code?
Short answer: A cache is a small, fast memory close to the CPU that keeps copies of recently used memory in fixed-size blocks called cache lines (32 bytes on Cortex-M7, 64 bytes on most Cortex-A). Accesses that hit in the cache take a few cycles; misses go to slower memory (tens to hundreds of cycles). Code is cache-friendly when it uses data that is close together (spatial locality) and reuses data soon (temporal locality), and when hot data structures are small and laid out in the order they are accessed.
Key terms
| Term | Meaning |
|---|---|
| Line | unit of transfer (e.g. 64 bytes) |
| Set associativity | each address can go in N places (ways) within its set; 1-way = direct-mapped |
| Hit / miss | data found / not found |
| Write-back vs write-through | write-back updates memory only on eviction (faster); write-through writes memory every time |
| Write-allocate | on a write miss, load the line into the cache |
| Compulsory, capacity, conflict misses | first use; working set too big; addresses competing for the same set |
| Coherency | keeping caches of several cores (and DMA) consistent |
Example: loop order over a 2-D array
#define N 1024
static float m[N][N];
float sum_rows(void) /* row-major traversal: consecutive addresses, ~1 miss per 16 floats */
{
float s = 0;
for (int i = 0; i < N; i++)
for (int j = 0; j < N; j++)
s += m[i][j];
return s;
}
float sum_cols(void) /* column-major: each access 4 KB apart, a miss almost every time */
{
float s = 0;
for (int j = 0; j < N; j++)
for (int i = 0; i < N; i++)
s += m[i][j];
return s;
}
The second version can be several times slower with exactly the same arithmetic.
Techniques
- Traverse arrays in memory order; prefer arrays to linked lists for hot data.
- Structure of arrays vs array of structures: if a loop uses one field of many objects, store that field contiguously.
- Keep hot fields together and cold fields elsewhere; align heavily shared structures to cache lines.
- Blocking/tiling large computations (matrix multiply, image filters) so each block fits in the cache.
- Avoid false sharing on multi-core systems: two cores writing different variables in the same line make the line bounce between caches; pad per-core data to a full line.
- Prefetch (
__builtin_prefetch) for predictable access patterns the hardware prefetcher misses.
MCU specifics: Cortex-M0–M4 usually have no data cache (but vendors add flash accelerators such as ST's ART); Cortex-M7 and M55/M85 have L1 caches, which must be enabled (SCB_EnableICache(), SCB_EnableDCache()) and handled carefully with DMA. TCM (tightly coupled memory) gives single-cycle, deterministic access without cache unpredictability, which is often better for real-time code.