</>EmbeddedInterview

What is a CPU cache, and how do you write cache-friendly code?

MediumMicrocontrollers & ARM Cortex-M

Short answer: A cache is a small, fast memory close to the CPU that keeps copies of recently used memory in fixed-size blocks called cache lines (32 bytes on Cortex-M7, 64 bytes on most Cortex-A). Accesses that hit in the cache take a few cycles; misses go to slower memory (tens to hundreds of cycles). Code is cache-friendly when it uses data that is close together (spatial locality) and reuses data soon (temporal locality), and when hot data structures are small and laid out in the order they are accessed.

Key terms

Term Meaning
Line unit of transfer (e.g. 64 bytes)
Set associativity each address can go in N places (ways) within its set; 1-way = direct-mapped
Hit / miss data found / not found
Write-back vs write-through write-back updates memory only on eviction (faster); write-through writes memory every time
Write-allocate on a write miss, load the line into the cache
Compulsory, capacity, conflict misses first use; working set too big; addresses competing for the same set
Coherency keeping caches of several cores (and DMA) consistent

Example: loop order over a 2-D array

#define N 1024
static float m[N][N];

float sum_rows(void)            /* row-major traversal: consecutive addresses, ~1 miss per 16 floats */
{
    float s = 0;
    for (int i = 0; i < N; i++)
        for (int j = 0; j < N; j++)
            s += m[i][j];
    return s;
}

float sum_cols(void)            /* column-major: each access 4 KB apart, a miss almost every time */
{
    float s = 0;
    for (int j = 0; j < N; j++)
        for (int i = 0; i < N; i++)
            s += m[i][j];
    return s;
}

The second version can be several times slower with exactly the same arithmetic.

Techniques

  • Traverse arrays in memory order; prefer arrays to linked lists for hot data.
  • Structure of arrays vs array of structures: if a loop uses one field of many objects, store that field contiguously.
  • Keep hot fields together and cold fields elsewhere; align heavily shared structures to cache lines.
  • Blocking/tiling large computations (matrix multiply, image filters) so each block fits in the cache.
  • Avoid false sharing on multi-core systems: two cores writing different variables in the same line make the line bounce between caches; pad per-core data to a full line.
  • Prefetch (__builtin_prefetch) for predictable access patterns the hardware prefetcher misses.

MCU specifics: Cortex-M0–M4 usually have no data cache (but vendors add flash accelerators such as ST's ART); Cortex-M7 and M55/M85 have L1 caches, which must be enabled (SCB_EnableICache(), SCB_EnableDCache()) and handled carefully with DMA. TCM (tightly coupled memory) gives single-cycle, deterministic access without cache unpredictability, which is often better for real-time code.