How do Arm systems keep DMA and CPU caches coherent (ACP, I/O coherency)?
Short answer: If a DMA master accesses main memory directly while the CPUs work through write-back caches, the two can see different data. There are two solutions. Software coherency: the driver cleans and invalidates cache lines around each transfer (what Cortex-M7 firmware and many Linux drivers do via the DMA API). Hardware (I/O) coherency: the DMA master is connected so that its accesses are checked against the CPU caches, either through the Accelerator Coherency Port (ACP) on the CPU cluster, or through a coherent interconnect (CCI, CCN, CMN) using the ACE-Lite protocol.
ACP (Accelerator Coherency Port)
- An AXI slave port on clusters such as Cortex-A53/A9/A72 that lets an external master (DMA engine, accelerator, FPGA logic) read and write memory through the cluster's coherency logic (SCU and L2).
- A read via ACP returns the latest data even if it is dirty in a CPU's L1; a write via ACP updates or invalidates the cached copies.
- Accesses may also allocate into the L2, so an accelerator and the CPUs can share data with low latency (useful for small, frequent transfers).
- Limitations: limited bandwidth (it shares the cluster's ports), only certain access sizes and types, and heavy use pollutes the L2.
Coherent interconnects: in larger SoCs, the CCI/CMN snoops the CPU caches for I/O masters that use ACE-Lite, so DMA from a network controller or GPU is coherent without software maintenance. Linux learns this from the device tree property dma-coherent, and the DMA API then skips cache maintenance.
Choosing between them
| Software maintenance | ACP / coherent interconnect | |
|---|---|---|
| Hardware cost | none | needs the port/interconnect |
| CPU overhead | clean/invalidate per buffer (expensive for many small buffers) | none |
| Bugs | easy to forget a maintenance call or misalign a buffer | fewer software bugs |
| Large streaming transfers | fine | may thrash the L2 |
Interview follow-up: "What happens if a buffer is not cache-line aligned in the software approach?" Invalidating the buffer also invalidates neighbouring data in the same line, losing recent CPU writes, which is why DMA buffers must be aligned and padded to whole lines.