</>EmbeddedInterview

How does an RTOS work on a multi-core (SMP) processor?

HardRTOS & Real-Time Systems

Short answer: In SMP (symmetric multiprocessing) mode, one RTOS kernel schedules tasks across several cores sharing memory: on each core it runs the highest-priority ready task that is allowed to run there. This adds new concerns: kernel data structures need spinlocks instead of only disabling interrupts, tasks of different priorities can run at the same time, core affinity can pin tasks to cores, and caches and memory ordering must be handled. In AMP (asymmetric) mode, each core runs its own independent OS or bare-metal code and they communicate through shared memory and inter-core interrupts.

SMP (FreeRTOS SMP since v11, Zephyr SMP, ThreadX SMP)

  • Locks: a task-level critical section on one core does not stop the other core, so the kernel uses spinlocks (with interrupt masking) for its data.
  • Priority assumptions break: in single-core code, "a high-priority task never runs at the same time as a low-priority task" was often relied on to avoid locks. On SMP both can run in parallel, so shared data needs real protection.
  • Affinity: vTaskCoreAffinitySet() (FreeRTOS) or k_thread_cpu_mask_* (Zephyr) restricts tasks to cores, e.g. dedicate one core to a time-critical control loop.
  • Inter-processor interrupts: used to ask another core to reschedule.
  • Caches: cores need coherent caches (or uncached shared memory) and atomics with proper barriers.

AMP (common on heterogeneous SoCs)

  • e.g. Cortex-M7 + Cortex-M4 in STM32H7 dual-core, Cortex-A + Cortex-M in i.MX 8M / STM32MP1, or two Cortex-R cores in lock-step pairs.
  • Each core runs its own RTOS (or Linux on one, RTOS on the other).
  • Communication via OpenAMP/RPMsg (shared-memory ring buffers + mailbox interrupts), hardware semaphores (HSEM) for shared peripherals.
  • Simpler isolation and certification, but load cannot be balanced automatically.

Lock-step (Cortex-R5/R52 in safety MCUs) is different again: two cores execute the same code in parallel and hardware compares results to detect faults; it is one logical CPU for software.