</>EmbeddedInterview

What is PREEMPT_RT and what changes for drivers?

HardLinux Kernel & Device Drivers

Short answer: PREEMPT_RT (merged into mainline in 6.12, CONFIG_PREEMPT_RT) turns Linux into a real-time OS with bounded latencies of typically tens of microseconds. It makes almost all kernel code pre-emptible: interrupt handlers run in threads with configurable priorities, spinlock_t and rwlock_t become sleeping rt_mutexes with priority inheritance, softirqs run in thread context, and long non-pre-emptible sections are broken up. Drivers must not rely on "spinlock means interrupts and pre-emption are off".

What changes

Standard kernel PREEMPT_RT
Hard IRQ handlers run in interrupt context forced into threads (irq/NN-name, SCHED_FIFO 50) unless IRQF_NO_THREAD
spinlock_t spins, disables pre-emption sleeping lock (rt_mutex) with priority inheritance
raw_spinlock_t spins still spins: only for very short, truly atomic sections (IRQ chips, scheduler)
local_irq_disable() around data common (bad) practice breaks: use proper locks (local_lock)
Softirqs after IRQs, or ksoftirqd in thread context, per-CPU, pre-emptible
Memory allocation under spinlock allowed with GFP_ATOMIC spinlocks can sleep, so allocations in raw-spinlock or hard-IRQ sections are forbidden

Driver guidelines for RT

  • Use threaded IRQs and normal spinlock_t; use raw_spinlock_t only where required (interrupt controller, clock event code) and keep those sections tiny.
  • Do not disable interrupts or pre-emption to protect data; use locks or local_lock_t for per-CPU data.
  • Avoid unbounded loops with locks held.
  • Mark interrupt handlers that must stay in hard IRQ context with IRQF_NO_THREAD (timers, IPIs) and keep them minimal.

Tuning a real-time system

  1. RT application threads use SCHED_FIFO with priorities above the IRQ threads they don't depend on; set the relevant IRQ thread priorities with chrt.
  2. mlockall(MCL_CURRENT | MCL_FUTURE) and pre-fault stacks/heap to avoid page faults.
  3. Isolate CPUs (isolcpus=, nohz_full=, rcu_nocbs=, irqaffinity=) for critical threads.
  4. Disable deep CPU idle states and frequency scaling for lowest latency, or accept the trade-off.
  5. Measure with cyclictest under load (stress-ng, hackbench), and find latency sources with ftrace (osnoise, timerlat tracers, rtla).

Limits: RT Linux gives bounded latency for most workloads but is not formally certified for hard safety-critical use; those systems pair Linux with an RTOS on another core or a hypervisor.