What is a fork bomb, and how do ulimit and cgroups protect against it?
Short answer: A fork bomb is a program that creates processes as fast as it can, each of which creates more (:(){ :|:& };: in bash). The process table, PIDs, memory and CPU are exhausted within seconds, so even administrators cannot start a shell to stop it. It is a denial-of-service attack on the OS. Limits on the number of processes per user (ulimit -u / RLIMIT_NPROC) or per group of processes (the cgroup pids.max controller) stop it, because further fork() calls fail with EAGAIN.
Why the whole system becomes unusable
- Every process needs a PID, a
task_struct, a kernel stack (16 KB on ARM64) and page tables. The PID space (/proc/sys/kernel/pid_max) and memory run out. - The scheduler has thousands of runnable tasks, so each gets a tiny share; even the shell you use to fight back barely runs.
- New logins fail because they also need
fork().
Protections
ulimit -u 500 # max processes for this user (soft limit), in this shell and children
cat /etc/security/limits.conf # persistent per-user limits via PAM: @students hard nproc 200
echo 200 > /sys/fs/cgroup/myservice/pids.max # cgroup v2: max tasks in this group
RLIMIT_NPROCcounts processes per real user ID; root is exempt (unless it dropsCAP_SYS_RESOURCE/CAP_SYS_ADMIN).- systemd sets
TasksMax=for every service and for user sessions (via thepidscontroller), so a fork bomb inside one service or login session cannot take down the whole system. kernel.threads-maxandpid_maxare global ceilings.
Recovering from one: if you still have a shell, kill -STOP all processes of that user first (stopped processes cannot fork), then kill -KILL them. Killing one by one does not work, because the survivors keep forking.
Embedded relevance: a buggy service that forks in a loop (for example, retrying a helper program that crashes immediately) behaves like a fork bomb. Setting TasksMax= and restart limits (StartLimitBurst=) in the service units prevents it from taking down the device.