</>EmbeddedInterview

What happens in QEMU when you start a virtual machine? Explain the code flow.

HardVirtualization (QEMU & KVM)

Short answer: main() calls qemu_init(), which parses the command line, selects the accelerator (TCG or KVM), creates the machine object, and runs the machine's init function (e.g. machvirt_init() for -M virt on ARM, pc_init1()/pc_q35_init() on x86). That function creates the CPUs, RAM, interrupt controller and devices, generates the device tree or ACPI tables and loads the kernel/firmware. Then qemu_main_loop() runs the event loop while separate vCPU threads execute guest code (with TCG's cpu_exec() or KVM's kvm_cpu_exec()).

Main thread (paths from recent QEMU)

system/main.c       main()
system/vl.c           qemu_init(argc, argv)
                        parse options, qemu_create_machine()
                        configure_accelerators()      -> kvm_init() or tcg init
                        qmp_x_exit_preconfig()
                          qemu_init_board()
                            machine_run_board_init()
                              MachineClass->init()    -> machvirt_init()  (hw/arm/virt.c)
                                                      -> pc_q35_init()    (hw/i386/pc_q35.c)
                          qemu_machine_creation_done()
system/runstate.c     qemu_main_loop()                -> main_loop_wait(): timers, fd/IO events, BHs

What the board init does (ARM virt example)

  1. Create the vCPUs from the -cpu model (e.g. cortex-a53, defined with an init function in target/arm/cpu64.c) and realize them: this sets the ID registers and features, and with KVM creates the KVM vCPU.
  2. Create RAM (memory_region_init_ram or the machine's memory backend) and map it at the board's RAM base (0x40000000 on virt).
  3. Create the GIC, UARTs (PL011), RTC, GPIO, virtio-mmio transports, PCIe host bridge, flash for firmware, fw_cfg.
  4. Wire up interrupts (qdev_connect_gpio_out / sysbus_connect_irq).
  5. Generate the device tree describing all of this (or ACPI tables for UEFI boots).
  6. Set up the boot: load -kernel, -initrd, -dtb, or firmware (-bios), and set the CPU start state (PC and x0 = DTB address).

vCPU threads

  • TCG: mttcg_cpu_thread_fn() (one thread per vCPU when multi-threaded TCG is enabled) loops on tcg_cpu_exec() → cpu_exec().
  • KVM: kvm_vcpu_thread_fn() loops on kvm_cpu_exec() → ioctl(KVM_RUN). (Older QEMU called it qemu_kvm_cpu_thread_fn.)

The big QEMU lock (BQL): device emulation runs under a global mutex. A vCPU that exits to handle MMIO takes the BQL, calls the device model, releases it and goes back to running the guest.

x86 pc machine: same structure, but the board also loads SeaBIOS/OVMF into the top of the 4 GB space with rom_load_file(), the CPU starts in 16-bit real mode at the reset vector, and the firmware then loads the bootloader.