Skip to content

Update QEMU to 11.1.1 / kernel to 7.2.1, pin virtio PCI slots and harden QEMU - #63

Merged
aledbf merged 2 commits into
mainfrom
chore/qemu-11.1.1-kernel-7.2.1
Aug 27, 2026
Merged

Update QEMU to 11.1.1 / kernel to 7.2.1, pin virtio PCI slots and harden QEMU#63
aledbf merged 2 commits into
mainfrom
chore/qemu-11.1.1-kernel-7.2.1

Conversation

@aledbf

@aledbf aledbf commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

What

  • QEMU 11.0.1 → 11.1.1, kernel 7.1 → 7.2.1.
  • Kernel config committed as the fully resolved 7.2.1 config, with PREEMPT_LAZY=y pinned.
  • Virtio devices are modern-only (disable-legacy=on) and pinned to fixed PCI slots on pcie.0.
  • QEMU seccomp sandbox enabled; S3/S4 dropped from the ACPI tables.

Why the kernel config is regenerated, not carried over

olddefconfig on 7.2 silently changed the preemption model. 7.2 adds depends on !ARCH_HAS_PREEMPT_LAZY to PREEMPT_VOLUNTARY, and x86 sets that symbol, so the config's PREEMPT_VOLUNTARY=y became unselectable and the fallback landed on full PREEMPT=y — more scheduling overhead, the opposite of what the old setting asked for. PREEMPT_LAZY is the intended successor for that trade-off and is now pinned explicitly. PREEMPT_DYNAMIC stays on, so preempt= on the cmdline can still override at runtime.

PCI slot map

Fixed slots on pcie.0 replace QEMU's auto-assignment, so guest enumeration order no longer depends on the order of builder calls:

0x00        host bridge (q35)
0x02        vhost-vsock-pci
0x03        virtio-rng-pci
0x04        virtio-serial-pci (boot profiling only)
0x05-0x0f   virtio-blk-pci   (11 disks)
0x10-0x1e   virtio-net-pci   (15 NICs)
0x1f        ICH9 LPC/SATA/SMBus (q35)

This deliberately avoids PCIe root ports. They would force enumeration of buses 1-4 plus pciehp bring-up and invalidate the pci=lastbus=0 cmdline optimization, to buy PCI hotplug that spinbox does not use — CPU and memory hotplug go through ACPI/QMP. The ranges bound how many disks and NICs fit, checked before building the command line so the failure names the limit instead of surfacing as a QEMU slot collision at exec time.

On disable-legacy=on

Sitting on pcie.0 was not enough on its own — QEMU still realized the devices as transitional. Two identical virtio-blk-pci in one guest, one with the flag:

device id BARs
disable-legacy=on 0x1042 (virtio 1.0) 2× memory
default 0x1001 (transitional) io 128B + 2× memory

Measured neutral for boot time. Two rounds, same kernel and QEMU, only the flag changing:

guest → PID1 exec → vsock accept
modern-only 89.4 ms 163.3 ms
transitional 86.5 ms 162.6 ms

The difference is within run-to-run noise (n=7). The win here is a smaller device surface, not speed; that is recorded in a comment so it is not re-litigated as an optimization.

Not adopted

From the libvirt-style command line this started from: PCIe root ports (above), virtio-scsi (dead weight next to virtio-blk), virtio-balloon (overlaps the pc-dimm memory hotplug accounting), the qemu-guest-agent channel (vminitd over vsock already is the control plane), -no-shutdown (the shim relies on QEMU exiting), and -S / -boot strict / bootindex (we direct-boot the kernel).

-drive-blockdev is left for its own change: it must preserve file.locking=on on the rwlayer or the snapshotter's commit gate stops detecting running containers.

Testing

  • task lint (0 issues), go test -race ./internal/....
  • QEMU 11.1.1 and kernel 7.2.1 build; release tarball builds.
  • Full command line realizes on 11.1.1 — devices land on the intended slots, confirmed via query-pci and in-guest PCI enumeration.
  • VM boots to vminitd with vsock reachable from the host.
  • Integration suite run against an installed build of this branch: passing.

🤖 Generated with Claude Code

aledbf and others added 2 commits August 27, 2026 19:40
Bump the pinned QEMU and kernel versions and carry the guest config
forward.

The kernel config is committed as the fully resolved 7.2.1 config rather
than letting olddefconfig reinterpret the 7.1 one on every build, because
that reinterpretation silently changed the preemption model: 7.2 adds
`depends on !ARCH_HAS_PREEMPT_LAZY` to PREEMPT_VOLUNTARY, which x86 sets,
so the previous PREEMPT_VOLUNTARY=y became unavailable and olddefconfig
fell through to full PREEMPT=y. PREEMPT_LAZY is the intended successor for
that trade-off (full-preempt latency, near-voluntary throughput), so it is
now pinned explicitly. PREEMPT_DYNAMIC stays on, so the model can still be
overridden at runtime with preempt= on the cmdline.

Verified: QEMU 11.1.1 and kernel 7.2.1 build, and a VM boots to vminitd
with vsock reachable from the host.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CcXUA5oay5BiEcFKdhCEht
Take the parts of a libvirt-style command line that fit a microVM, and
leave out the parts that cost boot time or break shim assumptions.

Virtio devices now set disable-legacy=on. Sitting on pcie.0 was not enough
on its own: QEMU still realized them as transitional (device id 0x1001
with a 128-byte legacy I/O BAR); modern-only drops that BAR and the
transitional id (verified via query-pci, and in-guest: the devices come up
as 0x1042/0x1044 with memory BARs only). This was measured neutral for
boot time - the win is a smaller device surface, not speed.

Each device also gets a fixed slot on pcie.0 instead of QEMU's
auto-assignment, so guest enumeration order no longer depends on the order
of builder calls. This deliberately avoids PCIe root ports: they would
force enumeration of buses 1-4 plus pciehp bring-up, and would invalidate
the pci=lastbus=0 cmdline optimization, to buy PCI hotplug that spinbox
does not use (CPU and memory hotplug go through ACPI/QMP). The slot ranges
bound how many disks and NICs fit, checked before building the command
line so the failure names the limit instead of surfacing as a QEMU slot
collision at exec time.

Also enables the seccomp sandbox (the binary is built with
--enable-seccomp) and drops S3/S4 from the ACPI tables, which a microVM
never uses.

Not adopted from that command line: PCIe root ports, virtio-scsi (dead
weight next to virtio-blk), virtio-balloon (overlaps the pc-dimm memory
hotplug accounting), the qemu-guest-agent channel (vminitd over vsock
already is the control plane), -no-shutdown (the shim relies on QEMU
exiting), and -S/-boot strict/bootindex (we direct-boot the kernel). The
-drive to -blockdev migration is left for its own change: it must preserve
file.locking=on on the rwlayer or the snapshotter's commit gate stops
detecting running containers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CcXUA5oay5BiEcFKdhCEht
@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 33.33333% with 18 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/host/vm/qemu/start.go 0.00% 11 Missing ⚠️
internal/host/vm/qemu/qemu_command.go 56.25% 7 Missing ⚠️

📢 Thoughts on this report? Let us know!

@aledbf
aledbf merged commit c1b877b into main Aug 27, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant