Update QEMU to 11.1.1 / kernel to 7.2.1, pin virtio PCI slots and harden QEMU - #63
Merged
Conversation
Bump the pinned QEMU and kernel versions and carry the guest config forward. The kernel config is committed as the fully resolved 7.2.1 config rather than letting olddefconfig reinterpret the 7.1 one on every build, because that reinterpretation silently changed the preemption model: 7.2 adds `depends on !ARCH_HAS_PREEMPT_LAZY` to PREEMPT_VOLUNTARY, which x86 sets, so the previous PREEMPT_VOLUNTARY=y became unavailable and olddefconfig fell through to full PREEMPT=y. PREEMPT_LAZY is the intended successor for that trade-off (full-preempt latency, near-voluntary throughput), so it is now pinned explicitly. PREEMPT_DYNAMIC stays on, so the model can still be overridden at runtime with preempt= on the cmdline. Verified: QEMU 11.1.1 and kernel 7.2.1 build, and a VM boots to vminitd with vsock reachable from the host. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CcXUA5oay5BiEcFKdhCEht
Take the parts of a libvirt-style command line that fit a microVM, and leave out the parts that cost boot time or break shim assumptions. Virtio devices now set disable-legacy=on. Sitting on pcie.0 was not enough on its own: QEMU still realized them as transitional (device id 0x1001 with a 128-byte legacy I/O BAR); modern-only drops that BAR and the transitional id (verified via query-pci, and in-guest: the devices come up as 0x1042/0x1044 with memory BARs only). This was measured neutral for boot time - the win is a smaller device surface, not speed. Each device also gets a fixed slot on pcie.0 instead of QEMU's auto-assignment, so guest enumeration order no longer depends on the order of builder calls. This deliberately avoids PCIe root ports: they would force enumeration of buses 1-4 plus pciehp bring-up, and would invalidate the pci=lastbus=0 cmdline optimization, to buy PCI hotplug that spinbox does not use (CPU and memory hotplug go through ACPI/QMP). The slot ranges bound how many disks and NICs fit, checked before building the command line so the failure names the limit instead of surfacing as a QEMU slot collision at exec time. Also enables the seccomp sandbox (the binary is built with --enable-seccomp) and drops S3/S4 from the ACPI tables, which a microVM never uses. Not adopted from that command line: PCIe root ports, virtio-scsi (dead weight next to virtio-blk), virtio-balloon (overlaps the pc-dimm memory hotplug accounting), the qemu-guest-agent channel (vminitd over vsock already is the control plane), -no-shutdown (the shim relies on QEMU exiting), and -S/-boot strict/bootindex (we direct-boot the kernel). The -drive to -blockdev migration is left for its own change: it must preserve file.locking=on on the rwlayer or the snapshotter's commit gate stops detecting running containers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CcXUA5oay5BiEcFKdhCEht
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
11.0.1 → 11.1.1, kernel7.1 → 7.2.1.PREEMPT_LAZY=ypinned.disable-legacy=on) and pinned to fixed PCI slots onpcie.0.Why the kernel config is regenerated, not carried over
olddefconfigon 7.2 silently changed the preemption model. 7.2 addsdepends on !ARCH_HAS_PREEMPT_LAZYtoPREEMPT_VOLUNTARY, and x86 sets that symbol, so the config'sPREEMPT_VOLUNTARY=ybecame unselectable and the fallback landed on fullPREEMPT=y— more scheduling overhead, the opposite of what the old setting asked for.PREEMPT_LAZYis the intended successor for that trade-off and is now pinned explicitly.PREEMPT_DYNAMICstays on, sopreempt=on the cmdline can still override at runtime.PCI slot map
Fixed slots on
pcie.0replace QEMU's auto-assignment, so guest enumeration order no longer depends on the order of builder calls:This deliberately avoids PCIe root ports. They would force enumeration of buses 1-4 plus
pciehpbring-up and invalidate thepci=lastbus=0cmdline optimization, to buy PCI hotplug that spinbox does not use — CPU and memory hotplug go through ACPI/QMP. The ranges bound how many disks and NICs fit, checked before building the command line so the failure names the limit instead of surfacing as a QEMU slot collision at exec time.On
disable-legacy=onSitting on
pcie.0was not enough on its own — QEMU still realized the devices as transitional. Two identicalvirtio-blk-pciin one guest, one with the flag:disable-legacy=on0x1042(virtio 1.0)0x1001(transitional)Measured neutral for boot time. Two rounds, same kernel and QEMU, only the flag changing:
The difference is within run-to-run noise (n=7). The win here is a smaller device surface, not speed; that is recorded in a comment so it is not re-litigated as an optimization.
Not adopted
From the libvirt-style command line this started from: PCIe root ports (above), virtio-scsi (dead weight next to virtio-blk), virtio-balloon (overlaps the
pc-dimmmemory hotplug accounting), the qemu-guest-agent channel (vminitd over vsock already is the control plane),-no-shutdown(the shim relies on QEMU exiting), and-S/-boot strict/bootindex(we direct-boot the kernel).-drive→-blockdevis left for its own change: it must preservefile.locking=onon the rwlayer or the snapshotter's commit gate stops detecting running containers.Testing
task lint(0 issues),go test -race ./internal/....query-pciand in-guest PCI enumeration.🤖 Generated with Claude Code