Skip to content

Adopt bind-once microVM image slots - #406

Open
Enrique Saurez (esaurez) wants to merge 6 commits into
dev-pre-cpu-profilefrom
esaurez/nvx-image-slots-20261005
Open

Enrique Saurez (esaurez) wants to merge 6 commits into
dev-pre-cpu-profilefrom
esaurez/nvx-image-slots-20261005

Conversation

@esaurez

@esaurez Enrique Saurez (esaurez) commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Pins OpenVMM to nanvix/openvmm#116 (6aaf6fb7), which adds bind-once read-only virtio-blk image slots to the microVM profile, and adopts them in NVX. The repin contains only these commits on top of the current pin 4355c010:

  • 368b9d2a virtio_blk: add bind-once image slots
  • 6c12f620 openvmm_entry: add microVM image slots
  • 10106b13 microvm: admit declared image-slot transports in machine validation
  • bf98f65f microvm: advertise and gate image-slot restore targets
  • 6aaf6fb7 microvm: end host-control sessions quietly on a clean close

No other fork changes are included.

  • ABI: VMs that declare slots use microVM ABI 3; the source manifest's microvm_abi_version is 3. VMs without slots keep ABI 2 unchanged.
  • CLI and harness: --image-slot-boot-count for managed sandboxes, --restore-image-slots, query-image-slots, and bind-image-slot. Slot launches may omit the distro layer; ABI-2 launches keep the existing layer requirement.
  • Host control: authenticated protocol-v1 client, using the existing 32-byte capability and the owner-restricted local endpoint.
  • Guest:
    • kernel shared-status mapping for the four slot windows;
    • OPENVMM_ENTROPY_V4 parsing, with portb status bit 6 routing image-slot targets to restore repair;
    • restore-time prefix activation before acknowledgement;
    • capacity/config-change discovery, partition rescan, and stable read-only /dev/nvx-imageN links. Placeholders stay unbound.
  • CI: cross-backend image-slot correctness covers:
    • boot with one active slot, then bind and read;
    • idempotent bind, and rejection of a different identity;
    • restore to four active slots, then bind a restored slot.
  • Benchmarks: benchmark --suite image-slot-boot compares ABI 2, ABI 3 with one active slot, and ABI 3 with four active slots.
  • Docs: machine and device ABI, limits, remaining work, snapshot and restore packet and status bit, run/usage/setup/build guides, implementation status, and benchmark methodology.

Base

This PR is intentionally based on db5138e3, before the CPU-profile repin on dev (#394), and targets dev-pre-cpu-profile, created at that commit. CI workflows run only for pull requests into dev, so I validated this PR manually on an isolated MSHV host.

Validation

Local (Windows):

  • At 1cbf5c7d: python scripts/nvx.py verify, compileall, pyright (Linux and Windows), ruff check, ruff format --check, and git diff --check passed. Focused unit tests: 63 passed. Supporting suites: 108 passed, 17 skipped.
  • Later commits: ruff check, ruff format --check, and git diff --check passed. The benchmark tests passed (83, with 1 skipped), and the new console-ordering test fails without its fix.
  • shellcheck --shell=sh and shfmt -d -ln posix -i 4 -ci on the changed guest scripts: clean.
  • nvx-port-io.c compiles for x86_64-linux-musl with -static -Os -Wall -Wextra -Werror.

Linux/MSHV, head 402c01f2 with OpenVMM bf98f65f:

  • Host: AMD EPYC 9V74, Linux 6.6.148-200.mshv-b1f02da.azl3.

  • python3 scripts/nvx.py build-guest (Docker) and build-openvmm --backend mshv (musl, Rust 1.95.0). Both built from clean sources.

  • The CI CLI validation (compileall, the seven unittest modules, test_adversarial.py, verify) passed: 540 tests passed (29 skipped), and 70 adversarial tests passed.

  • test-microvm --backend mshv --scenario image-slots: passed.

    • Boot B = 1, then bind, idempotent rebind, rejection of a different identity, and a guest read of the bound media.
    • Capture an empty-slot snapshot, restore with --restore-image-slots 4, then bind and read slot 3.
  • test-microvm --backend mshv (default suite, 36 scenarios): all passed. This covers boot, console, lifecycle, sandbox blocks, scratch/SMP/network/filesystem snapshots, restore processors/memory/TSC, snapshot tiers, network policy, and image slots.

  • benchmark --suite image-slot-boot --backend mshv --processors 1 --warmups 5 --runs 51, measuring cold boot to ALPINE-MICROVM-BOOT-OK:

    Configuration p50 p95 Δ p50 vs ABI 2
    ABI 2 342.3 ms 392.6 ms —
    ABI 3, 1 active slot 345.6 ms 396.2 ms +3.3 ms
    ABI 3, 4 active slots 353.0 ms 405.3 ms +10.7 ms

    Configurations ran in consecutive blocks, not interleaved, on a host that also had an unrelated VM running.

Linux/MSHV, head 6da62e93 with OpenVMM 6aaf6fb7:

  • 6da62e93 changes only the OpenVMM pin and the design doc. OpenVMM 6aaf6fb7 stops reporting a client's normal close of the host-control connection as a failed connection.
  • Clean rebuild: build-guest (initramfs a04b57e4..., unchanged from 402c01f2) and build-openvmm --backend mshv (bf0ec482..., source revision 6aaf6fb7).
  • test-microvm --backend mshv --scenario image-slots: passed.

Live validation found four defects, all fixed in this PR:

  • OpenVMM rejected the slot transports during machine validation (fixed in 10106b13).
  • Restore-time activation had no status bit, so the guest never repaired the slots or released the gate (bf98f65f plus 46c6f595).
  • The benchmark closed its console before the guest consumed the exit command (402c01f2).
  • OpenVMM reported every normal host-control client close as a failed connection (6aaf6fb7, pinned by 6da62e93).

Not yet validated

KVM and WHP live runs.

Promote the OpenVMM gitlink to 6c12f620, advertise microVM ABI 3, and document the fixed four-slot machine and restore contract. The promotion contains only the two image-slot commits and no unrelated fork changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add managed sandbox boot and bind controls, ABI-3 guest discovery and restore repair, cross-backend correctness coverage, and matched ABI-2/B=1/B=4 boot-cost benchmarks.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
OpenVMM 6c12f620 rejected the image-slot transports during machine
validation, so every slot-declaring launch failed before boot. Advance the
gitlink to 10106b13, which admits exactly the declared slot transports and
leaves machines without slots unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Image-slot activation reached the guest as a version-4 restore packet with no
status bit of its own. A legacy-tier restore therefore skipped post-restore
repair, left newly active slots unbound, and never released the restore gate.

Pin OpenVMM bf98f65f, which advertises version-4 targets with portb status bit
6 and gates every explicit image-slot target. Route that bit to post-restore
repair and keep it out of the base-memory fast path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The image-slot boot benchmark closed its TCP console immediately after sending
the guest exit command. On MSHV the guest then never received the command, so
every sample timed out. Keep the console connected until OpenVMM exits, as the
correctness tests already do.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
OpenVMM 6aaf6fb7 treats a client closing the host-control connection after its last response as a normal end of session instead of logging a failed connection. The protocol and ABI are unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant