Skip to content

Add kernel-6.18-microvm - #585

Open
ginglis13 wants to merge 3 commits into
bottlerocket-os:developfrom
ginglis13:add-kernel-6.18-microvm
Open

ginglis13 wants to merge 3 commits into
bottlerocket-os:developfrom
ginglis13:add-kernel-6.18-microvm

Conversation

@ginglis13

@ginglis13 ginglis13 commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Issue

#606

Description of changes:

Add kernel-6.18-microvm, a guest kernel for microVMs built from the same SRPM and patch set as kernel-6.18, with a smaller static footprint.

The biggest change comes in the config overlay (packages/kernel-6.18-microvm/config-microvm, merged last, after the shared base fragments):

  • Boot path built in: VIRTIO_MMIO, VIRTIO_BALLOON, VIRTIO_NET, guest-side vsock (VIRTIO_VSOCKETS), NSM. virtio-blk, dm-verity, erofs and the serial console are already =y in the shared base. aarch64 adds RTC_DRV_PL031=y for Firecracker's PL031 wall clock.
  • Footprint: MAXSMP off / NR_CPUS=256, LOG_BUF_SHIFT=17, NVMe =m, DMA_CMA off, FB/fbcon/DRM_FBDEV_EMULATION off, DEBUG_INFO_BTF off (DEBUG_INFO_NONE=y, no vmlinux.h generated), IKCONFIG off (/boot/config still shipped), FTRACE/KPROBES off, PROC_KCORE off. No CPU microcode is embedded

This results in the tradeoff - with FTRACE and BTF off this kernel means no in-guest ftrace, kprobes/uprobes, tracepoint-based eBPF, BPF LSM, or sched_ext.

This PR also extracts out the shared custom kconfigs to a new package, kernel-6.18-shared-configs, for sharing between kernel-6.18 and kernel-6.18-microvm

latest-kernel-full-config.sh is now package-directory aware: it merges config-microvm when present, skips the microcode fragment for *-microvm, and skips source fetch for packages with no url = in Cargo.toml. The four config-full-bottlerocket-*-on-* files for the new package are generated by it.

For an annotated diff of this new kernel-6.18-microvm vs. kernel-6.18, see https://gist.github.com/ginglis13/39595b1ae57976546a9f5d221f8d0753

Testing done:

See full details of a boot test on x86 and arm via firecracker in this gist: https://gist.github.com/ginglis13/a762a343b0d29f3d3c9bdf52a579ca76

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch 3 times, most recently from eda0c66 to 5703d82 Compare October 8, 2026 22:05
@ginglis13

Copy link
Copy Markdown
Contributor Author

^ force pushes have been commitlint fixes, bringing SRPM up to latest to match kernel-6.18 and a fix for the aarch64 config that is generated by make full-config

@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch 2 times, most recently from 1120c6d to e748a6a Compare October 9, 2026 02:33
@ginglis13

ginglis13 commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor Author

^ force pushes extract out the shared custom kconfigs to a new package, kernel-6.18-shared-configs, and enumerate packaged modules for kernel-6.18-microvm rather than glob.

@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch from e748a6a to 4544a11 Compare October 9, 2026 02:38
@ginglis13

Copy link
Copy Markdown
Contributor Author

^ fix tools/latest-kernel-full-config.sh to skip fetch of sources if no url declared in a Cargo.toml in a package: https://github.com/bottlerocket-os/bottlerocket-kernel-kit/compare/e748a6a61b02ca46ad0b5d3ddad12f7a82c616d1..4544a11c22522760bc6e056d56c66f41de5c382c

@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch from 4544a11 to 65430b0 Compare October 13, 2026 16:38
@ginglis13

Copy link
Copy Markdown
Contributor Author

^ force push fixes License on the new kernel-6.18-shared-configs pkg to be Apache-2.0 OR MIT, matching other 1st party sources shipped in the kernel-kit

@cbgbt

cbgbt commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Hello friends! I'm curious if you'd consider enabling virtio-mem as well?

@cbgbt

cbgbt commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

On second thought, I don't think I need to build virtio-mem directly in. The module is fine. Disregard!

@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch from 65430b0 to 6e7c4ac Compare October 28, 2026 23:54
@ginglis13

Copy link
Copy Markdown
Contributor Author

^ force push:

  • DRM fix: ship drm_client_lib.ko (was %excluded). keep the full DRM set modular, matching the shared base (no DRM disable).
  • Dead configs removed: dropped CONFIG_DM_CRYPT=y and CONFIG_INPUT_VIVALDIFMAP=y, which both silently resolved to =m.
  • PS/2 stack is now modular to match base kernel
  • FIPS enabled (match base): removed stale Conflicts: image-feature(fips); added the -fips subpackage, fipsmodules-{x86_64,aarch64} + check-fips-modules.drop-in.conf.in sources, the drop-in install loop, and conditional Requires: %{name}-fips.
  • Conflicts: kernel-6.18 added (both install to the same /boot + /lib/modules paths).
  • SRPM bump to 6.18.56-125.164, patches re-synced to base 6.18 kernel

https://github.com/bottlerocket-os/bottlerocket-kernel-kit/compare/65430b0529ce2b6f899ab7cdc7df1a3c60546638..6e7c4acaf9962c2b2ca1a73eafbd857cbee018be

@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch from 6e7c4ac to 5f99013 Compare October 29, 2026 00:10
@ginglis13

Copy link
Copy Markdown
Contributor Author

^ force push installs kernel as /boot/vmlinuz regardless of arch, matching what the base 6.18 kernel does https://github.com/bottlerocket-os/bottlerocket-kernel-kit/compare/6e7c4acaf9962c2b2ca1a73eafbd857cbee018be..5f990132a4e1b02871eb05d44647a6c1225439ac

@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch 2 times, most recently from 5ab2509 to c1ce6d4 Compare October 29, 2026 18:33
@ginglis13

Copy link
Copy Markdown
Contributor Author

^ build erofs zstd decompression frontend into guest config

@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch from c1ce6d4 to 34ebd4d Compare October 29, 2026 19:02
@ginglis13

Copy link
Copy Markdown
Contributor Author

^ force push

  • set CONFIG_RTC_DRV_PL031=y for aarch64 clock
  • drop firmware from microvm configs, and adjust config checks accordingly

https://github.com/bottlerocket-os/bottlerocket-kernel-kit/compare/c1ce6d43954dcd7a58984b77a3fd6d82ba13e22d..34ebd4d96ca9152e803a049b531947e25b83a0c8

@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch from 34ebd4d to 1cbdebe Compare October 29, 2026 21:41
@ginglis13

Copy link
Copy Markdown
Contributor Author

^ force push has some fixes to further trim the guest

  • NR_CPUS now 256 ( and MAXSMP off)
  • BTF off (DEBUG_INFO_NONE=y) + removed the spec's vmlinux.h BTF-dump build step
  • FTRACE/KPROBES off, IKCONFIG(+_PROC) off, PROC_KCORE off, LOG_BUF_SHIFT=17, NVMe=m
  • Relaxed the base-subset check so the config-microvm overlay may override shared-base pins
  • %files: added nvme modules (nvme, nvme-core), removed drm_client_lib

https://github.com/bottlerocket-os/bottlerocket-kernel-kit/compare/34ebd4d96ca9152e803a049b531947e25b83a0c8..1cbdebec4cbc1e56415cda800a24f84d31a602da

@ginglis13
ginglis13 requested review from KCSesh and vigh-m October 30, 2026 17:50
@ginglis13
ginglis13 requested a review from piyush-jena October 30, 2026 17:51
@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch from 1cbdebe to ba5c3b8 Compare October 30, 2026 20:35
@ginglis13

Copy link
Copy Markdown
Contributor Author

^ moved a few built-ins back to loadable modules for the microvm:

  • CONFIG_ACPI_AC, CONFIG_ACPI_BUTTON, CONFIG_ACPI_THERMAL, CONFIG_EEPROM_93CX6
  • CONFIG_ACPI_TINY_POWER_BUTTON is not set

https://github.com/bottlerocket-os/bottlerocket-kernel-kit/compare/1cbdebec4cbc1e56415cda800a24f84d31a602da..ba5c3b8006eac33ad14452aeeb9911b7c7c4607c

Comment thread packages/kernel-6.18-microvm/Cargo.toml Resolved
@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch from ba5c3b8 to c7c6af5 Compare October 30, 2026 21:49
@ginglis13

ginglis13 commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor Author

^ force push addresses @KCSesh 's comments (except for one, I have one more force push coming 🔜 ) https://github.com/bottlerocket-os/bottlerocket-kernel-kit/compare/ba5c3b8006eac33ad14452aeeb9911b7c7c4607c..c7c6af5f7ab0c95dc8a688d30abde52a12d7b370

@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch from c7c6af5 to ddd2f73 Compare October 30, 2026 21:56
@ginglis13

Copy link
Copy Markdown
Contributor Author

Introduce a kernel-6.18-shared-configs package that ships the base
Bottlerocket kernel configuration fragments (config-bottlerocket and
its per-architecture variants) and installs them into the build
sysroot.

Every 6.18-based kernel can then consume these fragments at build time
via a BuildRequires rather than carrying its own copy. Convert
kernel-6.18 to pull them from the shared package: drop its local
config-bottlerocket{,-x86_64,-aarch64} sources, add the build
dependency, and merge the fragments from %{kernel_configdir} in %prep.

Signed-off-by: Gavin Inglis <giinglis@amazon.com>
latest-kernel-full-config.sh reconstructed each package directory from
the SRPM's majorminor version (6.18.51 -> 6.18), so two packages that
share a majorminor (kernel-6.18 and kernel-6.18-microvm) could not be
told apart and the microvm flavor never got its own configuration
generated. Pass the actual package directory through merge/validate
instead.

Signed-off-by: Gavin Inglis <giinglis@amazon.com>
@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch from ddd2f73 to 425eaa2 Compare October 30, 2026 23:36
@ginglis13

Copy link
Copy Markdown
Contributor Author

^ latest force push

  • Trimmed the config-microvm overlay to match the intended built-in set. Removed ~32 =y promotions. Only 9 symbols actually change to =m. The other ~23 were redundant pins that still resolve =y via selection from kept built-ins. Added the 9 corresponding .ko entries to %files.
  • Fixed innaccurate KALLSYMS_ALL comment and actually disabled it. The old note claiming it was force-selected and undisablable was wrong
  • Hardened the full-config validator: An overlay =y/=m symbol that resolves in none of the four config-full files was previously silently skipped; it's now an error,
  • Reordered commits for reproducibility
  • Fixed tab indentation in latest-kernel-full-config.sh.

and addressed my 2 comments above

https://github.com/bottlerocket-os/bottlerocket-kernel-kit/compare/ddd2f7394a5f911b92c873000f7412e397517af3..425eaa29d315389ac54c93dba79bc184d2d1a8b2

add a new flavor of the 6.18 kernel to support booting microvms such as
Firecracker.

This flavor reuses the same source, patches and Bottlerocket config
fragments as kernel-6.18, adds config-microvm to build the virtio
transport, network and vsock drivers in, and ships bzImage or
vmlinuz.efi depending on architecture

Include a new config-microvm for setting configuration options specific
to guests so a microvm guest can boot a minimal virtio device model.

Signed-off-by: Gavin Inglis <giinglis@amazon.com>
@ginglis13
ginglis13 force-pushed the add-kernel-6.18-microvm branch from 425eaa2 to 8c10702 Compare October 31, 2026 00:07
@ginglis13

Copy link
Copy Markdown
Contributor Author

^ small doc comment fixup

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants