diff --git a/README.md b/README.md index 4d0b9b9..695ca11 100644 --- a/README.md +++ b/README.md @@ -495,9 +495,10 @@ already exists (Docker, a lab switch), reuse it or switch spotibox to DHCP. ## Validation ```bash -nix flake check --no-build # every system, both VM tests, the prod/debug split +nix flake check --no-build # every system, the VM tests, the prod/debug split nix build .#checks.x86_64-linux.spotibox-basic # boots the appliance in QEMU nix build .#checks.x86_64-linux.spotibox-xrdp-session # starts a real xrdp session in it +nix build .#checks.x86_64-linux.spotibox-xrdp-audio # an RDP client hears it, before and after a reconnect tools/closure.sh report # what the production image is made of tools/closure.sh diff # what the debug image adds on top of it tools/closure.sh why cups # who is still holding on to a store path @@ -666,7 +667,9 @@ purpose NixOS box. Measured with `tools/closure.sh` against nixpkgs 25.11: | `lsvmbus`, and with it the last Python interpreter (see below) | -108 MiB | | a kernel built for one machine instead of for all of them (see below) | -118 MiB | | the subsystems that machine cannot have: sound, radios, GPUs, KVM | -7 MiB | -| **production total** | **1.15 GiB (-70.0%)**, 1024 store paths down to 584 | +| sudo, once no account was left in `wheel` | -6 MiB | +| the ALSA-to-PulseAudio bridge, and the ffmpeg 8 only it was holding (see below) | -34 MiB | +| **production total** | **1.11 GiB (-71.1%)**, 1024 store paths down to 572 | Three of those were never asked for by anything in the appliance: @@ -1099,6 +1102,56 @@ switch either off; `environment.corePackages` would have to be replaced with an allowlist, and that is a separate experiment - system scripts expect GNU semantics, and "it still boots" is not the same as "nothing broke". +### The ALSA Bridge Nothing Crosses + +Enabling PulseAudio on NixOS also writes `/etc/alsa/conf.d/99-pulseaudio.conf`, +which makes `pcm.default` and `ctl.default` alsa-plugins' `pulse` types: a +program written against ALSA opens the default device and plays into +PulseAudio. That file was the production image's only reference to +alsa-plugins, and alsa-plugins its only reference to ffmpeg 8 - linked by the +`a52` encoder and the `lavrate` resampler, two plugins no configuration here +names. The whole chain, and what left with it: + +```text +toplevel -> etc -> etc-alsa-conf.d-99-pulseaudio.conf -> alsa-plugins-1.2.12 + -> ffmpeg-8.0-lib -> ffmpeg-8.0-data, libva, libvdpau, openapv + + ffmpeg-8.0-lib 33,821,496 libva-2.22.0 353,792 + openapv-0.2.0.4 726,200 ffmpeg-8.0-data 279,344 + alsa-plugins-1.2.12 390,456 libvdpau-1.5 103,728 + the file itself 528 + + closure, before and after, each measured in a fresh sandboxed store: + 1,222,866,800 -> 1,187,190,640 bytes (-35,676,160), 579 -> 572 paths +``` + +The ffmpeg 4 that stays is Spotify's own - nixpkgs links it next to the client +because Spotify wants a libavcodec older than 59 - and is a separate question. + +Nothing on this machine crosses the bridge. The production kernel has no sound +support, so there is no ALSA device for a program to reach, bridged or not. +xrdp's sink is a PulseAudio module. The one program here with an ALSA driver is +Spotify, and as of 1.2.74 it constructs its PulseAudio driver first - +`dlopen("libpulse.so.0")`, which its wrapper puts on the library path, the +symbols, a threaded main loop, `pa_context_connect` - and builds the ALSA driver +only when that fails; if ALSA fails as well, it logs "Unable to initialize +sounddriver, using dummy." The only road to the ALSA driver is a PulseAudio +that cannot be reached, and a bridge into PulseAudio cannot reach it either. + +`tests/xrdp-audio.nix` is the acceptance the TODO list asked for: a FreeRDP +client connects to the appliance, the session's PulseAudio plays through xrdp's +sink, and the client has to receive PCM; then it disconnects, reconnects and +has to receive it again. It passes with the bridge and without it, and the +image without it cold-boots under OVMF to an RDP answer (`tools/cold-boot.sh`). +What no VM test here can do is log in to Spotify, so the last word was a track +playing on real Hyper-V, through reconnects. It did, from the image built at +3d6793c (`wsl:spotibox-baseline-2026-09-28-12-g3d6793c`, store path +`7z7pn736…` - the same derivation the cold boot above booted): sound through +mstsc, and sound again after each of several reconnects. + +The debug image keeps the file, next to the stock kernel with ALSA and the +alsa-utils it also keeps. + ### Why The Image Is ext4, And What Compression Would Buy The store compresses about two and a half to one, measured rather than @@ -1317,6 +1370,7 @@ tools/ tests/ spotibox-basic.nix NixOS VM test: what the image contains xrdp-session.nix NixOS VM test: a real xrdp session, X, keyboard, kiosk + xrdp-audio.nix NixOS VM test: RDP audio at a real client, before and after a reconnect appliance-split.nix evaluation-only guard for the prod/debug split closure-budget.nix recorded production closure budget, enforced by CI image-budget.nix recorded image and release sizes, enforced at release @@ -1341,18 +1395,10 @@ spending that. If it turns out that saving 23 MiB of language tables costs a three-storey GTK override to carry forever, the right answer is no, and having the experiment in a separate branch is what makes saying no cheap. -- `alsa-plugins` pulls a full ffmpeg 8 (32 MiB) into an appliance that already - carries ffmpeg 4 for Spotify, and the production audio path never touches - the ALSA device layer at all - traced on a running kiosk, no `snd` module is - loaded and `/proc/asound` does not exist. The causal chain is short enough - to look like a clean override. Acceptance has to include Spotify playing - through RDP audio *after a reconnect*, because dependencies like this have a - habit of being unnecessary right up to the first fallback codec. - GTK3 pulls `iso-codes` (23 MiB) for a language list the kiosk never shows. - Harder: GTK may reach that data through localised country and language names - rather than through a visible picker, so the likely outcome is a patch to - carry rather than an option to set. Second, and only if the first one went - well. + Harder than the ALSA bridge was: GTK may reach that data through localised + country and language names rather than through a visible picker, so the + likely outcome is a patch to carry rather than an option to set. - The next kernel change, whenever it comes, is also the time to spend the rebuild on `THUNDERBOLT = no` becoming `USB4 = no` and on the seven `extra` requests in `tests/kernel-contract.nix`, device-mapper included unless the diff --git a/flake.nix b/flake.nix index 177acd7..698c6eb 100644 --- a/flake.nix +++ b/flake.nix @@ -255,6 +255,11 @@ inherit pkgs; }; + spotibox-xrdp-audio = + import ./tests/xrdp-audio.nix { + inherit pkgs; + }; + spotibox-kernel-contract = import ./tests/kernel-contract.nix { inherit pkgs lib; diff --git a/profiles/modes/prod.nix b/profiles/modes/prod.nix index a4babb3..107d8ff 100644 --- a/profiles/modes/prod.nix +++ b/profiles/modes/prod.nix @@ -276,6 +276,21 @@ lib.mkIf (config.qubix.mode == "prod") { xdg.icons.enable = lib.mkForce false; xdg.sounds.enable = lib.mkForce false; + # 34 MiB for a bridge nothing crosses. The PulseAudio module writes + # /etc/alsa/conf.d/99-pulseaudio.conf, which makes pcm.default and + # ctl.default alsa-plugins' `pulse` types so that programs written against + # ALSA play into PulseAudio. That file is the image's only reference to + # alsa-plugins, and alsa-plugins its only reference to ffmpeg 8 - linked by + # the a52 encoder and lavrate resampler, which nothing here configures. + # + # The kernel has no sound support, so there is no ALSA device to fall back + # to; xrdp's sink is a PulseAudio module; and Spotify, the one program here + # with an ALSA driver, tries PulseAudio first and only turns to ALSA when + # PulseAudio cannot be reached - which a bridge into PulseAudio cannot fix. + # tests/xrdp-audio.nix plays over RDP, and again after a reconnect. The + # debug image keeps the file, with the ALSA kernel and alsa-utils it keeps. + environment.etc."alsa/conf.d/99-pulseaudio.conf".enable = lib.mkForce false; + # ~60 MiB: the default set is DejaVu, FreeFont, Gyre, Liberation, Unifont and # Noto Color Emoji. This appliance renders Latin, Cyrillic and the emoji # people put in playlist names. DejaVu covers the first two - it is also @@ -369,6 +384,7 @@ lib.mkIf (config.qubix.mode == "prod") { "zenity" "pavucontrol" "xterm" + "alsa-plugins" ]; # environment.corePackages calls itself "core packages for a normal diff --git a/tests/closure-budget.nix b/tests/closure-budget.nix index 03a9596..738ddf0 100644 --- a/tests/closure-budget.nix +++ b/tests/closure-budget.nix @@ -12,13 +12,13 @@ spotibox = { # `nix path-info -S` of # nixosConfigurations.spotibox.config.system.build.toplevel. - measuredBytes = 1222866800; + measuredBytes = 1187190640; # CI fails above this: the measurement plus 2% of headroom. - maxBytes = 1247324136; + maxBytes = 1210934452; # What the numbers above were measured against. nixosVersion = "25.11.20260501.26ef669"; - rev = "7cf3e813d742e41a284560b83dcc7858ceba7d8d"; + rev = "345c420de288eccbdf85d2614ad0e029bc28437f"; }; } diff --git a/tests/image-budget.nix b/tests/image-budget.nix index 0b6e994..9dc5ca1 100644 --- a/tests/image-budget.nix +++ b/tests/image-budget.nix @@ -14,16 +14,16 @@ spotibox = { # What the numbers below were measured against. nixosVersion = "hyperv-25.11.20260501.26ef669"; - rev = "7cf3e813d742e41a284560b83dcc7858ceba7d8d"; + rev = "345c420de288eccbdf85d2614ad0e029bc28437f"; - closureBytes = { measured = 1222866800; max = 1284010140; }; - closurePaths = { measured = 579; max = 607; }; + closureBytes = { measured = 1187190640; max = 1246550172; }; + closurePaths = { measured = 572; max = 600; }; kernelBytes = { measured = 24995768; max = 26245556; }; modulesBytes = { measured = 1791416; max = 1880986; }; initrdBytes = { measured = 23092240; max = 24246852; }; - vhdxApparentBytes = { measured = 1719664640; max = 1805647872; }; - vhdxBuilderAllocatedBytes = { measured = 1410596864; max = 1481126707; }; - releaseBytes = { measured = 520302296; max = 546317410; }; - homeReleaseBytes = { measured = 420843; max = 441885; }; + vhdxApparentBytes = { measured = 1686110208; max = 1770415718; }; + vhdxBuilderAllocatedBytes = { measured = 1374699520; max = 1443434496; }; + releaseBytes = { measured = 505700521; max = 530985547; }; + homeReleaseBytes = { measured = 420606; max = 441636; }; }; } diff --git a/tests/xrdp-audio.nix b/tests/xrdp-audio.nix new file mode 100644 index 0000000..0f0b02e --- /dev/null +++ b/tests/xrdp-audio.nix @@ -0,0 +1,133 @@ +{ pkgs }: + +# RDP audio end to end, and again after a reconnect. +# +# tests/xrdp-session.nix starts a session with xrdp-sesrun, which is enough to +# prove the X server, the window manager and the kiosk come up - and proves +# nothing about sound, because sesrun is not an RDP client and never opens the +# audio channel. This test is one: FreeRDP connects to the appliance's own +# xrdp, logs in as `rdp`, and asks for sound. PulseAudio in the session then +# plays noise into its default sink, which the xrdp module makes module-xrdp-sink; +# xrdp-chansrv carries it over RDPSND; and the client logs each block it +# receives. Its fake backend discards the audio, so the log is the evidence: +# a Wave PDU in PCM, the only format the xrdp in profiles/audio offers. +# +# Then the client goes away and comes back, which is where RDP audio has a +# habit of breaking: the session survives the disconnect, chansrv's socket +# does not, and the sink in the still-running PulseAudio has to find the new +# one. The second connection has to receive audio as well. +# +# The driver runs every command under `set -euo pipefail`, which shapes the +# script: nothing here pipes into a reader that stops early (`grep -q`, +# `head`), because the writer then dies of SIGPIPE and fails the pipeline +# whatever the reader found. + +let + uidOf = "id -u rdp"; + pactl = "${pkgs.pulseaudio}/bin/pactl"; + pacat = "${pkgs.pulseaudio}/bin/pacat"; + # WLog writes to stdout, which is a file here and so block-buffered: a line + # the test waits for could sit in the buffer for as long as the client has + # nothing more to say. + stdbuf = "${pkgs.coreutils}/bin/stdbuf -oL -eL"; +in + +pkgs.testers.nixosTest { + name = "spotibox-xrdp-audio"; + + nodes.machine = { lib, ... }: { + imports = [ ../machines/spotibox.nix ]; + + # Same as tests/xrdp-session.nix: no second disk, and room for Spotify, + # which the kiosk session starts whether or not anything plays. + qubix.homeDisk.enable = lib.mkForce false; + virtualisation.memorySize = 2048; + virtualisation.cores = 2; + }; + + testScript = '' + machine.start() + machine.wait_for_unit("multi-user.target") + machine.wait_for_unit("xrdp-sesman.service") + machine.wait_for_unit("xrdp.service") + + # The client needs a display of its own; the appliance has no X server + # outside xrdp sessions. + machine.succeed("${pkgs.xorg.xvfb}/bin/Xvfb :99 -screen 0 1280x800x24 >/dev/null 2>&1 &") + machine.wait_for_file("/tmp/.X11-unix/X99") + + uid = machine.succeed("${uidOf}").strip() + # xrdp 0.10 keeps a session's sockets in a directory per user. chansrv + # listens there for the sink while a client has sound, and stops listening + # while a sink is connected, so the listing is a picture, not a check. + sockdir = f"/run/xrdp/{uid}" + + def as_rdp(cmd): + return f"su rdp -s /bin/sh -c 'XDG_RUNTIME_DIR=/run/user/{uid} {cmd}'" + + def pactl_says(subcommand, pattern): + return as_rdp(f"${pactl} {subcommand}") + f" | grep -E '{pattern}' >/dev/null" + + def sink_running(): + return pactl_says("list sinks short", "xrdp-sink.*RUNNING") + + # Every command here runs as `timeout 900 bash -c ''`, so a + # pattern that matches whole command lines (pgrep -f) finds the command + # looking for it. The client is matched by process name instead. + def client_gone(): + machine.succeed("${pkgs.procps}/bin/pkill -x xfreerdp") + machine.wait_until_fails("${pkgs.procps}/bin/pgrep -x xfreerdp") + + def connect(n): + machine.succeed( + "DISPLAY=:99 ${stdbuf} ${pkgs.freerdp}/bin/xfreerdp /v:127.0.0.1 /u:rdp /p:1234 " + "/cert:ignore /size:1280x800 /sound:sys:fake /log-level:INFO " + "/log-filters:com.freerdp.channels.rdpsnd.client:DEBUG " + f">/tmp/xfreerdp-{n}.log 2>&1 &" + ) + # chansrv sends a client that asked for sound its formats and then a + # training PDU. That line in this connection's own log is what says + # its channel is up - not chansrv's socket, which could be left over + # from the connection before. + machine.wait_until_succeeds(f"grep -q 'Training Request' /tmp/xfreerdp-{n}.log", timeout=600) + print(machine.succeed(f"ls -l {sockdir}")) + + def plays(n): + # Noise for half a minute into the session's default sink, and the + # client has to log blocks of it arriving while it plays. + machine.succeed(as_rdp("timeout 30 ${pacat} --format=s16le --rate=44100 --channels=2 < /dev/urandom") + " >/dev/null 2>&1 &") + machine.wait_until_succeeds(sink_running(), timeout=60) + machine.wait_until_succeeds(f"grep -q -E 'Wave2PDU|Wave: cBlockNo' /tmp/xfreerdp-{n}.log", timeout=120) + log = machine.succeed(f"grep -m 5 -E 'WaveInfo|Wave2PDU|Opening device' /tmp/xfreerdp-{n}.log") + print(log) + assert "WAVE_FORMAT_PCM" in log, log + machine.wait_until_fails(sink_running(), timeout=90) + + def diagnose(): + for cmd in [ + "tail -n 30 /tmp/xfreerdp-*.log", + f"ls -la /run/xrdp {sockdir}", + "tail -n 40 /home/rdp/.local/share/xrdp/*.log", + ]: + print(machine.execute(cmd)[1]) + + try: + connect(1) + # The session's startup script loads the xrdp sink and makes it the + # default, on its own schedule: until it has, a stream goes to the + # null sink PulseAudio starts with. + machine.wait_until_succeeds(pactl_says("info", "Default Sink: xrdp-sink"), timeout=120) + plays(1) + + # Gone and back. The session outlives the client; the audio has to + # follow the new one. The pause is xrdp's time to notice the first + # client has gone, so the second is a reconnect and not a race. + client_gone() + machine.sleep(5) + connect(2) + plays(2) + except Exception: + diagnose() + raise + ''; +} diff --git a/tools/cold-boot.sh b/tools/cold-boot.sh index 86112bc..c3812a0 100755 --- a/tools/cold-boot.sh +++ b/tools/cold-boot.sh @@ -31,9 +31,12 @@ home=$(nix build --no-link --print-out-paths ".#spotibox-home-vhdx") vhdx=$(find -L "$system" -name '*.vhdx' -print -quit) test -n "$vhdx" || { echo "no .vhdx in $system" >&2; exit 1; } -read -r ovmf qemu < <(nix build --no-link --print-out-paths \ +# One read per line: `read` fails at an end of input that has no newline, and +# under `set -e` a single read of both paths joined on one line ended the +# script right here. +{ read -r ovmf; read -r qemu; } < <(nix build --no-link --print-out-paths \ --impure --expr 'let p = (builtins.getFlake (toString ./.)).nixosConfigurations.spotibox.pkgs; - in [ p.OVMF.fd p.qemu_kvm ]' | tr '\n' ' ') + in [ p.OVMF.fd p.qemu_kvm ]') cp "$ovmf/FV/OVMF_VARS.fd" "$work/vars.fd" chmod +w "$work/vars.fd"