Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 59 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -495,9 +495,10 @@ already exists (Docker, a lab switch), reuse it or switch spotibox to DHCP.
## Validation

```bash
nix flake check --no-build # every system, both VM tests, the prod/debug split
nix flake check --no-build # every system, the VM tests, the prod/debug split
nix build .#checks.x86_64-linux.spotibox-basic # boots the appliance in QEMU
nix build .#checks.x86_64-linux.spotibox-xrdp-session # starts a real xrdp session in it
nix build .#checks.x86_64-linux.spotibox-xrdp-audio # an RDP client hears it, before and after a reconnect
tools/closure.sh report # what the production image is made of
tools/closure.sh diff # what the debug image adds on top of it
tools/closure.sh why cups # who is still holding on to a store path
Expand Down Expand Up @@ -666,7 +667,9 @@ purpose NixOS box. Measured with `tools/closure.sh` against nixpkgs 25.11:
| `lsvmbus`, and with it the last Python interpreter (see below) | -108 MiB |
| a kernel built for one machine instead of for all of them (see below) | -118 MiB |
| the subsystems that machine cannot have: sound, radios, GPUs, KVM | -7 MiB |
| **production total** | **1.15 GiB (-70.0%)**, 1024 store paths down to 584 |
| sudo, once no account was left in `wheel` | -6 MiB |
| the ALSA-to-PulseAudio bridge, and the ffmpeg 8 only it was holding (see below) | -34 MiB |
| **production total** | **1.11 GiB (-71.1%)**, 1024 store paths down to 572 |

Three of those were never asked for by anything in the appliance:

Expand Down Expand Up @@ -1099,6 +1102,56 @@ switch either off; `environment.corePackages` would have to be replaced with an
allowlist, and that is a separate experiment - system scripts expect GNU
semantics, and "it still boots" is not the same as "nothing broke".

### The ALSA Bridge Nothing Crosses

Enabling PulseAudio on NixOS also writes `/etc/alsa/conf.d/99-pulseaudio.conf`,
which makes `pcm.default` and `ctl.default` alsa-plugins' `pulse` types: a
program written against ALSA opens the default device and plays into
PulseAudio. That file was the production image's only reference to
alsa-plugins, and alsa-plugins its only reference to ffmpeg 8 - linked by the
`a52` encoder and the `lavrate` resampler, two plugins no configuration here
names. The whole chain, and what left with it:

```text
toplevel -> etc -> etc-alsa-conf.d-99-pulseaudio.conf -> alsa-plugins-1.2.12
-> ffmpeg-8.0-lib -> ffmpeg-8.0-data, libva, libvdpau, openapv

ffmpeg-8.0-lib 33,821,496 libva-2.22.0 353,792
openapv-0.2.0.4 726,200 ffmpeg-8.0-data 279,344
alsa-plugins-1.2.12 390,456 libvdpau-1.5 103,728
the file itself 528

closure, before and after, each measured in a fresh sandboxed store:
1,222,866,800 -> 1,187,190,640 bytes (-35,676,160), 579 -> 572 paths
```

The ffmpeg 4 that stays is Spotify's own - nixpkgs links it next to the client
because Spotify wants a libavcodec older than 59 - and is a separate question.

Nothing on this machine crosses the bridge. The production kernel has no sound
support, so there is no ALSA device for a program to reach, bridged or not.
xrdp's sink is a PulseAudio module. The one program here with an ALSA driver is
Spotify, and as of 1.2.74 it constructs its PulseAudio driver first -
`dlopen("libpulse.so.0")`, which its wrapper puts on the library path, the
symbols, a threaded main loop, `pa_context_connect` - and builds the ALSA driver
only when that fails; if ALSA fails as well, it logs "Unable to initialize
sounddriver, using dummy." The only road to the ALSA driver is a PulseAudio
that cannot be reached, and a bridge into PulseAudio cannot reach it either.

`tests/xrdp-audio.nix` is the acceptance the TODO list asked for: a FreeRDP
client connects to the appliance, the session's PulseAudio plays through xrdp's
sink, and the client has to receive PCM; then it disconnects, reconnects and
has to receive it again. It passes with the bridge and without it, and the
image without it cold-boots under OVMF to an RDP answer (`tools/cold-boot.sh`).
What no VM test here can do is log in to Spotify, so the last word was a track
playing on real Hyper-V, through reconnects. It did, from the image built at
3d6793c (`wsl:spotibox-baseline-2026-09-28-12-g3d6793c`, store path
`7z7pn736…` - the same derivation the cold boot above booted): sound through
mstsc, and sound again after each of several reconnects.

The debug image keeps the file, next to the stock kernel with ALSA and the
alsa-utils it also keeps.

### Why The Image Is ext4, And What Compression Would Buy

The store compresses about two and a half to one, measured rather than
Expand Down Expand Up @@ -1317,6 +1370,7 @@ tools/
tests/
spotibox-basic.nix NixOS VM test: what the image contains
xrdp-session.nix NixOS VM test: a real xrdp session, X, keyboard, kiosk
xrdp-audio.nix NixOS VM test: RDP audio at a real client, before and after a reconnect
appliance-split.nix evaluation-only guard for the prod/debug split
closure-budget.nix recorded production closure budget, enforced by CI
image-budget.nix recorded image and release sizes, enforced at release
Expand All @@ -1341,18 +1395,10 @@ spending that. If it turns out that saving 23 MiB of language tables costs a
three-storey GTK override to carry forever, the right answer is no, and having
the experiment in a separate branch is what makes saying no cheap.

- `alsa-plugins` pulls a full ffmpeg 8 (32 MiB) into an appliance that already
carries ffmpeg 4 for Spotify, and the production audio path never touches
the ALSA device layer at all - traced on a running kiosk, no `snd` module is
loaded and `/proc/asound` does not exist. The causal chain is short enough
to look like a clean override. Acceptance has to include Spotify playing
through RDP audio *after a reconnect*, because dependencies like this have a
habit of being unnecessary right up to the first fallback codec.
- GTK3 pulls `iso-codes` (23 MiB) for a language list the kiosk never shows.
Harder: GTK may reach that data through localised country and language names
rather than through a visible picker, so the likely outcome is a patch to
carry rather than an option to set. Second, and only if the first one went
well.
Harder than the ALSA bridge was: GTK may reach that data through localised
country and language names rather than through a visible picker, so the
likely outcome is a patch to carry rather than an option to set.
- The next kernel change, whenever it comes, is also the time to spend the
rebuild on `THUNDERBOLT = no` becoming `USB4 = no` and on the seven `extra`
requests in `tests/kernel-contract.nix`, device-mapper included unless the
Expand Down
5 changes: 5 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,11 @@
inherit pkgs;
};

spotibox-xrdp-audio =
import ./tests/xrdp-audio.nix {
inherit pkgs;
};

spotibox-kernel-contract =
import ./tests/kernel-contract.nix {
inherit pkgs lib;
Expand Down
16 changes: 16 additions & 0 deletions profiles/modes/prod.nix
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,21 @@ lib.mkIf (config.qubix.mode == "prod") {
xdg.icons.enable = lib.mkForce false;
xdg.sounds.enable = lib.mkForce false;

# 34 MiB for a bridge nothing crosses. The PulseAudio module writes
# /etc/alsa/conf.d/99-pulseaudio.conf, which makes pcm.default and
# ctl.default alsa-plugins' `pulse` types so that programs written against
# ALSA play into PulseAudio. That file is the image's only reference to
# alsa-plugins, and alsa-plugins its only reference to ffmpeg 8 - linked by
# the a52 encoder and lavrate resampler, which nothing here configures.
#
# The kernel has no sound support, so there is no ALSA device to fall back
# to; xrdp's sink is a PulseAudio module; and Spotify, the one program here
# with an ALSA driver, tries PulseAudio first and only turns to ALSA when
# PulseAudio cannot be reached - which a bridge into PulseAudio cannot fix.
# tests/xrdp-audio.nix plays over RDP, and again after a reconnect. The
# debug image keeps the file, with the ALSA kernel and alsa-utils it keeps.
environment.etc."alsa/conf.d/99-pulseaudio.conf".enable = lib.mkForce false;

# ~60 MiB: the default set is DejaVu, FreeFont, Gyre, Liberation, Unifont and
# Noto Color Emoji. This appliance renders Latin, Cyrillic and the emoji
# people put in playlist names. DejaVu covers the first two - it is also
Expand Down Expand Up @@ -369,6 +384,7 @@ lib.mkIf (config.qubix.mode == "prod") {
"zenity"
"pavucontrol"
"xterm"
"alsa-plugins"
];

# environment.corePackages calls itself "core packages for a normal
Expand Down
6 changes: 3 additions & 3 deletions tests/closure-budget.nix
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,13 @@
spotibox = {
# `nix path-info -S` of
# nixosConfigurations.spotibox.config.system.build.toplevel.
measuredBytes = 1222866800;
measuredBytes = 1187190640;

# CI fails above this: the measurement plus 2% of headroom.
maxBytes = 1247324136;
maxBytes = 1210934452;

# What the numbers above were measured against.
nixosVersion = "25.11.20260501.26ef669";
rev = "7cf3e813d742e41a284560b83dcc7858ceba7d8d";
rev = "345c420de288eccbdf85d2614ad0e029bc28437f";
};
}
14 changes: 7 additions & 7 deletions tests/image-budget.nix
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,16 @@
spotibox = {
# What the numbers below were measured against.
nixosVersion = "hyperv-25.11.20260501.26ef669";
rev = "7cf3e813d742e41a284560b83dcc7858ceba7d8d";
rev = "345c420de288eccbdf85d2614ad0e029bc28437f";

closureBytes = { measured = 1222866800; max = 1284010140; };
closurePaths = { measured = 579; max = 607; };
closureBytes = { measured = 1187190640; max = 1246550172; };
closurePaths = { measured = 572; max = 600; };
kernelBytes = { measured = 24995768; max = 26245556; };
modulesBytes = { measured = 1791416; max = 1880986; };
initrdBytes = { measured = 23092240; max = 24246852; };
vhdxApparentBytes = { measured = 1719664640; max = 1805647872; };
vhdxBuilderAllocatedBytes = { measured = 1410596864; max = 1481126707; };
releaseBytes = { measured = 520302296; max = 546317410; };
homeReleaseBytes = { measured = 420843; max = 441885; };
vhdxApparentBytes = { measured = 1686110208; max = 1770415718; };
vhdxBuilderAllocatedBytes = { measured = 1374699520; max = 1443434496; };
releaseBytes = { measured = 505700521; max = 530985547; };
homeReleaseBytes = { measured = 420606; max = 441636; };
};
}
133 changes: 133 additions & 0 deletions tests/xrdp-audio.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
{ pkgs }:

# RDP audio end to end, and again after a reconnect.
#
# tests/xrdp-session.nix starts a session with xrdp-sesrun, which is enough to
# prove the X server, the window manager and the kiosk come up - and proves
# nothing about sound, because sesrun is not an RDP client and never opens the
# audio channel. This test is one: FreeRDP connects to the appliance's own
# xrdp, logs in as `rdp`, and asks for sound. PulseAudio in the session then
# plays noise into its default sink, which the xrdp module makes module-xrdp-sink;
# xrdp-chansrv carries it over RDPSND; and the client logs each block it
# receives. Its fake backend discards the audio, so the log is the evidence:
# a Wave PDU in PCM, the only format the xrdp in profiles/audio offers.
#
# Then the client goes away and comes back, which is where RDP audio has a
# habit of breaking: the session survives the disconnect, chansrv's socket
# does not, and the sink in the still-running PulseAudio has to find the new
# one. The second connection has to receive audio as well.
#
# The driver runs every command under `set -euo pipefail`, which shapes the
# script: nothing here pipes into a reader that stops early (`grep -q`,
# `head`), because the writer then dies of SIGPIPE and fails the pipeline
# whatever the reader found.

let
uidOf = "id -u rdp";
pactl = "${pkgs.pulseaudio}/bin/pactl";
pacat = "${pkgs.pulseaudio}/bin/pacat";
# WLog writes to stdout, which is a file here and so block-buffered: a line
# the test waits for could sit in the buffer for as long as the client has
# nothing more to say.
stdbuf = "${pkgs.coreutils}/bin/stdbuf -oL -eL";
in

pkgs.testers.nixosTest {
name = "spotibox-xrdp-audio";

nodes.machine = { lib, ... }: {
imports = [ ../machines/spotibox.nix ];

# Same as tests/xrdp-session.nix: no second disk, and room for Spotify,
# which the kiosk session starts whether or not anything plays.
qubix.homeDisk.enable = lib.mkForce false;
virtualisation.memorySize = 2048;
virtualisation.cores = 2;
};

testScript = ''
machine.start()
machine.wait_for_unit("multi-user.target")
machine.wait_for_unit("xrdp-sesman.service")
machine.wait_for_unit("xrdp.service")

# The client needs a display of its own; the appliance has no X server
# outside xrdp sessions.
machine.succeed("${pkgs.xorg.xvfb}/bin/Xvfb :99 -screen 0 1280x800x24 >/dev/null 2>&1 &")
machine.wait_for_file("/tmp/.X11-unix/X99")

uid = machine.succeed("${uidOf}").strip()
# xrdp 0.10 keeps a session's sockets in a directory per user. chansrv
# listens there for the sink while a client has sound, and stops listening
# while a sink is connected, so the listing is a picture, not a check.
sockdir = f"/run/xrdp/{uid}"

def as_rdp(cmd):
return f"su rdp -s /bin/sh -c 'XDG_RUNTIME_DIR=/run/user/{uid} {cmd}'"

def pactl_says(subcommand, pattern):
return as_rdp(f"${pactl} {subcommand}") + f" | grep -E '{pattern}' >/dev/null"

def sink_running():
return pactl_says("list sinks short", "xrdp-sink.*RUNNING")

# Every command here runs as `timeout 900 bash -c '<command>'`, so a
# pattern that matches whole command lines (pgrep -f) finds the command
# looking for it. The client is matched by process name instead.
def client_gone():
machine.succeed("${pkgs.procps}/bin/pkill -x xfreerdp")
machine.wait_until_fails("${pkgs.procps}/bin/pgrep -x xfreerdp")

def connect(n):
machine.succeed(
"DISPLAY=:99 ${stdbuf} ${pkgs.freerdp}/bin/xfreerdp /v:127.0.0.1 /u:rdp /p:1234 "
"/cert:ignore /size:1280x800 /sound:sys:fake /log-level:INFO "
"/log-filters:com.freerdp.channels.rdpsnd.client:DEBUG "
f">/tmp/xfreerdp-{n}.log 2>&1 &"
)
# chansrv sends a client that asked for sound its formats and then a
# training PDU. That line in this connection's own log is what says
# its channel is up - not chansrv's socket, which could be left over
# from the connection before.
machine.wait_until_succeeds(f"grep -q 'Training Request' /tmp/xfreerdp-{n}.log", timeout=600)
print(machine.succeed(f"ls -l {sockdir}"))

def plays(n):
# Noise for half a minute into the session's default sink, and the
# client has to log blocks of it arriving while it plays.
machine.succeed(as_rdp("timeout 30 ${pacat} --format=s16le --rate=44100 --channels=2 < /dev/urandom") + " >/dev/null 2>&1 &")
machine.wait_until_succeeds(sink_running(), timeout=60)
machine.wait_until_succeeds(f"grep -q -E 'Wave2PDU|Wave: cBlockNo' /tmp/xfreerdp-{n}.log", timeout=120)
log = machine.succeed(f"grep -m 5 -E 'WaveInfo|Wave2PDU|Opening device' /tmp/xfreerdp-{n}.log")
print(log)
assert "WAVE_FORMAT_PCM" in log, log
machine.wait_until_fails(sink_running(), timeout=90)

def diagnose():
for cmd in [
"tail -n 30 /tmp/xfreerdp-*.log",
f"ls -la /run/xrdp {sockdir}",
"tail -n 40 /home/rdp/.local/share/xrdp/*.log",
]:
print(machine.execute(cmd)[1])

try:
connect(1)
# The session's startup script loads the xrdp sink and makes it the
# default, on its own schedule: until it has, a stream goes to the
# null sink PulseAudio starts with.
machine.wait_until_succeeds(pactl_says("info", "Default Sink: xrdp-sink"), timeout=120)
plays(1)

# Gone and back. The session outlives the client; the audio has to
# follow the new one. The pause is xrdp's time to notice the first
# client has gone, so the second is a reconnect and not a race.
client_gone()
machine.sleep(5)
connect(2)
plays(2)
except Exception:
diagnose()
raise
'';
}
7 changes: 5 additions & 2 deletions tools/cold-boot.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,12 @@ home=$(nix build --no-link --print-out-paths ".#spotibox-home-vhdx")
vhdx=$(find -L "$system" -name '*.vhdx' -print -quit)
test -n "$vhdx" || { echo "no .vhdx in $system" >&2; exit 1; }

read -r ovmf qemu < <(nix build --no-link --print-out-paths \
# One read per line: `read` fails at an end of input that has no newline, and
# under `set -e` a single read of both paths joined on one line ended the
# script right here.
{ read -r ovmf; read -r qemu; } < <(nix build --no-link --print-out-paths \
--impure --expr 'let p = (builtins.getFlake (toString ./.)).nixosConfigurations.spotibox.pkgs;
in [ p.OVMF.fd p.qemu_kvm ]' | tr '\n' ' ')
in [ p.OVMF.fd p.qemu_kvm ]')

cp "$ovmf/FV/OVMF_VARS.fd" "$work/vars.fd"
chmod +w "$work/vars.fd"
Expand Down
Loading