Repository navigation
Conversation
ubuntu-latest now resolves to Ubuntu 26.04, which ships uutils coreutils. Its /bin/sleep is one multicall binary that picks the tool to run from argv[0], so the decoys made by copying it to .../llama-server print "coreutils: unknown program 'llama-server'" and exit at once. The reclaim self-test then fails three checks and the xtask contract test that runs the real --assert-clear passes a scan it should fail, which turns build-and-test, Coverage and Test (affected crates) red for every run on the new image. Copy bash instead: it ignores the name it runs under, so argv[0] stays the real-shaped path the rule is meant to match. The body is a loop because bash execs a lone `sleep 300` in place, which would put `sleep` back in argv[0]. Its `sleep 1` children name no E2E root, so they are never selected, and SIGTERM still ends the decoy. Verified with the self-test (plain and with TMPDIR redirected) and the xtask test in ubuntu:26.04 and ubuntu:24.04 containers; both failed on 26.04 before this change. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
The delimiter decoy's comment said bash would swallow an extra argument as $0; it does not, and the argument reaches /proc/<pid>/cmdline. It is a separate helper only because spawn_decoy passes no arguments through. Detach the xtask test's decoy from the test's stdio, as the script's decoy already is: its sleep 1 child outlives the decoy's SIGKILL by up to a second and would hold the inherited output open meanwhile. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
juhovainio
left a comment
There was a problem hiding this comment.
Reviewed the decoy-spawn fix for the uutils multicall /bin/sleep breakage.
Traced the actual reclaim-matching logic (cmdline_matches_rule, same_selected_process in reclaim-gpu.sh) against the new bash -c 'for _ in {1..300}; do sleep 1; done' decoy body: the trailing args don't collide with any E2E_ROOTS/ENGINE_MARKERS substring, and the static loop body (no embedded pid/timestamp) keeps pid-recycling detection correct. SIGTERM still kills the bash wrapper immediately since no trap is installed (SIG_DFL disposition), and the orphaned sleep 1 grandchild is already accounted for by the added stdio-detach on the xtask side. A repo-wide grep confirms no other cp /bin/sleep-style decoy was left unfixed.
Diff matches the PR description's claims exactly, scoped to the two call sites, no CI workflow changes, no scope creep.
One non-blocking note: the bash-vs-sleep rationale is restated in prose at three separate comment sites (reclaim-gpu.sh twice, workflow_contract.rs once). Reasonable given the shell/Rust split, not asking for a change.
The two failing checks (Strix Halo Windows E2E download timeout and its downstream consolidated-report failure) look like the known repo.amd.com download-infra flake, not caused by this change.
LGTM, approving.
Why
ubuntu-latestnow resolves to Ubuntu 26.04, which ships uutils coreutils. There/bin/sleepis a single multicall binary that chooses the tool to run fromargv[0]. The GPU reclaim self-test builds its stand-in engine processes by copying/bin/sleepto a path like.../e2e-prewarm/.../llama-server, so on 26.04 every one of them printscoreutils: unknown program 'llama-server'and exits immediately.With no live decoys:
bash scripts/reclaim-gpu.sh --self-testfails three checks (could not read a decoy's command line,pre-warm engine process was not selected,bystander decoy is not running) and refuses to run the real reclaim. That failsbuild-and-testand skips its remaining steps.preflight_runs_the_real_reclaim_assert_clear, which copies/bin/sleepthe same way, sees the real--assert-clearscan pass with "no leaked processes". That failsTest (affected crates)andCoverage.This hits every PR and merge-queue run that lands on a 26.04 runner, whatever the change.
What changes
The decoys are copies of
bashinstead ofsleep. bash ignores the name it runs under, soargv[0]is still the real-shaped path the reclaim rule is meant to match and the decoy keeps the shape a live engine process has. The decoy body is asleep 1loop rather thansleep 300: bash execs a lone simple command in place, which would putsleepback inargv[0]. Thosesleep 1children name no E2E root, so they are never selected, and SIGTERM still ends the decoy. The xtask test's decoy is detached from the test's stdio, as the script's already is, so asleep 1child that outlives the decoy's SIGKILL does not hold the test's output open.Verification
reclaim-gpu.sh --self-test, both plainly and withTMPDIRredirected as CI runs it: fails onmainand passes with this change in anubuntu:26.04container; passes inubuntu:24.04and on a 24.04-style host.preflight_runs_the_real_reclaim_assert_clear: the test binary built frommainfails insideubuntu:26.04and the one from this branch passes; it also passes natively.cargo test -p xtask(313 tests),cargo clippy -p xtask --all-targets -- -D warnings,cargo fmt --checkandshellcheckare clean.Not verified locally: the
build-and-teststeps that run after the self-test (release-readiness self-test, pinned-key check, install-lifecycle e2e). They have been skipped on every 26.04 run so far, so this PR's CI is their first run on the new image.Scope
This changes test fixtures only, so there's no user-observable CLI behaviour and no e2e scenario applies (AGENTS.md §3). It doesn't pin runners back to 24.04; the decoys now work on both images. Every other hosted job already passed on 26.04.
tests/e2e-cucumber/expectations.tomlfor the fixed ticket ID and removed/narrowed any now-stale xfail rows.docs/architecture.md(the clap declaration and dispatch wiring staying inmain.rs/lib.rsis expected, not a violation). (N/A: no new subcommand.)