Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 13 additions & 6 deletions scripts/reclaim-gpu.sh
Original file line number Diff line number Diff line change
Expand Up @@ -452,13 +452,21 @@ assert_clear() {

# Spawn a decoy whose argv[0] is a real-shaped path, so the match is tested
# against the same string a live engine process would present.
#
# The executable is a copy of bash, not of sleep: where coreutils is a multicall
# binary (uutils, the default since Ubuntu 25.10), /bin/sleep picks the tool to
# run from argv[0], so a copy named llama-server prints "unknown program" and
# exits before the self-test can see it. bash ignores the name it runs under.
# The body is a loop, not `sleep 300`, because bash execs a lone simple command
# in place and argv[0] would become `sleep`. Its `sleep 1` children name no E2E
# root, so they are never selected, and a TERM still ends the decoy.
spawn_decoy() {
local path="$1"
mkdir -p "$(dirname "${path}")"
cp /bin/sleep "${path}"
cp "$(command -v bash)" "${path}"
# Detach stdio: this runs inside a command substitution, and a background
# child holding the capture pipe open would block the caller until it exits.
"${path}" 300 >/dev/null 2>&1 &
"${path}" -c 'for _ in {1..300}; do sleep 1; done' >/dev/null 2>&1 &
echo $!
}

Expand Down Expand Up @@ -504,7 +512,7 @@ spawn_zombie() {
}

# Spawn a decoy that IGNORES SIGTERM, so the TERM -> grace -> KILL escalation is
# exercised. A plain `cp /bin/sleep` decoy dies on the first TERM and leaves the
# exercised. A plain spawn_decoy decoy dies on the first TERM and leaves the
# escalation branch unreached, which is how it went untested.
spawn_stubborn_decoy() {
local path="$1"
Expand All @@ -524,9 +532,8 @@ DECOY
# Spawn a decoy carrying the record delimiters INSIDE one argument — a newline
# and a tab, as a `--chat-template` or `--prompt` value does.
#
# Not spawn_decoy with an extra argument: that decoy is `cp /bin/sleep`, and
# sleep rejects a non-numeric argument and exits before it can be observed. This
# needs something that ignores what it is handed and stays alive.
# A helper of its own because spawn_decoy passes no arguments through. Like that
# decoy, it ignores what it is handed and stays alive.
#
# Ordinary in every other respect — it must die on SIGTERM, so it is the stubborn
# decoy's body without the trap. Its `sleep 1` children name no E2E root, so they
Expand Down
18 changes: 14 additions & 4 deletions xtask/src/workflow_contract.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2897,12 +2897,22 @@ fi

// argv[0] names an E2E root and an engine marker, which is what the
// rule matches, and sits under the scope so nothing else can.
//
// A copy of bash running a loop, for the reasons `spawn_decoy` in
// reclaim-gpu.sh gives: a copied multicall `sleep` exits at once as an
// unknown program, and bash would exec a lone `sleep 300` in place.
let decoy_dir = scope.path().join("e2e-prewarm/bin");
std::fs::create_dir_all(&decoy_dir).expect("decoy dir");
let decoy = decoy_dir.join("llama-server");
std::fs::copy("/bin/sleep", &decoy).expect("copy /bin/sleep as the decoy");
std::fs::copy("/bin/bash", &decoy).expect("copy /bin/bash as the decoy");
let mut child = std::process::Command::new(&decoy)
.arg("300")
.args(["-c", "for _ in {1..300}; do sleep 1; done"])
// Detached, as in the script: a `sleep 1` child outlives the
// SIGKILL below by up to a second and would hold the test's
// inherited stdio open meanwhile.
.stdin(std::process::Stdio::null())
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.spawn()
.expect("spawn the decoy engine process");
let decoy_pid = child.id();
Expand All @@ -2913,8 +2923,8 @@ fi
&env,
&reclaim,
);
// Reaped before any assertion: a panic here must not leave a `sleep
// 300` behind on a self-hosted runner, of all things.
// Reaped before any assertion: a panic here must not leave a 300s
// decoy behind on a self-hosted runner, of all things.
let _ = child.kill();
let _ = child.wait();
let (code, log) = got;
Expand Down
Loading