From 6490e6e3d730517a7d291c11fd09f961f6230362 Mon Sep 17 00:00:00 2001 From: Jussi Elo Date: Wed, 14 Oct 2026 05:08:46 +0000 Subject: [PATCH] fix(e2e): make ComfyUI install/status fixtures run on Windows MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The ComfyUI install-path e2e scenarios never ran on Windows — not because the CLI logic they test is Linux-specific (production already resolves .dll/.so and python.exe/uv.exe paths per host via runtime_is_windows() and friends), but because their test fixtures hardcoded POSIX shell shims. comfyui-01/02/03's fixtures in tests/e2e-cucumber/tests/e2e/comfyui_steps.rs now plant the uv/Python subprocesses they shim as real compiled executables (fake-uv, fake-python), declared as ordinary [[bin]] targets of this crate, the same convention remote_steps.rs's fake-tailscale stand-in already used. Their behavior (exit code, stdout line) is configured through env vars rather than generated per scenario. fake-uv panics on an unparseable FAKE_UV_EXIT_CODE instead of silently defaulting; fake-python rejects an argv shape matching neither of the runtime Python's two invocation forms (probe_torch_stack_versions's -c, probe_comfyui's script form) instead of guessing — both failing loudly, the same rule fake-tailscale already follows. plant_ready_runtime — the Given every comfyui scenario (01 through 13) builds its runtime from — now plants fake-python as that runtime's default Python, instead of the old #!/bin/sh exit 0 stand-in that failed the post-install GPU check. used_comfyui_install (comfyui-05..13's shared Given) used to immediately overwrite that planted Python with its own GPU-check shell shim; that overwrite is now dropped, since fake-python already answers the same post-install probe unconditionally. comfyui-11's GPU-check transition used a dedicated shell-shim helper for both halves (failing, then passing again); the passing-again half now plants fake-python too, like every other scenario, so only the failing half still needs a real shell shim, and the now-single-purpose helper is deleted. Planting a fixture binary goes through rocm_core::fake_executable:: write_fake_executable rather than std::fs::copy, so it does not reopen #649's ETXTBSY race: a plain copy holds a write fd on the destination for the duration of the copy, which is exactly the window a concurrent scenario's spawn can inherit. That helper needs the rocm-core dev-dependency, which this crate's library target cannot reach, so install_fixture_binary() lives in tests/e2e.rs (the test binary that can) rather than alongside the pure fixture_binary()/fixture_target_name() lookups in src/fixture_binary.rs. A .bat script was tried first and rejected: Rust's CVE-2024-24576 mitigation rejects any argument containing a newline once the target resolves to a batch file, and the pre-install torch-stack probe passes its probe script as exactly such an argument — a real compiled executable has no such restriction. The planted rocm_sdk library stubs use a small fixture-local rocm_library_filename() helper instead of importing rocm_core::runtime_rocm_library_filename: readiness only checks path.is_file() on whatever the manifest lists, never the filename, so the import would buy nothing, so it stays a local stand-in. The [[bin]]-locating, suffix-naming, and plant logic is shared between this and remote_steps.rs's fake-tailscale stand-in, which previously duplicated the same lookup-and-copy body privately. tests/e2e-cucumber/ README.md documents the [[bin]] fixture pattern for the next stand-in. @requires-os:linux is dropped from comfyui-01/comfyui-02/comfyui-03 in comfyui.feature, since their fixtures no longer need it. comfyui-04 keeps @requires-os:linux: it drives rocm under a real PTY, the same pre-existing limitation dash.feature's header comment already documents — unrelated to the fixture changes here, now cross-referenced instead of left unexplained. comfyui-05/-06/-07/-09/-10/-13 also lose their shell dependency as a side effect of the plant_ready_runtime change above, but stay @requires-os:linux too: their reinstall behavior has not been run on Windows yet, which is tracked separately in #697 to keep this change scoped to #629's comfyui-01/02/03 ask. No production code changes — apps/rocm/src/comfyui.rs and crates/rocm-core already resolve platform-correct paths; only the test fixtures needed to catch up. Part of #629 (remaining scope tracked in #697) Signed-off-by: Jussi Elo --- tests/e2e-cucumber/Cargo.toml | 11 ++ tests/e2e-cucumber/README.md | 9 ++ tests/e2e-cucumber/features/comfyui.feature | 30 ++-- tests/e2e-cucumber/src/bin/fake-python.rs | 48 ++++++ tests/e2e-cucumber/src/bin/fake-uv.rs | 28 ++++ tests/e2e-cucumber/src/fixture_binary.rs | 47 ++++++ tests/e2e-cucumber/src/lib.rs | 1 + tests/e2e-cucumber/tests/e2e.rs | 28 ++++ tests/e2e-cucumber/tests/e2e/comfyui_steps.rs | 151 ++++++++++-------- tests/e2e-cucumber/tests/e2e/remote_steps.rs | 38 +---- 10 files changed, 277 insertions(+), 114 deletions(-) create mode 100644 tests/e2e-cucumber/src/bin/fake-python.rs create mode 100644 tests/e2e-cucumber/src/bin/fake-uv.rs create mode 100644 tests/e2e-cucumber/src/fixture_binary.rs diff --git a/tests/e2e-cucumber/Cargo.toml b/tests/e2e-cucumber/Cargo.toml index 268fb36bd..fe2265a92 100644 --- a/tests/e2e-cucumber/Cargo.toml +++ b/tests/e2e-cucumber/Cargo.toml @@ -22,6 +22,17 @@ path = "src/bin/rocm-demo-env.rs" name = "fake-tailscale" path = "src/bin/fake-tailscale.rs" +# Stand-ins for `uv` and the runtime's Python interpreter, planted by the +# `rocm comfyui install` scenarios so they run deterministically and offline +# on every platform cargo builds for (see `src/fixture_binary.rs`). +[[bin]] +name = "fake-uv" +path = "src/bin/fake-uv.rs" + +[[bin]] +name = "fake-python" +path = "src/bin/fake-python.rs" + [dependencies] axum.workspace = true cucumber = { version = "0.23", features = ["output-json", "output-junit"] } diff --git a/tests/e2e-cucumber/README.md b/tests/e2e-cucumber/README.md index 6dbc445ab..d4199265d 100644 --- a/tests/e2e-cucumber/README.md +++ b/tests/e2e-cucumber/README.md @@ -238,6 +238,15 @@ The `.feature` file is both the spec and the test input — cucumber reads it at 3. Implement step functions: Given = setup, When = action, Then = assertion. 4. Register the module in `tests/e2e.rs` with `pub mod _steps;`. 5. Run to verify. +6. If a step needs to fake a subprocess the CLI shells out to (`uv`, a runtime's + `python`, `tailscale`), add a `[[bin]]` under [`src/bin/`](src/bin/) rather + than writing a shell/batch script: it builds on every platform cargo already + targets. [`fixture_binary`](src/fixture_binary.rs)'s `fixture_binary()` + locates it and `fixture_target_name()` names it; `install_fixture_binary` in + [`tests/e2e.rs`](tests/e2e.rs) plants it (it needs the `rocm-core` + dev-dependency the library target can't reach). See + [`fake-uv`](src/bin/fake-uv.rs), [`fake-python`](src/bin/fake-python.rs), + and [`fake-tailscale`](src/bin/fake-tailscale.rs) for the pattern. ## Design principles diff --git a/tests/e2e-cucumber/features/comfyui.feature b/tests/e2e-cucumber/features/comfyui.feature index 51cf58663..a7caa87d8 100644 --- a/tests/e2e-cucumber/features/comfyui.feature +++ b/tests/e2e-cucumber/features/comfyui.feature @@ -10,9 +10,8 @@ Feature: ComfyUI install reports progress and makes failures actionable # # The runtime, its rocm_sdk probe, the Python interpreter and `uv` itself are # all planted so the scenario is hermetic: no GPU, no network, no real - # package resolution. Linux-only because the planted shims are POSIX shell - # scripts. - @id:comfyui-uv-install-failure-names-log @requires-os:linux + # package resolution. + @id:comfyui-uv-install-failure-names-log Scenario: comfyui-01 - A failed dependency install names the log it wrote Given a ready ROCm runtime with a ComfyUI checkout pending dependencies And the ComfyUI dependency install with uv fails @@ -32,7 +31,7 @@ Feature: ComfyUI install reports progress and makes failures actionable # both `install` and `status` output -- a user-observable line that AGENTS.md # requires a Gherkin scenario for, not just the unit tests that assert each # command's `folder:` line in isolation. - @id:comfyui-uv-install-progress-streamed @requires-os:linux + @id:comfyui-uv-install-progress-streamed Scenario: comfyui-02 - A dependency install streams progress output Given a ready ROCm runtime with a ComfyUI checkout pending dependencies And the ComfyUI dependency install with uv prints progress and succeeds @@ -60,9 +59,8 @@ Feature: ComfyUI install reports progress and makes failures actionable # # No GPU is needed: runtime readiness is filesystem + manifest state, so the # scenario plants two ready wheel runtimes and asserts the refusal names every - # remediation and lists both keys. Linux-only because the planted rocm_sdk stub - # uses `.so` library names; the selection logic it exercises is platform-agnostic. - @id:comfyui-ambiguous-runtime-actionable @requires-os:linux + # remediation and lists both keys. + @id:comfyui-ambiguous-runtime-actionable Scenario: comfyui-03 - ComfyUI install refuses an ambiguous runtime choice and names every remediation Given two ready ROCm runtimes and no active default When the user installs ComfyUI without choosing a runtime @@ -89,6 +87,11 @@ Feature: ComfyUI install reports progress and makes failures actionable # label-truncation fix in `cli_progress::assemble_status_line` — the # tarball scenario in `download_progress_pty.feature` is the regression # test for that. + # Linux-only: this scenario drives `rocm` under a real PTY, the same + # suite-wide limitation `dash.feature`'s header comment documents + # (portable-pty compiles on Windows via ConPTY, but that path isn't yet + # promoted to a blocking contract) — unrelated to this file's other + # scenarios, whose own Linux gate is explained below. @id:comfyui-source-download-shows-live-progress @requires-os:linux @serial Scenario: comfyui-04 - The source-archive download spinner renders progress and clears on completion Given a paced ComfyUI source archive fixture @@ -109,8 +112,17 @@ Feature: ComfyUI install reports progress and makes failures actionable # started during the download), an interrupted reinstall, a user's entry set # aside, a failure after the new code is in place, and a download that is # not a ComfyUI release. Each asserts what the CLI prints together with what - # is actually on disk afterwards. Linux-only because the planted runtime - # uses `.so` names and a POSIX-shell Python stand-in. + # is actually on disk afterwards. Only three of these still need the POSIX + # shell: comfyui-11's GPU-check failing transition plants a `#!/bin/sh` + # script (its passing-again half uses the Windows-portable `fake-python` + # stand-in, like every other scenario here), and comfyui-08/comfyui-12 each + # start their stand-in running ComfyUI process through `sh -c` (see + # `spawn_stand_in`). comfyui-05/-06/-07/-09/-10/-13 plant the same + # Windows-portable `fake-python` stand-in `comfyui-01`/`-02`/`-03` use and + # have no shell dependency of their own, but stay `@requires-os:linux` too: + # their reinstall behavior has not been run on Windows yet. Ungating them + # is tracked separately (#697), to keep this fix scoped to #629's + # comfyui-01/02/03 ask. # # rocm-cli's saved record of a ComfyUI it started is unreadable here. `stop` # cannot read it either, so it must not block the reinstall; the reinstall diff --git a/tests/e2e-cucumber/src/bin/fake-python.rs b/tests/e2e-cucumber/src/bin/fake-python.rs new file mode 100644 index 000000000..7df1f3e83 --- /dev/null +++ b/tests/e2e-cucumber/src/bin/fake-python.rs @@ -0,0 +1,48 @@ +// Copyright © Advanced Micro Devices, Inc., or its affiliates. +// +// SPDX-License-Identifier: MIT + +//! A stand-in for the runtime's Python interpreter, for the ComfyUI install +//! scenarios' two probes. +//! +//! `apps/rocm/src/comfyui.rs` shells out to the runtime's Python twice: +//! `-c