Repository navigation
fix: read /etc/os-release with one parser that matches sh - #544
Conversation
jussielo-amd
left a comment
There was a problem hiding this comment.
This is excellent work — one of the more careful parser fixes I've seen come through. Routing every reader through one function validated against a live /bin/sh oracle (not just hand-picked fixtures) is exactly the right way to kill "four parsers, four opinions" bugs like #543, and the fail-closed design (one non-assignment line makes the whole file unreadable, loudly, rather than guessing) is well-justified: a shell command can retroactively mutate any variable already read, so there's no safe way to salvage a partial parse.
Ran the full suite locally: rocm-core (474 tests) and rocm (975 tests) pass, clippy and fmt are clean, and the old parse_os_release_field helpers are fully gone with zero dangling references anywhere in the repo. The deferred e2e scenarios are reasonably justified by the #510 dependency and backed by solid integration-level tests in the meantime.
Two small things worth a look, neither blocking:
- The new
reasonstring for an unreadable file embeds the raw offending line verbatim, which is a different register from the other staticreasonstrings nearby (e.g. the Windows/DKMS ones). Worth a glance at whether that's the tone you want here, though the Debug-quoting already handles the control-character safety angle. docs/testing.md/docs/manual-testing.mddon't mention the newprobe_failuresentry orunreadable_os_releasepolicy. Not required since no flag/default changed, but a one-liner would save the next person some head-scratching.
Approving.
ce6c0e7 to
014c92c
Compare
|
Rebased onto Re-applied by hand. Since the approval, main moved code this PR edits:
In both cases the moved copies still called the old parsers, so resolving the conflicts toward main would have compiled with the fix gone. I re-applied the edits at the new locations:
I also deleted the old One new commit, Verified on the new head:
Each re-applied hunk, mutated back to the old behaviour, turns a test red. I re-ran one of them myself: swallowing the unreadable-file error fails On the two notes in the approval:
Heads-up for merge order: #510 will conflict textually with this PR in |
Stale: approved ce6c0e7; head is now 014c92c, after rebasing onto main. #540 and #528 moved the driver plan and the host distro name, so those edits were re-applied at the new locations, and 014c92c adds the requested docs plus a test for the distro name. A fresh look is needed on the current head. Details are in the PR conversation.
siloteemu
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · 014c92c
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — needs work
Full review of the whole change (prw-base...HEAD, merge-base 7f72556d, head 014c92c6).
Description names issue #543 (not retrieved) and claims every reader of /etc/os-release now goes through the new parser, with unreadable files "reported, not hidden", each hunk mutation-checked, and the missing scenario deferred to #510.
Blocking
-
[new] Several readers still hide an unreadable file, and now fail on files they used to accept —
apps/rocm/src/main.rs:6849-6852,apps/rocm/src/main.rs:7024-7027,crates/rocm-core/src/openmpi.rs:831-833,:856-858,:881-883,crates/rocm-core/src/host_gpu.rs:1884-1893and:1897
field()turns an unreadable file intoNone(os_release.rs:96-98), and these callers then useunwrap_or_default(). On a CRLF file or one with a stray non-assignment line, the OpenMPI/torch auto-install paths and the three install hints now print "Could not map distribution (ID=<unknown>)" (openmpi.rs:570-583) and give generic advice. The offending line is never named. The old parserstrim()ed the\rand skipped non-assignment lines, so these paths used to work on such files.
The human-readablerocm examineshowsdistro: Linuxwith no explanation.detect_distro_namefalls back to "Linux" (host_gpu.rs:1884-1893), andprobe_failuresis only reachable through--json.
The description's "An unreadable file is reported, not hidden" and theos_release.rs:55-57doc hold only for the driver plan androcm examine --json. The misattribution to the distro is the same one this change fixes for the driver plan.
Confidence 85 · logic · Fix: useparse()on these paths and put theUnreadableLinein the user-facing reason, or narrow the claims. Say in the description that CRLF and stray-line files lose auto-install. -
[new] No Gherkin scenario for changed user-observable behaviour, and the stated reason does not hold —
tests/e2e-cucumber/features/driver_install.feature,tests/e2e-cucumber/features/examine.feature,AGENTS.md§3
The change alters therocm install driver --dry-runreason and policy, therocm examinedistro fields,probe_failures, and thestatusvalue (degraded, viacompute_statusatexamine.rs:546). Only unit tests cover them.driver_install.featurehas only WSL scenarios.examine-12runs against the real host file and checks only thatrepo_versionhas no${.
AGENTS.md §3 exempts only purely internal changes and gated lanes. "No seam to plant a file" is neither, and commit 4da0340 ends in a broken sentence ("the host-root re-rooting proposed in / against that seam") with the reference missing.
A scenario needs no planted file. On a@requires-os:linuxlane, assert thatrocm examine --jsondistro_idanddistro_versionequal whatshsources from the host's/etc/os-release.
Confidence 85 · architectural · Fix: add that scenario. If the single-quoted and unreadable cases genuinely wait on #510, say so explicitly and fix the commit message. -
[new] Added tests stay green when the code they name is broken —
crates/rocm-core/src/os_release.rs:138-139,:154,:194,:203;apps/rocm/src/driver_install.rs:845-860;apps/rocm/src/main.rs:6850,:7025;crates/rocm-core/src/openmpi.rs:858,:883
I ran each mutation alone and found the suite green every time:- Parser,
cargo test -p rocm-core --lib os_release(17 passed):tilde_expands = false→true(194):a\:~and\a~would readNone, which the oracle allows, and only\~is inMUST_READ.is_control()→is_ascii_control()(154): no test uses DEL or a C1 character.- Removing
'<'(203) or')'(203) from the refusal arm: neither is in the unit lists, and the oracle alphabet leaves them out. is_ascii_alphabetic→is_ascii_alphanumericfor the first name character (138): no test names1A=x, whichshruns as a command.- Accepting
-in names (139): noVERSION-ID=test. - Accepting an empty name (135-140): no
=xtest.
- Unreadable driver plan,
driver_install(57 passed):os_idset to"ubuntu"(848),mutating: true(846),reboot_required: true(859), andchecksemptied (858) all stay green. The test pinsreason,supported,commandsandpolicyonly. An unreadable plan that still claims a distro is the exact contradiction the change exists to prevent. - Callers, no test reaches them: replacing
"ID"with a bad key atmain.rs:6850and:7025leaves all 1027rocmtests green. Mangling the libatomic and libnumaID_LIKElookups atopenmpi.rs:858and:883leaves all 604rocm-coretests green. This contradicts the description's "each hunk … reverting it … turns a test red". examine.rs:546(degraded): the new examine test assertsprobe_failuresbut nevercompute_status(). Thedegradedconsequence is untested and undocumented.
Confidence 100 · mechanical · Fix: add one case per surviving branch. Assertos_id,version_id,codename,mutatingandreboot_requiredin the unreadable-plan test. Add anID/ID_LIKEresolution test through theensure_*and hint paths. Assertcompute_statusafterrecord_distro.
- Parser,
-
[new] Docs and comments describe behaviour the code does not have —
crates/rocm-core/src/os_release.rs:18-19,:22-29,:43-45;docs/testing.md:1292-1306os_release.rs:43-45lists "a space or tab between=and the value" as a refusal. The code acceptsK= #aas"", andMUST_READcontains" #a", which passed in my run.os_release.rs:18-29claimssh's value for every file made only of spec-allowed assignments.K=a\(escaped trailing space) is valid and assigns"a ", but trailing blanks are trimmed before decoding, leaving a dangling\, so the whole file is refused. The test comment at:444-447admits it. The refusal is safe; the doc is wrong.docs/testing.mdsays every reader goes through the module and thatrocm examine"leaves the distro empty and records the same line inprobe_failures". The human report does not, as in the first finding. The WSL-from-Windows probe still sources the file throughshand splits lines itself (examine.rs:830,:909-923).- The section's
cargo test … os_releasefilters do not matchpretty_name_reads_as_sh_reads_it, although the section lists the host distro name as covered. The reader list omitsensure_openmpi_for_vllmandensure_torch_runtime_dep.
Confidence 85 · mechanical · Fix: the code governs in the first two cases, because it is what the oracle tests pin. Correct the module doc and the testing section to match.
Non-blocking
- [new] Oracle tests are gated on
cfg(unix), which includes macOS —crates/rocm-core/src/os_release.rs:484
AGENTS.md §6 says tests that apply only to supported hosts should be gated ontarget_os = "linux"or ignored with a reason. They also spawn/bin/sh, whose behaviour was only verified against dash and bash.
Confidence 70 · mechanical · Fix:cfg(target_os = "linux"). - [new]
ScratchDirreinvents a unique-name helper —os_release.rs:499-508
crate::test_support::unique_suffix()exists (test_support.rs:35) for this.
Confidence 70 · mechanical. - [new]
UnreadableLinehasDisplaybut nostd::error::Error—os_release.rs:64-89
It is theErrof a public fallibleparse, so callers cannot use?intoanyhow.
Confidence 70 · mechanical. - [new] Module doc enumerates callers —
os_release.rs:7-10
That list goes stale as callers change.
Confidence 70 · mechanical. - [new] Test doc comments narrate removed behaviour —
apps/rocm/src/driver_install.rs:1767-1790
"The parser stripped only"" belongs in the commit message.
Confidence 70 · mechanical. - [new]
a_file_that_runs_a_command_never_reads_as_something_sh_does_not_assignpins only "parser returnsNone" —os_release.rs:716-731
It is a regression guard for the fail-closed rule. It should not be counted as coverage of the command-line rules.
Confidence 70 · mechanical.
Decisions for the author
- Whole-file fail-closed versus line-tolerant parsing — tradeoff
For:shis a precise reference, the driver plan runs privileged commands, and os-release(5) forbids the refused forms. Against: systemd's env-file reader treats\ras a newline and Python'splatform.freedesktop_os_releaseskips lines it cannot parse. So a CRLF or hand-edited file that those tools read now blocks the driver plan and hides the distro on every path. A middle option is to strip a trailing\ronly when every line has one, and keep fail-closed for anythingshwould run. Please confirm fail-closed everywhere was deliberate. - Several lines the parser refuses are accepted by
sh(lone$, C1 controls, escaped trailing blank) — tradeoff
These fail closed, within the stated rule. Confirm that over-refusal is the intended side.
Positive signals
- The
/bin/shoracle tests check the parser against the reference itself, so the encoder and parser cannot share a misunderstanding.MUST_READplus theread > len/10bar stops a parser that refuses everything from passing vacuously. - Reading through one shared parser fixes the real driver-plan versus
rocm examinedisagreement on duplicated keys, and the single-quote misread. - The unreadable-file message names the line, 1-based and
Debug-quoted, so control characters are shown rather than acted on.
Deployment notes
- Behaviour change for hosts whose
/etc/os-releasehas CRLF endings or any non-assignment line:rocm install driverplans nothing,rocm examine --jsonreportsstatus: degradedwith an empty distro, and the OpenMPI/torch auto-install paths and hints fall back to generic "unsupported" text. rocm examineJSONstatuscan now bedegradedwhere it wasok.- The OpenMPI hints no longer accept
ID =x.
What this covered
Every file in the diff, read completely: os_release.rs (all 733 lines), the changed regions and surrounding functions in driver_install.rs, main.rs, examine.rs, host_gpu.rs and openmpi.rs, and docs/testing.md. Also read: callers of the new API across apps/, crates/ and engines/vllm, and docs/testing.md against AGENTS.md. Range: merge-base 7f72556d … 014c92c6.
Run: four fan-out workers (parser, callers, source passes, design questions). The description was read after the findings were settled (Step 9). My own runs:
cargo test -p rocm-core --lib os_releaseand the new examine andhost_gputests passed (17 and 2 tests).- The mutation runs listed in the third blocking finding.
- A worker's differential fuzz of a Python port of
parse/decodeagainst dash found 0 mismatches across 3,866 accepted cases. It was not re-run by me and is not a finding.
Per the CI summary supplied with this review, every check that reported on this head passed. "Skill checks" and "Sphinx docs build" did not report, so nothing here rests on them.
Not run: a full-workspace build or clippy, scripts/smoke_local.py, and the Windows lane. History, prior-changes and comment passes ran. Issue #543 was not retrieved. The Gherkin scenarios were not executed.
Not reconciled against any discussion here.
jussielo-amd
left a comment
There was a problem hiding this comment.
Re-reviewed at HEAD 014c92c, which supersedes the dismissed review I left on ce6c0e7 after the 10/06 round. Both follow-up points from that review are resolved: docs/testing.md now documents the /etc/os-release parser, the unreadable_os_release policy, and how to run the tests (docs/testing.md:1292-1304), and the duplicated-key/unreadable-file scenarios I asked about are covered by dedicated tests in driver_install.rs, examine.rs and host_gpu.rs. The reason-string register point I raised was left as-is, which is fine since I'd already called it non-blocking.
Checks performed on this diff (origin/main...HEAD, three-dot): read through crates/rocm-core/src/os_release.rs's decode/parse/ends_cleanly logic by hand against the module docs' stated sh-compatibility rules; ran cargo test -p rocm-core --lib os_release locally (17/17 pass, including the /bin/sh oracle and brute-force encoding tests); confirmed the old parse_os_release_field helper and all call sites are fully removed with no dangling references; ran a leak scan and a conflict-marker scan against the diff (both clean); and confirmed all CI checks on 014c92c are green, including Commit signatures + sign-off and the coverage-floor gate.
Two small follow-up opportunities, neither blocking: (1) the new unreadable-file diagnostic that names the offending line is wired into the driver plan (apps/rocm/src/driver_install.rs) and rocm examine (crates/rocm-core/src/examine.rs), but the three other callers this PR touched — ensure_openmpi_for_vllm and ensure_torch_runtime_dep (apps/rocm/src/main.rs:6850-6851, 7025-7026) and the OpenMPI/libatomic/libnuma hints (crates/rocm-core/src/openmpi.rs:832, 857, 882) — still collapse an unreadable file into the same empty-ID fallback as a merely-absent or unrecognized one, so a user hitting that path doesn't get pointed at the specific malformed line the way driver-install and examine users now do; (2) those same five call sites each parse the full file twice (once per field) via repeated os_release::field(...) calls instead of parsing once and reading both keys off the map, unlike build_driver_install_plan, which already does this correctly. Both are consistency/efficiency nits on an otherwise very solid and thoroughly-tested fix, not correctness issues.
Approving.
|
Rebased onto main The automated review (5488764142):
@jussielo-amd's follow-ups are both in Every
All 22 mutations are caught, including routing Verified locally:
Note: open PR #526 also uses the numbers examine-20 to 22, so whichever lands second renumbers. |
014c92c to
f32fd94
Compare
This approved 014c92c. The head is now f32fd94: a rebase onto main 1e5aac8 (#510), then 3895862, 2c32fb8, d8673df and f32fd94, which address both reviews. The unreadable line is named in the OpenMPI and torch installs and the hints, examine-20/21/22 are added on the host-root seam, and there are tests for the surviving mutants. A fresh approval is needed on the current head.
jussielo-amd
left a comment
There was a problem hiding this comment.
Re-reviewed at head f32fd941 (rebased onto main 1e5aac8f, includes #510). Verified rather than taken on trust — checked out the branch locally and confirmed each claim in the latest commits against the code, not just the description:
- B1 (automated review):
ensure_openmpi_for_vllm/ensure_torch_runtime_dep(apps/rocm/src/main.rs) and the OpenMPI/libatomic/libnuma hints (crates/rocm-core/src/openmpi.rs) now route throughopenmpi::plan_for_os_release/install_hint_for, which embed the unreadable line in the reason instead of silently falling back to an empty ID.id_and_id_likenow parses the file once for both keys, closing my earlier nit about the double parse. - B2:
examine-20/-21/-22intests/e2e-cucumber/features/examine.featurecover the single-quote, duplicate-key and unreadable-line cases against a planted host root. - B3: the unreadable driver-plan test in
apps/rocm/src/driver_install.rsnow assertsos_id/version_id/codename/mutating/reboot_required/checksexplicitly, closing the mutation gaps the automated review listed. - B4:
docs/testing.mdnow documents every reader, therocm diagnose --distroexception, and how to run the new scenarios.
Ran cargo test -p rocm-core --lib os_release (19/19, including the /bin/sh oracle), openmpi:: (28/28) and driver_install:: (57/57) locally against this head — all green.
Approving.
f32fd94 to
b761c4d
Compare
|
Rebased onto main Verified locally:
|
jussielo-amd
left a comment
There was a problem hiding this comment.
Re-reviewed at head b761c4dc (rebased onto main 2844d797 after #599's lib.rs split and #625). My previous approval was dismissed automatically by the force-push, so I wanted to be sure that was just bookkeeping and not something worth a fresh look.
It was. I pulled both this head and the one I last approved (f32fd941), diffed each against its own base, and compared the two patches with line numbers stripped out — they're identical. The only change anywhere is in crates/rocm-core/src/lib.rs, where pub mod os_release; moved to sit next to the new net/paths/process modules; everything else in the fix is untouched.
This has had a thorough couple of rounds — the automated review's findings, the two rebases to keep up with a moving main, the new examine scenarios. Nice work keeping it all straight through that. Approving again, happily.
siloteemu
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · b761c4d
This automation never files a GitHub approval, so no approving review will
appear here whatever the outcome — the merge decision stays with a human
reviewer.
Review — needs work
Full review of the whole change.
Fixes #543 ("/etc/os-release is parsed four different ways, and they disagree on spec-valid files"). The change does what that issue asks: every reader now goes through one parser, and an unreadable file is reported by the line that makes it so.
Blocking
-
[new] The PR description no longer matches the diff — PR description, "Scenarios" and "The fix" sections
The description says: "There is no e2e scenario in this PR … Scenarios for a single-quoted file and a duplicatedID, throughrocm install driver --dkms --dry-run, … follow once it lands."- The diff adds three scenarios, examine-20 to examine-22, and they go through
rocm examine --json, not the driver plan. - The new comment in
examine.featuresays the driver plan "cannot be exercised this way". That contradicts the promised driver scenarios. - The description also leaves out:
- the libatomic and libnuma plans and hints, which now use the shared parser too;
- the new
unreadable_os_releasedriver-plan policy value; - the
distro: Linux (/etc/os-release line N …)human-readable output; - the new
docs/testing.mdsection.
AGENTS.md §3 asks the PR text to name the
@id:s that cover user-observable behaviour and to justify any missing scenario.
Confidence 85 · mechanical · Fix: update the description to match the diff. Name@id:examine-reads-single-quoted-os-release,@id:examine-reads-last-os-release-assignmentand@id:examine-reports-unreadable-os-release, and give the real reason there is no driver-plan scenario. - The diff adds three scenarios, examine-20 to examine-22, and they go through
-
[new] No test catches an
ID_LIKEregression in the new plan and hint helpers —crates/rocm-core/src/os_release.rs:109-115,crates/rocm-core/src/openmpi.rs:857-860, tests atcrates/rocm-core/src/openmpi.rs:1140-1170
I ran two mutations and the wholeopenmpi::andos_releasetest set stayed green for both:- swapping
IDandID_LIKEinid_and_id_like; - changing
install_hint_for'sOkarm tobuild(&os_id, ""), which dropsID_LIKEfrom the hints.
Why the tests miss it:
- The only derivative case uses a single-token
ID_LIKE='fedora', andresolve_package_manageraccepts either argument for a single token. - The hint assertion is only
starts_with("install it with")onID=ubuntu`.
Real impact: a derivative with a multi-token
ID_LIKE(for exampleID=foo,ID_LIKE="rhel fedora") would plan nothing under the first mutation. Under the second, every derivative would get the manual hint and no command.
Confidence 100 · logic · Fix:- Assert
id_and_id_like("ID=rocky\nID_LIKE=\"rhel fedora\"\n")returns the two values in order. - Add a multi-token derivative case to the plan and hint tests.
- Assert the hint equals the joined plan commands rather than a prefix.
- swapping
-
[new] An indented comment line is not tested, so a parser that refuses it stays green —
crates/rocm-core/src/os_release.rs:145-146
The module doc promisessh's value for any file made of blank lines,#comments and spec-valid assignments, and leading spaces and tabs are allowed.
I changed the comment check toraw_line.starts_with('#'), so an indented comment makes the whole file unreadable. All 133os_release,pretty_name,openmpi::andexaminetests still passed.
Theshoracle cannot catch this: it acceptsNoneeverywhere, andMUST_READcovers single right-hand sides only.
Confidence 100 · logic · Fix: add" \t# comment\nID=ubuntu\n"expectingSome("ubuntu"). Also consider an indented invalid line, asserting thatUnreadableLine.textkeeps the indentation. -
[new] A test comment claims to catch something its assertions cannot fail on —
crates/rocm-core/src/examine.rs, doc onos_release_reads_follow_the_simulated_host_root
The comment says "one that bypassed the parser would keep the quotes." For the distro-name read that is false.- The old
PRETTY_NAMEreader also stripped single quotes. - I put the old lenient
PRETTY_NAMEparsing back in front ofdistro_name_from_os_release, and this test still passed under--features e2e-test-hooks. - The shared parser for the name is in fact covered by
pretty_name_reads_as_sh_reads_it. The problem is that this comment points readers at the wrong test.
Confidence 100 · logic · Fix: limit the claim to routing through the host root plus the quote handling of
distro_id/distro_version. Or plant aPRETTY_NAMEthe old reader got wrong, such as a repeated key. - The old
-
[new]
os_release::fieldis dead code —crates/rocm-core/src/os_release.rs:117-124
It ispub, but a workspace grep finds callers only in this module's tests. Production code callsparse(driver plan,examine,host_gpu) andid_and_id_like(openmpi).
The module doc justifies it as being "for a caller that only needs the value", and no such caller exists.
Confidence 85 · mechanical · Fix: make it#[cfg(test)](or a test helper), or drop it and update the doc sentence. -
[new]
docs/testing.mdputs the unreadable line in the wrong place for the install hints —docs/testing.md("The OpenMPI, libatomic and libnuma installs and hints" bullet),crates/rocm-core/src/openmpi.rs:857-862
The doc says "no commands, and the line ahead of the manual-install advice."- That is true of
plan_for_os_release's reason. install_hint_forreturns"{manual}; /etc/os-release {line}, so the distribution cannot be read", so in the hints the line comes after the advice.- The code governs: the function's own doc comment ("followed … by the line") and its test both pin that order.
Confidence 85 · mechanical · Fix: say "ahead of" for the plans and "after" for the hints.
- That is true of
-
[new] A CI comment is now wrong because of this PR —
.github/workflows/ci.yml:539-556
The comment on "Test rocm-core with e2e-test-hooks" says the step covers "the planted-root probe test inexamine.rs" (one test). It also lists/etc/os-releaseamong reads that "have no planted-root unit assertion".
This PR addsos_release_reads_follow_the_simulated_host_rootto that step's run, and it asserts both/etc/os-releasereads.
Confidence 85 · mechanical · Fix: move/etc/os-releaseinto the list of sampled reads and mention the new test. -
[standing] The description still does not say what an unreadable file costs — PR description, "An unreadable file is reported, not hidden" and "CRLF files are unreadable too"
Standing since the review at 014c92c (first blocking finding there), not re-found blind this round. Settled by putting the objection to this round's reviewer after its report was frozen; that verdict is anchored, not blind. The code part no longer holds: every reader now names the line. The description part still holds. It says CRLF files are unreadable but not that, on a CRLF file or one with a stray non-assignment line, the driver plan and the OpenMPI, libatomic and libnuma auto-installs now plan nothing where the old readers worked. Its "reported" sentence still names only the driver plan andprobe_failures.
Fix: state that consequence in the description, and list every place the line is now reported. -
[standing] The PR text does not justify the missing scenarios as AGENTS.md §3 requires — PR description "Scenarios" section,
crates/rocm-core/src/examine.rs:546-557, AGENTS.md §3
Standing since the review at 014c92c (second blocking finding there), not re-found blind this round. Settled the same way; anchored, not blind. Therocm examinedistro fields andprobe_failuresare now covered by examine-20 to examine-22, and the commit-message sentence is fixed. Two parts still hold:- The reason the driver plan's
--dry-runreason andpolicy: unreadable_os_releasehave no scenario is stated in the code andexamine.feature, but not in the PR text, which instead promises driver-plan scenarios. - The
degradedstatus has no scenario and no named gated lane. The planted host root has no GPU, so examine-22 lands onno-amd-gpu, and only a unit test callscompute_status. §3 asks for a scenario, or for the PR text to name the gated lane that will exercise it.
Fix: put the driver-plan justification in the PR text, and either add a GPU-lane scenario for
degradedor name the lane and why it is not covered here. - The reason the driver plan's
Non-blocking
-
[new] The vLLM warning still offers
--yeswhen the release file is unreadable —engines/vllm/src/runtime.rs:162-178,engines/vllm/src/process.rs:206-236
These callers wrap each hint as"{hint}, or rerunrocm engines install vllm --yes.".- For an unreadable file the plan is unsupported, so
ensure_openmpi_for_vllmandensure_torch_runtime_deponly warn, and--yesinstalls nothing. - The same wording was already misleading for unknown distros. This PR adds a case where the CLI knows it cannot install and still suggests it.
Confidence 70 · logic · Fix: for the unreadable case, tell the user to fix the named line rather than rerun with
--yes. - For an unreadable file the plan is unsupported, so
-
[new] The rewritten hint functions are not tested for which builder and manual text each one uses —
crates/rocm-core/src/openmpi.rs:901-955
libatomic_install_hintandlibnuma_install_hintare now one-line calls intoinstall_hint_for. Swapping their builder or*_MANUAL_HINTargument would keep every test green, because theBUILDERStable in the tests is separate from production code. This wiring was untested before the PR too.
Confidence 70 · logic · Fix: test the three public wrappers against fixed text, or have production code and tests share one table.
Decisions for the author
- One unusual line makes the whole file unreadable — tradeoff
- For: after a line
shwould run as a command, no earlier value can be trusted. Planningsudopackage commands from a valueshwould not assign is worse than planning nothing. - Against: a file the old readers handled now blocks
rocm install driverand the OpenMPI/libatomic/libnuma auto-installs. Examples are a CRLF file, or one stray line such asANSI_COLOR=0;31orID =x. The message names the line but gives no fix (for example "quote the value") and there is no override. - The description's check against 396 of 397 public os-release files suggests mainstream distros are unaffected; hand-edited and custom images are the exposure.
- Confirm the fail-closed scope is what you want.
- For: after a line
Positive signals
- The
/bin/shoracle runs every spec-valid encoding, and every right-hand side up to three characters over a hostile alphabet, against a real shell. It runs with an empty environment, a sentinelHOMEand aPATHthat does not exist. That compares the parser to its stated reference, not to the author's own idea of the format. - An unreadable file is reported as itself everywhere: the driver-plan policy,
probe_failures, and the humandistro:line. The reported line isDebug-quoted, so control characters in it cannot reach a terminal. The old behaviour blamed an unsupported distro. - The install plans keep reading the real
/etc/os-releaseand the descriptive reads follow the simulated host root. That split is stated consistently in code comments, the feature file anddocs/testing.md.
Deployment notes
rocm install driverJSON output can now carry a newpolicyvalue,unreadable_os_release, withsupported: falseand no commands. Anything that matches onpolicyshould expect it.- Hosts with a CRLF or otherwise non-plain
/etc/os-releasewill no longer get automatic OpenMPI, libatomic, libnuma or driver plans.rocm examineon those hosts reportsdegraded, unless the host is WSL or has no AMD GPU.
What this covered
What was read: every file in prw-base…b761c4dc (ee3131be…b761c4dc), all 6 commits. This includes os_release.rs in full, the consumers in openmpi.rs, driver_install.rs, main.rs, examine.rs and host_gpu.rs, the e2e feature and steps, and docs/testing.md. Outside the diff I also read the hint callers in engines/vllm, report.rs's use of distro_id, and hardware_root.rs.
What was run:
cargo test -p rocm-core --libfiltered toos_release,pretty_name,openmpi::andexamine: 133 passed.- The same crate with
--features e2e-test-hooks, the host-root test: passed. cargo test -p rocm --bin rocmfiltered toos_releaseanddriver_plan: 30 passed.- Four single-branch mutations, three reported above as surviving plus the
ID_LIKEswap.
What did not run, and why:
- The e2e scenarios were not run locally. They were judged by reading the code: each of examine-20 to examine-22 fails against the old code. For examine-21, only the
summary/distroassertion makes it fail, because the oldexamineloop already kept the lastID. - The Step 7 design questions were worked through by the driver, not by independent workers.
- The prior-review-comments pass did not run, by design.
- CI on this head:
- "E2E tests (MI300X)" was still queued, so it has not reported.
- "Skill checks" and "Sphinx docs build" were skipped. The Sphinx tree is
docs/rocm-docs, which the PR does not touch, and no skill files changed. - Every other check passed.
Discussion: not reconciled against any PR discussion here.
Standing objections from the review at 014c92c
That round filed a change request with four blocking findings; it has since been dismissed. Each was put to this round's reviewer after its report was frozen (anchored evidence, not blind). Two still hold in part and are listed under Blocking as standing. The other two no longer hold and are withdrawn: every mutant listed in the third finding is now killed by a named test, and all four documentation points in the fourth finding are fixed.
Four places parsed `/etc/os-release` — the driver plan (which also picks the distro for the package installs it approves in `ensure_openmpi_for_vllm` and `ensure_torch_runtime_dep`), `rocm examine`, the OpenMPI install hint, and the `PRETTY_NAME` reader — each with its own `trim_matches`. Against `sh` sourcing the same file, they disagreed with each other: file sh driver plan rocm examine OpenMPI hint ID='ubuntu' ubuntu 'ubuntu' 'ubuntu' ubuntu ID=debian, then ID=ubuntu ubuntu debian ubuntu debian `os-release(5)` allows single quotes, so a spec-valid single-quoted file read as an unsupported distro. And on a file with a duplicated `ID`, `rocm examine` reported Ubuntu while the driver plan was built for Debian 12. All four now read through `rocm_core::os_release`, which keeps one rule: the value `sh` would assign, or nothing. For a file of blank lines, comments and spec-valid assignments it returns `sh`'s value — one matching quote pair removed, `\"`, `\\`, `` \` `` and `\$` decoded inside double quotes, nothing decoded inside single quotes, `\x` meaning `x` in a bare value, a `#` comment after a value, the last assignment of a key winning. Any other line makes the whole file unreadable, not just the key it names. To `sh` such a line is a command or the start of one — `export ID=…` and `X=1; ID=…` assign, `unset ID` removes, an unterminated quote swallows the lines after it — and a command can change any variable, including ones set before it. Reading past it would mean reporting values `sh` does not assign. A CRLF file is unreadable too: `sh` assigns `ubuntu\r`, which matches no distro. An unreadable file is reported, not hidden. `parse` names the first offending line, and the driver plan's reason becomes `/etc/os-release line N is not a plain assignment: "<line>"; no driver commands were planned because the distro cannot be read.` instead of "this distro is not supported", while `rocm examine` records the same line in `probe_failures`. Behaviour changes: - `rocm examine` reports single-quoted and duplicated keys as `sh` reads them, and an unreadable file as a probe failure naming its line. - The driver plan, and the package installs it approves, read the same distro `rocm examine` reports. - The OpenMPI hint no longer accepts `ID =x`; to `sh` that runs a command named `ID`. Real files: run against `dash`, all 50 container images checked and 396 of the 397 files in which-distro/os-release (10dfd5bc) read exactly as `dash` reads them; the one exception is a CRLF file, which reads as unreadable. No file read a value `dash` does not assign. Tests: - Against `/bin/sh` itself (empty environment, a sentinel `HOME`, a `PATH` that does not exist): every spec-valid encoding of every value up to three characters, and every right-hand side up to three characters over a quote-, backslash- and newline-heavy alphabet, checked for the key it sets and a key after it. Each must read as `sh` reads it or not at all, a list of well-formed cases must read as a value, and no earlier value may come back. - Nine mutations of the parser's rules each turn the suite red. - End to end through the driver plan: every supported distro rewritten in single quotes plans exactly as in double quotes; a duplicated `ID` plans from its last assignment; an unreadable file is refused with the message above and no commands. `rocm examine`'s probe failure is asserted with its state. These fail on the code before this change. No e2e scenario in this commit: a later commit in this change plants /etc/os-release under the simulated host root #510 added, and runs `rocm examine --json` against it. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
The PRETTY_NAME reader moved to host_gpu.rs on main, and reverting it to its old line scan left every test green: the one fixture it had reads the same through both parsers. Add a case the two disagree on (a duplicated key with a trailing comment, and an unreadable file), and name it in the module's list of readers. Document in docs/testing.md what an unreadable /etc/os-release does to the driver plan and to `rocm examine`, and how to run the tests. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Only the driver plan and `rocm examine --json` said which line made /etc/os-release unreadable. The OpenMPI, libatomic and libnuma installs `rocm` runs before serving, and the three install hints naming them, fell back to "Could not map distribution (ID=<unknown>)" or generic advice, blaming the distribution for a fixable line. The human `rocm examine` printed `distro: Linux` with no reason. - `rocm_core::openmpi::plan_for_os_release` builds those plans from the file: an unreadable one plans nothing, and the reason names the line ahead of the manual-install advice. `ensure_openmpi_for_vllm` and `ensure_torch_runtime_dep` use it, and the hints share the same lookup, which now appends the line to the manual advice. - `os_release::id_and_id_like` is the one place `ID` and `ID_LIKE` are looked up for those plans, parsing the file once rather than per key. - The distro name `rocm examine` prints is `Linux (/etc/os-release line N is not a plain assignment: "...")` for an unreadable file. - `UnreadableLine` implements `std::error::Error`. The plans and hints still read the real /etc/os-release, not the simulated host root: their commands run on the real machine. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
`rocm examine` reads /etc/os-release through `rocm_core::host_path` since #510, so a scenario can describe a machine other than the runner. Three scenarios run `rocm examine --json` against a planted file: - examine-20: every value single-quoted reads as RHEL 9.4. - examine-21: a file naming Debian 12 and then Rocky Linux 9.5 reads as Rocky, the last assignment, for the distro fields and the name alike. - examine-22: a file with `export ID=debian` on line 3 reports no distro, and names the line both in `probe_failures` and in the distribution name a person reads; message and state in one scenario. The planted distros are ones no runner here has, so a probe that read the real file would report a different one. The driver plan is not covered this way: it builds commands for the real machine, so it reads the real file by design. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Mutations a review ran each left the suite green. Each now fails a test: - The parser: `~` after an escaped character or an escaped `:` (`\a~`, `a\:~`) must read as itself; `<` and `)` in a bare value, a name that starts with a digit, contains `-` or is empty, and DEL and C1 controls must make the file unreadable. - The unreadable driver plan: no `os_id`, `version_id` or `codename`, not mutating, no reboot, no preflight checks, and `rocm examine` as the one check. - `rocm examine`: an unreadable file makes the examination `degraded`, a readable one leaves it `ok`. The module doc listed "a space or tab between `=` and the value" as a refusal, but `K= #c` assigns the empty string; it now says anything after the value but blanks and a comment is refused. It also claimed `sh`'s value for every spec-valid file, though `K=a\ ` is refused; that exception is stated. The caller list in the module doc is gone, since it goes stale. docs/testing.md now names every reader, the `rocm diagnose --distro` probe that has the distribution's own `sh` source the file, what each reader shows for an unreadable file, and the new scenarios. The `/bin/sh` oracle tests are Linux-only, and name their scratch directories with `test_support::unique_suffix`. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
The planted-root test from #510 covers the KFD, WSL and container probes but not /etc/os-release, so reverting either of `rocm examine`'s two os-release reads (the distro fields and the distro name) to the real path left every unit test green; only the e2e scenarios noticed. Plant a single-quoted file naming a distro no runner has, and assert both reads report it, which also fails if either read skips the shared parser. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
…fix the line vLLM's missing-runtime messages offered `rocm engines install vllm --yes` even when the plan that command runs had no commands, as it has for an unreadable /etc/os-release (and for an unknown distribution). The hints now return whether their plan installs anything, read from the same plan_for_os_release the install runs, and the offer follows from that. An unreadable line now comes with what to change on it — LF line endings, single quotes around the value, or a NAME=value line — in the driver plan's reason, the package plans' reasons and the hints. Tests close the gaps mutants survived: ID and ID_LIKE order with a multi-word ID_LIKE, a derivative's hint equal to its plan's commands, the three hint wrappers' builder and manual text, an indented comment, and an indented unreadable line keeping its indentation. os_release::field had no production caller and is now a test helper. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
The host-root test's doc said a reader bypassing the parser would fail it, which was false for the distro name: the old PRETTY_NAME reader also stripped single quotes. A repeated PRETTY_NAME, which the old reader read as the first and sh as the last, makes the claim true. The CI step's comment now lists /etc/os-release among the sampled reads. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
examine-23 plants a GPU in the simulated host's kernel topology next to an unreadable /etc/os-release, and puts an lspci listing an empty bus on PATH so a runner without lspci adds no failure of its own. The verdict is then `degraded` with the release-file line as its only failure, on every Linux lane with or without a GPU. Without the GPU the verdict is `no-amd-gpu` whatever the file says, which is why examine-22 could not show it. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
b761c4d to
ac3c4d0
Compare
|
Rebased onto main This push answers the automated review of Blocking
Non-blocking
Fail-closed scope: kept. After a line Verified locally on
examine-23 has not run on a GPU lane yet. |
jussielo-amd
left a comment
There was a problem hiding this comment.
Re-reviewed at head ac3c4d03 (rebased after #653 moved the os_release readers and ensure_* callers into runtime_install_finalize.rs). I checked this round's claim against the diff directly rather than taking it on trust: every blocking and non-blocking finding from the automated review at b761c4dc is genuinely fixed in the code —
id_and_id_like_come_back_in_that_ordercovers the multi-wordID_LIKEcasean_indented_comment_is_a_comment/an_indented_unreadable_line_keeps_its_indentationcover the indentation handlingos_release::fieldis now test-only, moved insidemod testsdocs/testing.mdand theci.ymlcomment both match the actual line-order/coverage behaviorexamine-23(@id:examine-degrades-on-unreadable-os-release) coversdegradedon a GPU host with an unreadable release file- the vLLM
--yesoffer (the_automatic_install_is_offered_only_when_its_plan_installs) and the hint wiring (each_hint_names_its_own_runtime) are both now pinned by tests
The PR description is also rewritten to cover what an unreadable file costs and why there's no driver-plan scenario, closing out the two standing findings from the earlier round.
The one red check, "E2E tests" (engine-shell-marks-the-prompt), is a pre-existing test-harness flake unrelated to this PR's changes — #663 explicitly names #544 as one of the PRs hit by it and has a fix in flight. Every other check is green; MI300X was still queued at review time.
Approving again.
…child-pipes #544 (313a640) added examine-20..23 for the os-release parser, colliding with this branch's examine-20..22 for the GPU-telemetry drain scenarios. The @id: tags were always distinct, so only the display indexes clashed; the feature_naming guard numbers scenarios positionally, so this branch's three renumber to examine-24/25/26 behind main's, keeping their order. Every cross-reference moved with them: the feature-file comments, the examine_steps.rs doc comments, and rocmd's `Pinned by examine-21` note. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
Fixes #543.
/etc/os-releasewas parsed in four places, each with its owntrim_matches. Compared withshsourcing the same file, they disagreed with each other:shrocm examineID='ubuntu'ubuntu'ubuntu''ubuntu'ubuntuID=debianthenID=ubuntuubuntudebianubuntudebianSo a spec-valid single-quoted file read as an unsupported distro. On a file with a duplicated
ID,rocm examinereported Ubuntu while the driver plan was built for Debian 12.The fix
Every reader now goes through
rocm_core::os_release. That covers the driver plan; the OpenMPI, libatomic and libnuma install plans and the hints that name them;rocm examine'sdistro_id/distro_version; and the distro name it prints. The module keeps one rule: the valueshwould assign, or nothing.sh's value. That means:\"\\\`\$decoded inside double quotes;\xmeaningxin a bare value;#comment after a value;shsuch a line is a command or the start of one:export ID=…,X=1; ID=…,unset ID, or an unterminated quote that swallows the lines after it. A command can change any variable, so reading past it would report valuesshdoes not assign. A CRLF file is unreadable too:shassignsubuntu\r.What an unreadable file costs, and where it is reported
This is fail-closed. On a CRLF file, or one with a single stray line (
ANSI_COLOR=0;31,ID =x), these now plan nothing where the old readers worked:rocm engines install vllm, and the engine step ofrocm install sdk).Against public files this is rare (see Real files), but hand-edited and custom images are exposed. There is no override; the fix is to edit the line.
The line is reported, as
line N is not a plain assignment: "<line>"(Debug-quoted so control characters cannot reach a terminal), in every place that uses the distro:rocm install driver:policy: unreadable_os_release,supported: false, no commands, and the line inreason. Anything matching onpolicyshould expect the new value.detail:.rocm engines install vllm --yes, because that install would plan nothing. The offer now comes from the same plan the install runs, so it is also dropped for an unknown distribution, where it was already misleading.rocm examine:distro: Linux (/etc/os-release line N …). With--json, also an emptydistro_id/distro_version, the same line inprobe_failures, andstatus: degradedon an otherwise healthy GPU host.Wherever the user has to act (driver reason, plan reasons, hints), the message also says what to change, judged from the whole line: save the file with LF line endings; put that line's value in single quotes; or remove that line or rewrite it as
NAME=value.Real files
Checked against
dash: all 50 container images, and 396 of the 397 files in which-distro/os-release at10dfd5bc, read exactly asdashreads them. The one exception is a CRLF file, which reads as unreadable. No file read a valuedashdoes not assign.Tests
/bin/shitself. The shell runs with an empty environment, a sentinelHOMEand aPATHthat does not exist. Inputs: every spec-valid encoding of every value up to three characters, and every right-hand side up to three characters over a hostile alphabet. Each must read asshreads it or not at all, and a list of well-formed cases must read as a value. Linux-only.ID_LIKE, and the fix advice for each kind of line.ID_LIKEderivative plans, and its hint equals the plan's commands exactly. Each of the three hints uses its own builder and manual text. The--yesoffer is checked together with whether the plan installs.driver_install.rs):a_single_quoted_os_release_builds_the_same_plan);IDplans from its last assignment (a_duplicated_os_release_key_is_planned_from_its_last_assignment);an_unreadable_os_release_names_its_line_and_plans_nothing).os_release_reads_follow_the_simulated_host_root(withe2e-test-hooks) checks that bothrocm examinereads answer from a planted root through the shared parser. The planted file is one the old readers misread.Scenarios
In
tests/e2e-cucumber/features/examine.feature, all@requires-os:linuxand all planting/etc/os-releaseunder the simulated host root:@id:examine-reads-single-quoted-os-release(examine-20): a single-quoted file is read asshreads it.@id:examine-reads-last-os-release-assignment(examine-21): the last of a repeated key wins.@id:examine-reports-unreadable-os-release(examine-22): an unreadable file names line 3 inprobe_failuresand in the distro name.@id:examine-degrades-on-unreadable-os-release(examine-23):status: degraded, with the release-file line as the only failure. The planted root has a GPU in its kernel topology, and a stand-inlspcion PATH lists an empty bus, so this runs on every Linux lane with or without a GPU. Without the planted GPU the verdict would beno-amd-gpuwhatever the file says.There is no scenario for the driver plan, the package plans, the hints, or vLLM's messages. They deliberately read the real
/etc/os-release(read_os_releaseinruntime_install_finalize.rs, andreal_os_releaseinopenmpi.rs), because their commands run against the real machine and a simulated host must not choose their distro. A scenario could only exercise them by rewriting the runner's own/etc/os-release. The unit tests above cover them, each asserting the message together with the plan it describes.Docs
docs/testing.mdhas a new/etc/os-releasesection: what each reader reports for an unreadable file, the--yesrule, and the test commands.docs/release-trust.md's Simulated Host Root section is updated to match./etc/os-releaseamong the sampled reads.Verification
On the current head, rebased onto
ee3131be:cargo fmt --checkrocm-core --lib(651 passed), pluse2e-test-hooks(6 passed)rocm --bin rocm(1150 passed), vllm (98), xtask (252)--lib(142),feature_naming(4) andcheck --test e2ecargo xtask e2e -- -n examine-2(4 scenarios, 15 steps passed)-D warningsNot run locally: examine-23 on a GPU lane (no GPU here), and the Windows lane.