From 8852572fdacac55a4245b8923e4847a2845f999b Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 8 Oct 2026 18:10:49 -0400 Subject: [PATCH 01/18] docs: call out supported ROCm versions (7.14, 10.0, 10.1) State the default ROCm version and the supported versions in the ROCm installation intro and Getting started, and move the "ROCm 10 and newer" section ahead of driver installation. Explain that ROCm 10 or newer needs a `--version` pin and the exact GPU arch as a raw gfx code via `--family`. Signed-off-by: pmoutsias-amd --- README.md | 55 +++++++++++++++++-------------- docs/rocm-docs/getting-started.md | 12 +++---- 2 files changed, 37 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index ab0c4b1ce..7063b11a6 100644 --- a/README.md +++ b/README.md @@ -203,11 +203,17 @@ rocm install sdk This downloads TheRock ROCm wheels and a matching PyTorch stack into a managed environment. On machines with an existing ROCm install, `rocm examine` will show it as `legacy_rocm_status: detected_unmanaged` — running `rocm install sdk` -creates a separate managed runtime alongside it. Running the command when a -managed runtime is already the active default asks first, because the new -install takes over as the active default; see +creates a separate managed runtime alongside it. + +By default, `rocm install sdk` installs ROCm 7.14. For ROCm 10.0 or newer, add +`--version` and `--family` with the exact GPU arch from `rocm examine`, for +example `--version 10.1.0 --family gfx1200`. + +Running the command when a managed runtime is already the active default asks +first, because the new install takes over as the active default; see [ROCm installation](https://github.com/ROCm/rocm-cli/blob/main/README.md#rocm-installation) -for that gate and the flags that approve it without a prompt. +for that gate, the flags that approve it without a prompt, and the ROCm 10 and +newer requirements. Then serve a model: @@ -348,7 +354,8 @@ rocm update [--apply] [--runtime KEY] [--activate] [--dry-run] ``` `install sdk` downloads TheRock ROCm wheels into a Python environment managed -by rocm-cli. +by rocm-cli. It can install ROCm 7.14, 10.0, or 10.1. To install 10.0 or newer, +follow the steps under ROCm 10 and newer, below. #### Compiler toolchain (--devel) @@ -403,24 +410,6 @@ already there no longer runs its own Python, it is removed outright and rebuilt. The approval prompt doesn't cover this, because it asks only about changing the active default runtime, not about what a named prefix loses. -#### Driver installation - -`install driver` installs the AMD kernel driver on Linux, using DKMS or a native -package. - -#### Updates - -`update` checks for a newer ROCm package. - -| Flag | Description | -| --- | --- | -| `--apply` | Installs the update. Never prompts and needs no approval flag, because selecting a runtime to update is the approval. Leaves the active default alone unless you add `--activate`. | -| `--dry-run` | Previews what `--apply` would do without changing anything. Doesn't require `--apply`. | -| `--runtime`, `--activate` | Require `--apply` or `--dry-run`. | -| `--json` | Prints the check result as a single line of JSON instead of text. Conflicts with `--apply` and `--dry-run`. | -| `--timeout-secs` | Bounds the network calls of the check. Requires `--json`. Conflicts with `--apply`. | -| `--yes` | Accepted for consistency with other mutating commands, but grants nothing on `update`. The approval line the update path prints never credits it. | - #### ROCm 10 and newer ROCm 10 and newer ship from a different source layout. You opt in by passing two @@ -428,7 +417,7 @@ things together: pin the version with `--version`, and name the exact GPU arch, using the raw `gfx` code rather than a family label: ``` -rocm install sdk --version 10.0.0 --family gfx1200 --dry-run +rocm install sdk --version 10.1.0 --family gfx1200 --dry-run ``` A family label such as `--family gfx120X-all` is rejected for those versions @@ -446,6 +435,24 @@ Nothing about this happens on its own. Without a `--version` of 10 or newer, quietly retry against the ROCm 10 sources when a lookup finds nothing; it tells you what it couldn't find instead. +#### Driver installation + +`install driver` installs the AMD kernel driver on Linux, using DKMS or a native +package. + +#### Updates + +`update` checks for a newer ROCm package. + +| Flag | Description | +| --- | --- | +| `--apply` | Installs the update. Never prompts and needs no approval flag, because selecting a runtime to update is the approval. Leaves the active default alone unless you add `--activate`. | +| `--dry-run` | Previews what `--apply` would do without changing anything. Doesn't require `--apply`. | +| `--runtime`, `--activate` | Require `--apply` or `--dry-run`. | +| `--json` | Prints the check result as a single line of JSON instead of text. Conflicts with `--apply` and `--dry-run`. | +| `--timeout-secs` | Bounds the network calls of the check. Requires `--json`. Conflicts with `--apply`. | +| `--yes` | Accepted for consistency with other mutating commands, but grants nothing on `update`. The approval line the update path prints never credits it. | + ### Runtime management Manage multiple side-by-side ROCm runtimes: diff --git a/docs/rocm-docs/getting-started.md b/docs/rocm-docs/getting-started.md index 6a25b6efd..86bfab3a9 100644 --- a/docs/rocm-docs/getting-started.md +++ b/docs/rocm-docs/getting-started.md @@ -19,16 +19,16 @@ SPDX-License-Identifier: MIT ``` + cross-reference retargeted to this site. The end of the sentence is then + reused as the `:start-after:` anchor for the next include, which also + matches the original sentence in README.md; edit both together. --> Running the command when a managed runtime is already the active default asks first, because the new install takes over as the active default; see -[ROCm installation](commands.md#rocm-installation) for that gate and the flags -that approve it without a prompt. +[ROCm installation](commands.md#rocm-installation) for that gate, the flags +that approve it without a prompt, and the ROCm 10 and newer requirements. ```{include} ../../README.md -:start-after: "for that gate and the flags that approve it without a prompt." +:start-after: "newer requirements." :end-before: "You can also serve any compatible Hugging Face model directly" ``` From b29cf4e10cde743e0e66447f0b55e489e96078f2 Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 8 Oct 2026 22:38:44 -0400 Subject: [PATCH 02/18] docs: copyedit README, CONTRIBUTING, and vLLM guide Editorial pass for style and consistency: remove em dashes and spaced double hyphens, use "instead of" and "through", replace and/or and word slashes, spell out TUI on first use, use American spellings, split long paragraphs, and use absolute GitHub URLs for doc links in CONTRIBUTING.md and README.md so they resolve on the docs site. No changes to technical content or to the anchor strings the docs-site include directives depend on. Signed-off-by: pmoutsias-amd --- CONTRIBUTING.md | 33 +++--- README.md | 294 +++++++++++++++++++++++++++--------------------- docs/vllm.md | 164 ++++++++++++++------------- 3 files changed, 271 insertions(+), 220 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e0e1cdc19..d955f96f5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -6,7 +6,7 @@ SPDX-License-Identifier: MIT # Contributing to ROCm CLI -Thank you for your interest in contributing. This document explains how to get started, how work is tracked, and what to expect from the review process. +Thank you for your interest in contributing. This page covers how to get started, how work is tracked, and what to expect from the review process. ## Code of Conduct @@ -36,25 +36,27 @@ fix/vram-probe-fallback ## Development setup -Prerequisites: Rust (see `rust-toolchain.toml` for the pinned version) and [uv](https://github.com/astral-sh/uv) (for prek and scripts). +Prerequisites: Rust (see `rust-toolchain.toml` for the pinned version) and [uv](https://github.com/astral-sh/uv) (for `prek` and the scripts). ```bash git clone https://github.com/ROCm/rocm-cli cd rocm-cli -uv tool install prek # or: cargo install --locked prek +uv tool install prek # alternative: cargo install --locked prek prek install # fast checks on every commit prek install -t pre-push # heavier checks on push (clippy + tests) ``` `prek` runs the same checks locally that CI enforces: `cargo fmt`, `clippy`, `cargo test`, `ruff` (Python), `shellcheck` (shell), PowerShell syntax, and the generated manifests (`MANIFEST.md`, `THIRD_PARTY_NOTICES.txt`). -The manifest hooks only run when you change the dependency graph, and they *rewrite* the generated file rather than just reporting it stale — when that happens the commit stops so you can re-stage the refreshed file. `THIRD_PARTY_NOTICES.txt` additionally needs the pinned generator; without it that hook skips and CI remains the gate: +The manifest hooks only run when you change the dependency graph. They *rewrite* the generated file instead of only reporting that it is stale. When that happens, the commit stops so you can re-stage the refreshed file. + +The `THIRD_PARTY_NOTICES.txt` hook also needs the pinned generator. Without it, the hook skips and CI remains the gate: ```bash cargo install cargo-about@0.9.1 --locked --features cli # optional, for THIRD_PARTY_NOTICES.txt ``` -The license-headers hook (`hawkeye`) runs on both commit and push for matching files. prek runs it like the others, but doesn't provision the `hawkeye` binary — and, unlike the manifest hooks above, it fails hard rather than skipping when the binary is missing: +The license-headers hook (`hawkeye`) runs on both commit and push for matching files. `prek` runs it like the other hooks but doesn't install the `hawkeye` binary. Unlike the manifest hooks, this hook fails instead of skipping when the binary is missing. Install it with: ```bash cargo install hawkeye@7.0.0 --locked # pinned to match the CI license-headers job @@ -67,31 +69,33 @@ cargo install hawkeye@7.0.0 --locked # pinned to match the CI license-headers | `apps/rocm` | Main CLI binary | | `apps/rocmd` | Background daemon | | `crates/rocm-core` | Core library | -| `crates/rocm-dash-*` | Dashboard TUI libraries | +| `crates/rocm-dash-*` | Dashboard terminal user interface (TUI) libraries | | `crates/rocm-engine-protocol` | Engine IPC protocol | | `engines/` | Inference engine adapters (lemonade, vllm) | ### Module organization -New subcommands and subsystems default to their own file from day one — don't let them grow inside `main.rs`/`lib.rs` waiting for a future extraction pass. See `docs/architecture.md` for the two extraction patterns in use, the current module map, and the module-organization convention in full — its path citations are relative markdown links, checked for resolution (not prose accuracy) by the `docs-links` job below. +Put new subcommands and subsystems in their own file from the start. Don't let them grow inside `main.rs` or `lib.rs` while waiting for a later extraction pass. + +See [docs/architecture.md](https://github.com/ROCm/rocm-cli/blob/main/docs/architecture.md) for the two extraction patterns in use, the current module map, and the full module-organization convention. Its path citations are relative Markdown links. The `docs-links` job described below checks that they resolve, but not that the prose is accurate. -Crate-layering invariants (e.g. `rocmd` must never depend on `rocm`) are enforced by `cargo xtask check-crate-edges` (`xtask/src/crate_edges.rs`). +Crate-layering invariants, such as `rocmd` never depending on `rocm`, are enforced by `cargo xtask check-crate-edges` (`xtask/src/crate_edges.rs`). -Every local (relative-path) markdown link and `#anchor` fragment in the repo — outside `docs/rocm-docs/`, which `docs-build` covers instead — is checked by the `docs-links` CI job (`lychee.toml`); it runs offline only, so it doesn't catch broken external `https://` links (see `lychee.toml` for the current exclusions). +The `docs-links` CI job checks every local (relative-path) Markdown link and `#anchor` fragment in the repository. It skips `docs/rocm-docs/`, which the `docs-build` job covers instead. The job runs offline, so it doesn't catch broken external `https://` links. See `lychee.toml` for the current exclusions. ### Test commands | Component | Command | | --- | --- | | Rust (all crates) | `cargo test` | -| Lint + format check | `prek run --all-files` | +| Lint and format check | `prek run --all-files` | -See `docs/testing.md` for the full test guide and `docs/manual-testing.md` for manual QA steps. +See the [test guide](https://github.com/ROCm/rocm-cli/blob/main/docs/testing.md) for the full set of tests and the [manual testing guide](https://github.com/ROCm/rocm-cli/blob/main/docs/manual-testing.md) for manual QA steps. ## Making changes 1. **Fork** the repository and create a branch from `main`. -2. **Make your changes.** Keep commits focused — one logical change per commit. +2. **Make your changes.** Keep commits focused, with one logical change per commit. 3. **Add or update tests** for any new behavior. 4. **Run the relevant test suite** before opening a PR. 5. **Open a pull request** against `main` with a clear title and description following the Conventional Commits format. @@ -108,13 +112,14 @@ git config --global user.signingkey ~/.ssh/id_ed25519.pub git config --global commit.gpgsign true ``` -See `docs/commit-signatures.md` for GPG signing, GitHub "Verified" status, and troubleshooting. +See [Commit signatures](https://github.com/ROCm/rocm-cli/blob/main/docs/commit-signatures.md) for GPG signing, GitHub "Verified" status, and troubleshooting. ### What reviewers look for - Tests cover the new behavior - No secrets, credentials, or internal hostnames in committed files -- Third-party dependencies declared in `Cargo.lock`; license headers present on new source files (see `licenserc.toml`) +- Third-party dependencies recorded in `Cargo.lock` +- License headers on new source files (see `licenserc.toml`) ## Reporting security issues diff --git a/README.md b/README.md index 7063b11a6..32443e492 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ SPDX-License-Identifier: MIT ``` ROCm CLI is a command-line tool for setting up and running local AI on AMD GPUs, with a -full-screen TUI dashboard for GPU telemetry, model serving, and chat. +full-screen terminal user interface (TUI) dashboard for GPU telemetry, model serving, and chat. It ships as a single prebuilt binary for Linux and Windows (x86_64), needs no Python, Rust, or existing ROCm install, and includes inference engine adapters @@ -48,7 +48,7 @@ back to. Ubuntu 22.04 ships glibc 2.35 and cannot run it; Ubuntu 24.04 provides glibc 2.39 and `GLIBCXX_3.4.33`. > [!IMPORTANT] -> **Tech Preview** -- This software is provided as-is, without warranty or +> **Tech Preview:** This software is provided as-is, without warranty or > guarantee of stability. APIs, commands, and behavior might change without > notice. Intended for experimentation and early feedback only. @@ -80,7 +80,7 @@ The installer downloads a prebuilt bundle, verifies its SHA-256 checksum, installs the `rocm` and `rocmd` binaries into `~/.local/bin`, and adds that directory to your shell `PATH`. Rerun it any time to upgrade. -### Linux and WSL (x86_64) +### Linux and WSL2 (x86_64) ```bash curl -fsSL https://raw.githubusercontent.com/ROCm/rocm-cli/main/install.sh | sh @@ -110,7 +110,7 @@ irm https://raw.githubusercontent.com/ROCm/rocm-cli/main/install.ps1 | iex ## Build from source Building requires [Rust](https://rustup.rs/); the pinned toolchain in -`rust-toolchain.toml` (currently 1.96.0) installs automatically via `rustup`. +`rust-toolchain.toml` (currently 1.96.0) installs automatically through `rustup`. ```bash git clone https://github.com/ROCm/rocm-cli @@ -120,8 +120,8 @@ cargo build --release This produces the two binaries under `target/release/`: -- `rocm` — the CLI and interactive interfaces -- `rocmd` — the background telemetry daemon used by the dashboard +- `rocm`: the CLI and interactive interfaces +- `rocmd`: the background telemetry daemon used by the dashboard Run without installing: @@ -146,22 +146,25 @@ Launch ROCm CLI with no arguments: rocm ``` -With no arguments on an interactive terminal, `rocm` opens the **launcher** — a +With no arguments on an interactive terminal, `rocm` opens the **launcher**, a small front-door menu that gets you to the common tasks: -- **Set up this system** — install or update ROCm -- **Serve a model** — run a model on your GPU -- **Diagnose & fix** — check GPU, driver, and ROCm -- **Chat** — talk to a local or API-backed model -- **Open full dashboard →** — escalate into the live dashboard (`rocm dash`) +- **Set up this system**: install or update ROCm +- **Serve a model**: run a model on your GPU +- **Diagnose & fix**: check GPU, driver, and ROCm +- **Chat**: talk to a local or API-backed model +- **Open full dashboard →**: escalate into the live dashboard (`rocm dash`) + +Pick a row with the arrow keys and `Enter`. Press `q` or `Ctrl-C` to quit. +`Ctrl-C` quits from the launcher and the dashboard alike and restores your +terminal. -Pick a row with the arrow keys and `Enter`; press `q` — or `Ctrl-C`, which quits -from the launcher and the dashboard alike and restores your terminal — to quit. The one exception is the dashboard's console for a **running** job, where -`Ctrl-C` keeps its existing meaning of "cancel this job" and does not quit; once -that job finishes, `Ctrl-C` quits there too. On a non-interactive terminal (or -piped output), `rocm` prints a one-shot status summary instead of opening the -launcher. +`Ctrl-C` keeps its existing meaning of "cancel this job" and does not quit. Once +that job finishes, `Ctrl-C` quits there too. + +On a non-interactive terminal (or piped output), `rocm` prints a one-shot status +summary instead of opening the launcher. ## Interactive interfaces @@ -176,7 +179,7 @@ the legacy full-screen setup assistant has been retired. ### The dashboard (`rocm dash`) -The full-screen telemetry dashboard — every instrument and action on one screen. +The full-screen telemetry dashboard shows every instrument and action on one screen. It auto-starts an embedded `rocmd` daemon when none is running, then presents five tabs (switch with `Tab`/`Shift+Tab` or number keys `1`–`5`): @@ -189,7 +192,7 @@ five tabs (switch with `Tab`/`Shift+Tab` or number keys `1`–`5`): | **Chat** | Assistant chat backed by a local server or configured provider | Live mode reads telemetry over a Unix domain socket, so it requires Linux or -WSL. Use `rocm dash --demo` for a synthetic session that runs anywhere without a +WSL2. Use `rocm dash --demo` for a synthetic session that runs anywhere without a GPU or daemon. ## Configure ROCm and serve a model @@ -202,7 +205,7 @@ rocm install sdk This downloads TheRock ROCm wheels and a matching PyTorch stack into a managed environment. On machines with an existing ROCm install, `rocm examine` will -show it as `legacy_rocm_status: detected_unmanaged` — running `rocm install sdk` +show it as `legacy_rocm_status: detected_unmanaged`. Running `rocm install sdk` creates a separate managed runtime alongside it. By default, `rocm install sdk` installs ROCm 7.14. For ROCm 10.0 or newer, add @@ -223,7 +226,7 @@ rocm serve qwen `qwen` is a built-in alias for a small assistant model that serves out of the box. You can also serve any compatible Hugging Face model directly. See -[Model serving](#model-serving) for the GGUF-vs-safetensors rule, since which +[Model serving](#model-serving) for the GGUF versus safetensors rule, because which form works depends on the engine your GPU selects. ## Quick reference @@ -232,7 +235,7 @@ form works depends on the engine your GPU selects. |---|---| | `rocm` | Open the launcher menu (setup, serve, diagnose, chat, dashboard) | | `rocm examine` | Check GPU, ROCm install, engines, and managed folders | -| `rocm diagnose` | Match this machine against known ROCm/PyTorch/llama.cpp failure modes | +| `rocm diagnose` | Match this machine against known ROCm, PyTorch, and llama.cpp failure modes | | `rocm diagnose --model ` | Say whether a model will run here, before downloading it | | `rocm diagnose --report` | Show what this machine would contribute to a problem report, and send nothing | | `rocm fix []` | Apply a fix reported by `rocm diagnose` | @@ -243,7 +246,7 @@ form works depends on the engine your GPU selects. | `rocm dash` | Open the full-screen telemetry dashboard | | `rocm bench load --endpoint ` | Load-test a local OpenAI-compatible endpoint | | `rocm setup status` | Show first-time setup state | -| `rocm version` | Print the rocm-cli version, release tag or branch, and commit hash, plus the ROCm SDK and GPU driver in use | +| `rocm version` | Print the ROCm CLI version, release tag or branch, and commit hash, plus the ROCm SDK and GPU driver in use | | `rocm completions ` | Print a shell completion script (bash, zsh, fish, elvish, powershell) | ## Commands @@ -255,15 +258,18 @@ rocm examine [--json] [--framework auto|pytorch|llama-cpp|skip] ``` Checks this computer's GPU, ROCm install, engines, and managed setup -folders — the command to run first to see whether a system is ready, and -what `rocm install sdk` and `rocm serve` will see. `--json` emits a +folders. Run this command first to see whether a system is ready and what +`rocm install sdk` and `rocm serve` will see. `--json` emits a machine-readable report for diagnosis tooling instead of the human-readable summary. `--framework` controls which ML framework the `--json` report probes -for its ROCm build and compiled GPU architectures: `auto` (the default) tries -PyTorch, then falls back to llama.cpp; `pytorch` or `llama-cpp` probe only -that framework; `skip` runs no framework probe at all, which is fastest and -still enough to answer GPU and driver questions. `--framework` only affects -the JSON report, not the human-readable one. +for its ROCm build and compiled GPU architectures: + +- `auto` (the default) tries PyTorch, then falls back to llama.cpp. +- `pytorch` or `llama-cpp` probe only that framework. +- `skip` runs no framework probe at all, which is fastest and still enough to + answer GPU and driver questions. + +`--framework` only affects the JSON report, not the human-readable one. ### Diagnose and fix @@ -274,47 +280,56 @@ rocm fix [] [--yes] [--dry-run] [--device-index N] ``` `diagnose` matches this machine against a fixed catalog of known -ROCm/PyTorch/llama.cpp misconfigurations and ranks what it finds. It can only -recognise failure modes that are in the catalog: no match means "not -recognised", not "nothing is wrong" — in that case it points you at where to -report the symptom. Each result prints an `id:` and an `apply with:` command; -the leading `#1`, `#2` are ranking positions for reading order only — `rocm -fix` takes the id, not the position. +ROCm, PyTorch, and llama.cpp misconfigurations and ranks what it finds. It can +only recognize failure modes that are in the catalog: no match means "not +recognized", not "nothing is wrong". In that case it points you at where to +report the symptom. + +Each result prints an `id:` and an `apply with:` command. The leading `#1`, `#2` +are ranking positions for reading order only. `rocm fix` takes the id, not the +position. - `--symptom` takes raw error text to sharpen keyword scoring. - `--top` caps how many matches are shown in the human-readable output - (default 5) — `--json` always emits the full, untruncated report. + (default 5). `--json` always emits the full, untruncated report. - `--distro` diagnoses a WSL distribution from the Windows host instead of - this machine (nothing needs to be installed inside the distribution — name + this machine (nothing needs to be installed inside the distribution; name it only when more than one is installed). Inspecting remotely this way skips checks that need to read the distribution's own environment - (`HSA_OVERRIDE_GFX_VERSION`, `PATH`, the framework/ROCm pairing) — run + (`HSA_OVERRIDE_GFX_VERSION`, `PATH`, the framework and ROCm pairing). Run `rocm diagnose` inside the distribution for those. - `--report` shows exactly what this machine would contribute to a problem - report, and sends nothing — there is no transport yet, and there will be no - automatic one: a report leaves a machine only by its owner's own action. The - content is deliberately narrow (a schema version, the matched entry, whether + report, and sends nothing. There is no transport yet, and there will be no + automatic one: a report leaves a machine only by its owner's own action. + + The content is deliberately narrow (a schema version, the matched entry, whether a fix was offered for it, the GPU architecture and which compatibility matrix snapshot it was checked against, the OS family, distribution and major version, the ROCm release, the inference engine and its release, the CLI version), and it carries no host name, user name, file path, or error - text. The ROCm release and the inference engine's release are each cut + text. + + The ROCm release and the inference engine's release are each cut back to a release, so a build number that would narrow toward one machine never appears there; the CLI's own version is the exception, since it names - the tool that wrote the report rather than something read off the machine. - The distribution is checked against a list of known names rather than - repeated from the machine. Hardware that is not on AMD's published + the tool that wrote the report instead of something read off the machine. + The distribution is checked against a list of known names instead of + being repeated from the machine. + + Hardware that is not on AMD's published compatibility matrix produces no report at all, and the CLI says why. So does a WSL machine, for a different reason: this CLI does not inspect the GPU on WSL yet, so it cannot confirm the hardware is on the compatibility - matrix and says that rather than claiming the architecture could not be + matrix and says that instead of claiming the architecture could not be read. - `--send`, which requires `--report`, additionally offers a prefilled mail carrying that report. It still sends nothing: the mail opens already filled in with the content `--report` just printed, addressed to `ROCmCLI@amd.com`, and it leaves the machine only when you send it yourself. Requiring `--report` is what guarantees the content is shown before the mail is - offered. A mail client opens only when you asked and the machine looks like + offered. + + A mail client opens only when you asked and the machine looks like a desktop you are at; over SSH, with no display, or with `ROCM_NO_BROWSER` set, the address and the link are printed instead, which is also what happens on a machine with no mail client. It is not combinable with @@ -322,15 +337,17 @@ fix` takes the id, not the position. read a mail before sending it. Note that a mail carries your address, which the report itself does not. -`fix` applies a known fix by the `id:` that `diagnose` reported — not the +`fix` applies a known fix by the `id:` that `diagnose` reported, not the ranking position noted above, which isn't a stable name. Run it with no id -to list the whole catalog. Each fix carries a marker saying what happens on +to list the whole catalog. + +Each fix carries a marker saying what happens on this machine: AUTO (this command carries out the change), NEEDS-ARG (it will, once given the argument it names), PRINT-ONLY (it prints the steps for you to -run yourself — usually because the right command depends on a choice only you +run yourself, usually because the right command depends on a choice only you can make, sometimes because it also needs sudo or a reboot), or DIAGNOSE-ONLY -(no reliable fix exists, so nothing will be changed -- no catalog entry -carries this marker today; it is reserved for a future detect-but-cannot-repair +(no reliable fix exists, so nothing will be changed; no catalog entry +carries this marker today, and it is reserved for a future detect-but-cannot-repair failure). - `--dry-run` shows any fix's plan without changing anything. @@ -354,7 +371,7 @@ rocm update [--apply] [--runtime KEY] [--activate] [--dry-run] ``` `install sdk` downloads TheRock ROCm wheels into a Python environment managed -by rocm-cli. It can install ROCm 7.14, 10.0, or 10.1. To install 10.0 or newer, +by ROCm CLI. It can install ROCm 7.14, 10.0, or 10.1. To install 10.0 or newer, follow the steps under ROCm 10 and newer, below. #### Compiler toolchain (--devel) @@ -385,7 +402,7 @@ To reclaim the space, uninstall the one you don't want with If no managed runtime is the active default, `install sdk` doesn't prompt. Otherwise it asks first, because the new install becomes the active default. The prompt applies to any install, including a `--family` or `--channel` you -haven't installed before, just as it does for a same-family upgrade. +haven't installed before, as it does for a same-family upgrade. To approve without a prompt, for example in scripts or CI, where the prompt would otherwise refuse: @@ -414,14 +431,14 @@ active default runtime, not about what a named prefix loses. ROCm 10 and newer ship from a different source layout. You opt in by passing two things together: pin the version with `--version`, and name the exact GPU arch, -using the raw `gfx` code rather than a family label: +using the raw `gfx` code instead of a family label: ``` rocm install sdk --version 10.1.0 --family gfx1200 --dry-run ``` A family label such as `--family gfx120X-all` is rejected for those versions -rather than resolved to a guess, because the ROCm 10 packages publish one +instead of being resolved to a guess, because the ROCm 10 packages publish one payload per exact arch and there is no bucket payload to fall back to. Run `rocm examine` to see the arch this machine reports. @@ -496,25 +513,29 @@ rocm storage remove-downloads [--dry-run] [--yes] `remove-old-installs` keeps the two most recent installs for each channel, format, GPU family, and toolchain choice, and never touches the install in use, -the rollback target, or a folder rocm-cli did not create. "Most recent" means -most recently installed rather than highest version, so after a deliberate -downgrade the older version counts as the newer install. Because the count -applies per channel, format, GPU family, and toolchain choice, a machine that -has tried several channels keeps `--keep` installs for each of them — and a -runtime-only install never evicts a `--devel` one, since the two are separate -runtimes serving different purposes rather than newer and older versions of the -same thing. Anything it declines to remove is -listed with the reason, and `--dry-run` shows the whole plan without changing -anything. `remove-downloads` clears cached archives that rocm-cli can download -again; a cache folder that is a link to somewhere else is left alone rather -than followed. The two archive rows the report prints are tagged +the rollback target, or a folder ROCm CLI did not create. "Most recent" means +most recently installed instead of highest version, so after a deliberate +downgrade the older version counts as the newer install. + +Because the count applies per channel, format, GPU family, and toolchain +choice, a machine that has tried several channels keeps `--keep` installs for +each of them. A runtime-only install never evicts a `--devel` one, because the +two are separate runtimes serving different purposes instead of newer and +older versions of the same thing. Anything it declines to remove is listed with +the reason, and `--dry-run` shows the whole plan without changing anything. + +`remove-downloads` clears cached archives that ROCm CLI can download again; a +cache folder that is a link to somewhere else is left alone instead of +being followed. The two archive rows the report prints are tagged `note: can be downloaded again; safe to remove`, so it is visible which rows -`remove-downloads` acts on. The report also lists `local server records` — one -JSON record plus the engine's log for each `rocm serve --managed` launch, kept -after the server stops; `rocm services list --all` lists them, and no `rocm -storage` command removes them — and the `uv` package cache, the Hugging Face -model cache, and downloaded models; those last three are shared with other -tools and are never removed by rocm-cli. +`remove-downloads` acts on. + +The report also lists `local server records`: one JSON record plus the engine's +log for each `rocm serve --managed` launch, kept after the server stops. +`rocm services list --all` lists them, and no `rocm storage` command removes +them. It also lists the `uv` package cache, the Hugging Face model cache, and +downloaded models. Those last three are shared with other tools and are never +removed by ROCm CLI. ### Inference engines @@ -534,7 +555,7 @@ Ask before downloading anything: rocm diagnose --model [--json] ``` -Answers in seconds, from the curated recipe and this machine's GPU — it +Answers in seconds, from the curated recipe and this machine's GPU. It fetches no weights and makes no network call. The verdict is `ready`, `degraded`, `blocked`, or `undetermined`. A `ready` answer also names the engine `rocm serve` would use; a `blocked` one names curated models that would @@ -544,8 +565,8 @@ run here instead. recipe catalog could not be read, when this machine's GPU memory could not be measured, or when the model is not one of the curated recipes (`rocm model` lists those). None of those say anything about whether the model fits, so none -of them are reported as though they did — `rocm serve` still accepts a model -outside the catalog, this just cannot tell you in advance how it will go. +of them are reported as though they did. `rocm serve` still accepts a model +outside the catalog; it cannot tell you in advance how it will go. ### Model serving @@ -569,7 +590,7 @@ server-wide sampling defaults for the launched engine. They apply only to into a single `--override-generation-config` JSON object (`--max-tokens` maps to vLLM's `max_new_tokens`); for Lemonade they pass straight through as llama.cpp's `--temperature`, `--top-p`, and `--n-predict` flags. Each control -is optional and independent — omit any of them to keep the engine's own +is optional and independent, so omit any of them to keep the engine's own default. `rocm serve` only reuses an already-running service for the same engine and @@ -577,46 +598,53 @@ model if its sampling controls (and other recipe settings) match the ones requested this time; otherwise it errors out instead of silently serving with different settings. If you previously started a service with `--temperature` (or another sampling flag) and now run `rocm serve` for the same model without -flags — or with different ones — stop the existing service first (`rocm +flags, or with different ones, stop the existing service first (`rocm services stop`) or match the original flags. -By default the server runs in the background under rocm-cli's supervision and -prints a deployment summary — a progress indicator while it starts, then a table +By default the server runs in the background under ROCm CLI's supervision and +prints a deployment summary: a progress indicator while it starts, then a table with the status, the full inference endpoint, the API-qualified model name, and a -quick smoke test (time to first token and approximate tokens/sec). Control +quick smoke test (time to first token and approximate tokens per second). Control returns to your shell with the server still running; manage it later with `rocm services` (below). `--verbose` (or `--foreground`) instead attaches to the server in the current -terminal and streams every engine log line — use it to debug a startup problem. -The server still runs as a managed background process, so while streaming you can -press **Ctrl-D to detach** — the log stream stops, your shell comes back, and the -server keeps running (manage it afterward with `rocm services`). Press **Ctrl-C** -to stop the server instead. `--managed` is the explicit form of the default -background behavior. `--no-smoke-test` skips the post-startup inference probe. +terminal and streams every engine log line. Use it to debug a startup problem. +The server still runs as a managed background process, so while streaming you +can: + +- Press **Ctrl-D** to detach. The log stream stops, your shell comes back, and + the server keeps running. Manage it afterward with `rocm services`. +- Press **Ctrl-C** to stop the server instead. + +`--managed` is the explicit form of the default background behavior. +`--no-smoke-test` skips the post-startup inference probe. Which model form to pass depends on the engine your GPU selects. The Lemonade -engine (Ryzen AI or Radeon) serves llama.cpp **GGUF** models — pass a GGUF repo +engine (Ryzen AI or Radeon) serves llama.cpp **GGUF** models. Pass a GGUF repo with an explicit quantization variant, for example, -`rocm serve unsloth/Qwen3-0.6B-GGUF:Q4_0`. The vLLM engine (Instinct) serves -**safetensors** repos, such as `rocm serve Qwen/Qwen2.5-1.5B-Instruct`. A -safetensors-only id has no GGUF build, so serving it through Lemonade fails -rather than silently substituting a different model. +`rocm serve unsloth/Qwen3-0.6B-GGUF:Q4_0`. + +The vLLM engine (Instinct) serves +**safetensors** repos, such as `rocm serve Qwen/Qwen2.5-1.5B-Instruct`. +A safetensors-only id has no GGUF build, so serving it through Lemonade fails +instead of silently substituting a different model. -Some models (such as Llama) are gated and require HuggingFace authentication. +Some models (such as Llama) are gated and require Hugging Face authentication. Log in with `huggingface-cli login` or set `HF_TOKEN` in your environment before serving gated models. `--gpu` selects which AMD GPU the server runs on. `auto` (the default) probes -per-GPU VRAM (via `amd-smi`, or the amdgpu DRM sysfs counters when `amd-smi` is -not installed) and picks the lowest-numbered GPU that is idle and -not already used by another rocm-cli server (managed or foreground), falling -back to the GPU with the most free memory. Pass a single index (`--gpu 1`) to -pin a specific device. The -selected GPU is exposed to the engine via `HIP_VISIBLE_DEVICES`. Serving one -model across multiple GPUs is not supported. Because selection uses the -`amd-smi` ordinal but is applied via `HIP_VISIBLE_DEVICES`, rocm-cli warns when -`ROCR_VISIBLE_DEVICES` is set, since the two orderings can diverge. +per-GPU VRAM (through `amd-smi`, or the amdgpu DRM sysfs counters when `amd-smi` +is not installed) and picks the lowest-numbered GPU that is idle and not already +used by another ROCm CLI server (managed or foreground), falling back to the GPU +with the most free memory. Pass a single index (`--gpu 1`) to pin a specific +device. + +The selected GPU is exposed to the engine through `HIP_VISIBLE_DEVICES`. Serving +one model across multiple GPUs is not supported. Because selection uses the +`amd-smi` ordinal but is applied through `HIP_VISIBLE_DEVICES`, ROCm CLI warns +when `ROCR_VISIBLE_DEVICES` is set, because the two orderings can diverge. Manage background servers started with `--managed`: @@ -633,21 +661,26 @@ rocm services prune [--older-than-hours | --any-age] [--dry-run] [--yes] its engine state file, and its endpoint key file; a running server is refused, so stop it first. `prune` does the same in bulk, always leaves running servers alone, and additionally clears leftover files whose record is already gone. + Removal destroys both the log and the `restart` option for the records it takes, so `prune` only considers records untouched for 24 hours. Age is measured from when the record file was last written, so a stop, a restart, or a status correction all count as touching it. Pass `--older-than-hours ` for a different threshold, or `--any-age` to take every record that is not running -however recent — that is the flag `prune` names in its own summary when it +however recent. That is the flag `prune` names in its own summary when it reports how many records it kept for being too recent. The two cannot be -combined. A file whose record has not been written *yet* belongs to a server +combined. + +A file whose record has not been written *yet* belongs to a server that is still starting, not to something left behind, so `prune` waits for any managed launch already under way to finish publishing its record before it looks at the directory. That wait lasts as long as the launch does and has no timeout, so it is usually imperceptible but is not bounded: on an interactive terminal `prune` prints `Waiting for a launch already under way…` while it waits, including under `--dry-run`. That notice goes to stderr and is suppressed when -stderr is not a terminal, so a piped or scripted prune waits silently. The same +stderr is not a terminal, so a piped or scripted prune waits silently. + +The same lock runs in the other direction, so a `rocm serve` started while a `prune` is scanning waits for the prune. @@ -660,7 +693,7 @@ Run a model on a different GPU machine and reach it from your own. Both machines join a [Tailscale](https://tailscale.com) network; the GPU machine serves the model on its own loopback address and publishes that port onto the network, so the endpoint keeps working after the command exits and answers from any of your -machines rather than only the one that started it. +machines instead of only the one that started it. ```console rocm remote targets [--tag ] @@ -672,20 +705,20 @@ rocm remote attach rocm remote stop [--force] ``` -`targets` lists machines on your network — it does not check whether they can +`targets` lists machines on your network. It does not check whether they can actually serve, which is what `doctor` is for. `serve` prepares the machine (installing the CLI if it is missing), starts the model, publishes the endpoint and prints the address together with an API key. **The endpoint is reachable by every machine on your network that your network's access rules allow**, not just yours. That API key is what stops anyone else -using it, so `rocm remote` always sets one — unlike local serving, which is +from using it, so `rocm remote` always sets one. This differs from local serving, which is credential-free because only your own machine can reach it. `status` reports the model and the endpoint separately, because either can fail alone: a healthy model with no endpoint needs `attach`, not a restart. `stop` withdraws the endpoint and stops the model, and keeps the session listed if it -cannot confirm both — use `--force` to forget one whose machine is gone. +cannot confirm both. Use `--force` to forget one whose machine is gone. Communication with the machine uses your existing `ssh` setup. Set `ROCM_REMOTE_SSH_CONFIG` to point at a configuration file other than the @@ -697,15 +730,16 @@ default. rocm dash [--demo] [--replay ] ``` -Full-screen TUI with Home, ROCm, Serving, Observe, and Chat tabs — GPU +Full-screen TUI with Home, ROCm, Serving, Observe, +and Chat tabs. It shows GPU utilization graphs, active serving instances, benchmark results, guided actions, and a chat tab backed by any configured provider. See [Interactive interfaces](#interactive-interfaces) for the tab breakdown. -- `--demo` runs a deterministic synthetic session with no GPU or daemon needed, - works on all platforms. +- `--demo` runs a deterministic synthetic session with no GPU or daemon needed. + It works on all platforms. - `--replay ` replays a recorded NDJSON session. -- Live mode requires Unix domain sockets (Linux and WSL only). +- Live mode requires Unix domain sockets (Linux and WSL2 only). ### Bench @@ -715,21 +749,21 @@ rocm bench load --endpoint URL [--model NAME] [--concurrency N,N,...] ``` Saturates a local OpenAI-compatible endpoint and reports rough client-side -throughput — a local smoke test, **not** an official ROCm/AMD benchmark. +throughput. It is a local smoke test, **not** an official ROCm or AMD benchmark. `load` measures raw serving throughput with synthetic single-shot requests (the vLLM `benchmark_serving` shape); it does not reproduce agent-shaped, multi-turn, long-context tool traffic and isn't comparable to `*-agent-bench` quality harnesses. - `--endpoint` is the OpenAI-compatible URL shown by `rocm services list` (a - plain host address without `/v1` also works); only `http://` is accepted — - `https://` endpoints are rejected outright, since the load generator has no + plain host address without `/v1` also works). Only `http://` is accepted; + `https://` endpoints are rejected outright, because the load generator has no TLS backend compiled in. - `--concurrency` sweeps a comma-separated list of levels (default `1,8,32,64`, each 1-128); `--auto-ramp` ignores `--concurrency` and instead ramps `1,2,4,8,16,32,64,128` automatically, stopping early once generation throughput plateaus or the request queue backs up. -- `--isl`/`--osl` (input/output sequence length, default 1024 each) accept +- `--isl` and `--osl` (input and output sequence length, default 1024 each) accept 1-32768, and `--requests` (default 128) accepts 1-10000. - Results are written to `--out` (default `/bench/results.csv`, where `` is `~/.rocm` unless overridden), intended to match the @@ -780,7 +814,7 @@ Optional background checks that can propose or apply changes automatically. ### Configuration -Show or change rocm-cli's saved settings — the default engine and runtime, +Show or change ROCm CLI's saved settings: the default engine and runtime, which runtime each engine prefers, local GPU telemetry opt-in, and the provider used for chat, automations, and ambiguous natural-language plans (including enabling providers and storing their API keys). @@ -809,10 +843,10 @@ rocm setup reset ``` Manage first-time setup state. `status` shows whether first-time setup has -completed; `reset` clears the recorded completed/dismissed state (nothing -auto-triggers onboarding from this alone — open it manually from the -dashboard, `rocm dash`: switch to the **Observe** tab, then press `n`). ROCm -installs, API keys, and provider settings are left untouched. +completed. `reset` clears the recorded completed or dismissed state. Nothing +auto-triggers onboarding from this alone; to open it manually, run `rocm dash`, +switch to the **Observe** tab, and press `n`. ROCm installs, API keys, and +provider settings are left untouched. ### Logs and cleanup @@ -865,8 +899,8 @@ See [CONTRIBUTING.md](CONTRIBUTING.md). ## More docs -- Architecture and module map: `docs/architecture.md` -- Testing and verification: `docs/testing.md` -- Developer manual QA: `docs/manual-testing.md` -- Engine plugin policy: `docs/engine-plugins.md` -- vLLM adapter: `docs/vllm.md` +- [Architecture and module map](https://github.com/ROCm/rocm-cli/blob/main/docs/architecture.md) +- [Testing and verification](https://github.com/ROCm/rocm-cli/blob/main/docs/testing.md) +- [Developer manual QA](https://github.com/ROCm/rocm-cli/blob/main/docs/manual-testing.md) +- [Engine plugin policy](https://github.com/ROCm/rocm-cli/blob/main/docs/engine-plugins.md) +- [vLLM adapter](https://github.com/ROCm/rocm-cli/blob/main/docs/vllm.md) diff --git a/docs/vllm.md b/docs/vllm.md index c94c40ba7..1d2805c18 100644 --- a/docs/vllm.md +++ b/docs/vllm.md @@ -52,14 +52,14 @@ ROCM_CLI_DISABLE_TORCH_ALIGNMENT=1 rocm engines install vllm --yes Any value works, including an empty one: the variable being set is the signal. The install then reports `torch_alignment: disabled`, naming both the build it would have installed and the one it kept. The device check still runs, so an -opt-out that leaves the runtime unable to serve says so rather than failing later +opt-out that leaves the runtime unable to serve says so instead of failing later during serving. -Use it when you are deliberately running a torch the alignment would replace (a -locally built wheel, a version under test, or a stack pinned for a reproduction). It -is an escape hatch, not a supported configuration: the resulting combination is -not validated against the supported matrix, and a runtime that cannot execute a -kernel will fail at serving time. +Use it when you are deliberately running a torch that the alignment would +replace, such as a locally built wheel, a version under test, or a stack pinned +for a reproduction. It is an escape hatch, not a supported configuration: the +resulting combination is not validated against the supported matrix, and a +runtime that cannot execute a kernel will fail at serving time. ## ROCm 10.x wheel discovery @@ -67,13 +67,14 @@ For most ROCm SDK versions, `rocm engines install vllm` pins a fixed vLLM wheel and index URL. Any ROCm SDK 10.x version is different: AMD publishes vLLM, flash-attn, and amd-aiter there under a rotating dev-tag filename (for example `vllm-0.27.1.dev5+rocm10.0.0.gf46a9dfe2.d20260826-cp314-cp314-linux_x86_64.whl`), -so there is no fixed filename to pin in the adapter. The wheel's own tag shows -the other thing that changes on this route: ROCm 10.x's vLLM, flash-attn, and -amd-aiter are published for **`cp314` only**, unlike every earlier ROCm SDK -version's `cp312` wheels. ROCm CLI provisions a matching interpreter for a -10.x install automatically; a Python already on `PATH` or already installed -as ROCm CLI's managed Python is rejected if it is not `cp314`, with a message -naming the required tag. +so there is no fixed filename to pin in the adapter. + +The wheel's own tag shows the other thing that changes on this route: ROCm +10.x's vLLM, flash-attn, and amd-aiter are published for **`cp314` only**, unlike +every earlier ROCm SDK version's `cp312` wheels. ROCm CLI provisions a matching +interpreter for a 10.x install automatically. A Python already on `PATH`, or +already installed as ROCm CLI's managed Python, is rejected if it is not +`cp314`, with a message naming the required tag. This route is selected by `major.minor`, so `10.0.0` and `10.1.0` discover through *different* rows: they pin different vLLM minors and read vLLM from @@ -81,33 +82,37 @@ different index URLs, because AMD stages ROCm 10.1's vLLM builds on a separate host from 10.0's production index. Both rows take their torch stack from the same `whl-next` index, which is where `rocm install sdk` resolves either SDK's own `+rocmX.Y` torch from, but vLLM's discovery is independent of that: it -resolves its own torch/torchvision/torchaudio pins fresh from the row's own -static version prefixes, rather than reusing whatever the SDK install -resolved, which may be a different torch version than this row pins. Patch -and any dev/pre-release suffix are still ignored within a row, since AMD +resolves its own torch, torchvision, and torchaudio pins fresh from the row's +own static version prefixes, instead of reusing whatever the SDK install +resolved, which might be a different torch version than this row pins. Patch +and any dev or pre-release suffix are still ignored within a row, because AMD rotates those constantly. The install resolves each package's current wheel, including torch, from the row's index with `uv pip install --dry-run --reinstall`, parses the version it reports it would install, then reinstalls pinned to that exact version. -tensorizer is not discovered or pinned this way: it has no ROCm-specific -build, and vllm's own wheel metadata already declares an exact tensorizer -dependency, so it is left to vllm's own dependency resolution rather than -risk a conflicting pin of its own. Every resolved pin that carries a +`tensorizer` is not discovered or pinned this way: it has no ROCm-specific +build, and vLLM's own wheel metadata already declares an exact tensorizer +dependency, so it is left to vLLM's own dependency resolution instead of +risking a conflicting pin of its own. Every resolved pin that carries a `+rocmX.Y` local version is checked against the SDK's own major.minor before installing, so an index that happens to serve more than one ROCm line at once cannot silently install the wrong line's wheel onto this SDK. If AMD's index has no compatible build for a package, the resolver fails and the install -fails rather than falling back to an unpinned or CPU install. The final install -of vllm/flash-attn/amd-aiter also resolves vllm's own plain-PyPI transitive -dependencies (e.g. `lm-format-enforcer`), which AMD's index doesn't host; a -generated `uv.toml` sets `ignore-error-codes = [403]` for that index so `uv` -falls through to PyPI for those instead of treating the index's 403 as fatal. +fails instead of falling back to an unpinned or CPU install. + +The final install of vllm, flash-attn, and amd-aiter also resolves vLLM's own +plain-PyPI transitive dependencies (for example, `lm-format-enforcer`), which +AMD's index doesn't host. A generated `uv.toml` sets `ignore-error-codes = [403]` +for that index so `uv` falls through to PyPI for those instead of treating the +index's 403 as fatal. + That same full-dependency resolve can also pull in an unconstrained `torch` from PyPI, undoing the exact ROCm pin just installed, and transitively an -unconstrained `torchvision`/`torchaudio` with it; the install re-pins all -three back to the exact builds discovered above immediately afterwards. That -same realign also covers AMD's per-GPU-architecture device-kernel plugins +unconstrained `torchvision` and `torchaudio` with it. The install re-pins all +three back to the exact builds discovered above immediately afterwards. + +That same realign also covers AMD's per-GPU-architecture device-kernel plugins (`amd-torch-device-gfx*`, `amd-torchvision-device-gfx*`): each is versioned in lockstep with its own base package but resolved independently of it, so the full-dependency install can leave one ahead of the base package it ships @@ -115,14 +120,14 @@ kernels for. The install reads `uv pip freeze`, and re-pins every such plugin it finds to its own base package's resolved release. The mismatch is invisible at install time and only surfaces at serve time, as a HIP "Cannot find Symbol" crash. -Every other -ROCm SDK version, including 7.2.3, keeps using the static pin table; an SDK -version with no matching row there falls back to the table's default pin, -*unless* its major release matches a discovery-table entry, in which case -guessing the default pin would very likely install an ABI-incompatible build, -so the install fails closed instead with a message naming the detected -version and pointing at `ROCM_CLI_VLLM_ROCM_INDEX_URL` as the way to install -anyway. + +Every other ROCm SDK version, including 7.2.3, keeps using the static pin +table. An SDK version with no matching row there falls back to the table's +default pin, *unless* its major release matches a discovery-table entry. In that +case, guessing the default pin would very likely install an ABI-incompatible +build, so the install fails closed instead. The error message names the +detected version and points at `ROCM_CLI_VLLM_ROCM_INDEX_URL` as the way to +install anyway. ## Discovery paths and checks @@ -153,13 +158,18 @@ python3 scripts/vllm_therock_gpu_test.py \ --model facebook/opt-125m ``` -The acceptance script is Linux or WSL only. It requires vLLM to be discoverable -through a ROCm CLI-managed TheRock runtime manifest, launches with -`gpu_required`, checks `/health` and `/v1/completions`, and verifies loaded -ROCm libraries come from the managed TheRock SDK wheel directories. It rejects -external vLLM command overrides and does not allow CPU fallback. It defaults -to the active exact runtime key; if `--runtime-id` is passed, use an exact -runtime key or an unambiguous runtime id. +The acceptance script is Linux or WSL only. It does the following: + +- Requires vLLM to be discoverable through a ROCm CLI-managed TheRock runtime + manifest. +- Launches with `gpu_required`. +- Checks `/health` and `/v1/completions`. +- Verifies that loaded ROCm libraries come from the managed TheRock SDK wheel + directories. + +It rejects external vLLM command overrides and does not allow CPU fallback. It +defaults to the active exact runtime key. If you pass `--runtime-id`, use an +exact runtime key or an unambiguous runtime id. ### Source build notes @@ -179,7 +189,7 @@ GPTQ compatibility guard in Without that patch, `q_gemm.hip` fails to compile because TheRock 7.13 headers do not expose the `half`/`half2` `atomicAdd` overloads used by vLLM's GPTQ kernel. With the patch, the live acceptance harness passed on -`facebook/opt-125m` and verified HIP/BLAS libraries loaded from the managed +`facebook/opt-125m` and verified HIP and BLAS libraries loaded from the managed TheRock SDK wheel directories. ## Serve a model @@ -202,14 +212,14 @@ rocm serve Qwen/Qwen3.5-4B --engine vllm --managed rocm serve Qwen/Qwen3.5-4B --engine vllm --gpu 1 --managed ``` -ROCm CLI pins the device via `HIP_VISIBLE_DEVICES`. Serving one model across +ROCm CLI pins the device through `HIP_VISIBLE_DEVICES`. Serving one model across multiple GPUs is not supported. ## GPU memory -vLLM claims a fixed fraction of each GPU's **total** VRAM (not of the free -VRAM, and not scaled to the model) for weights plus KV cache. On a large card -a small model therefore still reserves a large slice. +vLLM claims a fixed fraction of each GPU's **total** VRAM for weights plus KV +cache. The fraction is not based on the free VRAM, and it is not scaled to the +model. On a large card, a small model therefore still reserves a large slice. ROCm CLI sets no `--gpu-memory-utilization` of its own, so vLLM's own default applies unless a value comes from somewhere else: either a model's catalog @@ -220,63 +230,65 @@ rocm serve --engine vllm --gpu-memory-utilization 0.3 --managed ``` The value is a fraction in `(0, 1]` of total device VRAM. Lower it to leave room -for a display, another workload, or a second server; raise it to give a large -model more KV cache. Applies to vLLM only; it is ignored, with a note in the +for a display, another workload, or a second server. Raise it to give a large +model more KV cache. It applies to vLLM only and is ignored, with a note in the serve output, for other engines. An out-of-range or unparsable value fails the -command rather than falling back silently. +command instead of falling back silently. -Earlier releases pinned this to `0.80` to leave display/WSL headroom. That pin is -gone, so an unchanged command now reserves vLLM's own (higher) default. Pass -`--gpu-memory-utilization 0.8` to restore the previous reservation. +Earlier releases pinned this to `0.80` to leave display or WSL headroom. That +pin is gone, so an unchanged command now reserves vLLM's own (higher) default. +Pass `--gpu-memory-utilization 0.8` to restore the previous reservation. ### Shared or busy GPUs Because the reservation is a fraction of **total** VRAM, it ignores memory already held by other workloads. On a shared multi-GPU node the default can collide with in-use memory and the engine fails with `HIP out of memory` even for -a tiny model. rocm-cli helps in three ways: +a tiny model. ROCm CLI helps in three ways: - **Auto-selection avoids busy cards.** `--gpu auto` ranks GPUs by free VRAM and - skips heavily-used ones. When `amd-smi` is not installed it falls back to the + skips heavily used ones. When `amd-smi` is not installed, it falls back to the amdgpu DRM sysfs counters (`/sys/class/drm/card*/device/mem_info_vram_{total,used}`), so selection still works on stripped-down container images with a single GPU. That fallback - withholds telemetry on a multi-GPU host, since its `card` numbering is not - guaranteed to match HIP's device ordinal there. + withholds telemetry on a multi-GPU host, because its `card` numbering is + not guaranteed to match HIP's device ordinal there. - **The serve summary warns on low free VRAM.** When the selected GPU is already - heavily used, `rocm serve` prints a note — before launch on the plain path, or - in the post-readiness summary in the default interactive mode — and, for vLLM, - points at `--gpu-memory-utilization` as the fix. + heavily used, `rocm serve` prints a note. The note appears before launch on + the plain path, or in the post-readiness summary in the default interactive + mode. For vLLM, the note also points at `--gpu-memory-utilization` as the fix. - **OOM failures hint the workaround.** When a startup failure log shows an out-of-memory error, the failure message suggests retrying with a smaller - reservation, e.g. `--gpu-memory-utilization 0.5`, or targeting a less-busy GPU - with `--gpu `, and points at `rocm diagnose --symptom ''` - for the full conditional remediation (busy GPU vs. a model that does not fit). - The printed command quotes your actual failing line when it can be rendered as - one intact single-quoted argument; a line carrying an apostrophe or terminal - control bytes falls back to the canonical symptom below rather than handing you - a command whose quoting the log text broke. + reservation, such as `--gpu-memory-utilization 0.5`, or targeting a less busy + GPU with `--gpu `. It also points at + `rocm diagnose --symptom ''` for the full conditional remediation + (busy GPU versus a model that does not fit). The printed command quotes your + actual failing line when it can be rendered as one intact single-quoted + argument. A line carrying an apostrophe or terminal control bytes falls back to + the canonical symptom below instead of handing you a command whose quoting the + log text broke. Explicitly, the workaround for an OOM on a shared card is: ```bash rocm serve --engine vllm --gpu-memory-utilization 0.5 -# optionally target a specific, less-busy GPU by index +# optionally target a specific, less busy GPU by index rocm serve --engine vllm --gpu 1 --gpu-memory-utilization 0.5 -# for the full busy-GPU-vs-model-too-large breakdown, pass the error to diagnose +# for the full breakdown of a busy GPU versus a model that is too large, pass the error to diagnose rocm diagnose --symptom 'vllm: torch.OutOfMemoryError: HIP out of memory' ``` ## Tool calling -The TUI chat tab attaches tool definitions to every chat request. vLLM rejects +The chat tab in the terminal user interface (TUI) attaches tool definitions to +every chat request. vLLM rejects those with HTTP 400 unless it is launched with `--enable-auto-tool-choice` **and** -a matching `--tool-call-parser`. vLLM does not auto-detect the parser and it is -model-specific, so ROCm CLI never guesses one: +a matching `--tool-call-parser`. vLLM does not auto-detect the parser, and the +parser is model-specific, so ROCm CLI never guesses one: - **Built-in catalog models** carry the correct parser in their recipe metadata, - so tool calling works out of the box (for example, Qwen family → `hermes`, - Llama 3 → `llama3_json`). + so tool calling works out of the box. For example, the Qwen family uses + `hermes` and Llama 3 uses `llama3_json`. - **Other models** (arbitrary Hugging Face repos, or a catalog model forced onto vLLM without authored metadata) need an explicit parser: From 7d2986bc4586739e3cbc78b4bd4b4e901d29364e Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 8 Oct 2026 22:50:43 -0400 Subject: [PATCH 03/18] docs: tighten README flag lists and add lead-ins Keep the rocm diagnose flag list tight by moving the long --report and --send detail below it, add colon lead-ins to the flag lists that had none, and clarify the --family sentence. Signed-off-by: pmoutsias-amd --- README.md | 68 +++++++++++++++++++++++++++++++------------------------ 1 file changed, 38 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index 32443e492..76ba5fb97 100644 --- a/README.md +++ b/README.md @@ -209,8 +209,8 @@ show it as `legacy_rocm_status: detected_unmanaged`. Running `rocm install sdk` creates a separate managed runtime alongside it. By default, `rocm install sdk` installs ROCm 7.14. For ROCm 10.0 or newer, add -`--version` and `--family` with the exact GPU arch from `rocm examine`, for -example `--version 10.1.0 --family gfx1200`. +`--version` and `--family`, and give `--family` the exact GPU arch (the raw `gfx` +code) from `rocm examine`. For example: `--version 10.1.0 --family gfx1200`. Running the command when a managed runtime is already the active default asks first, because the new install takes over as the active default; see @@ -289,6 +289,8 @@ Each result prints an `id:` and an `apply with:` command. The leading `#1`, `#2` are ranking positions for reading order only. `rocm fix` takes the id, not the position. +`rocm diagnose` takes these flags: + - `--symptom` takes raw error text to sharpen keyword scoring. - `--top` caps how many matches are shown in the human-readable output (default 5). `--json` always emits the full, untruncated report. @@ -301,27 +303,6 @@ position. - `--report` shows exactly what this machine would contribute to a problem report, and sends nothing. There is no transport yet, and there will be no automatic one: a report leaves a machine only by its owner's own action. - - The content is deliberately narrow (a schema version, the matched entry, whether - a fix was offered for it, the GPU architecture and which compatibility - matrix snapshot it was checked against, the OS family, distribution and - major version, the ROCm release, the inference engine and its release, the - CLI version), and it carries no host name, user name, file path, or error - text. - - The ROCm release and the inference engine's release are each cut - back to a release, so a build number that would narrow toward one machine - never appears there; the CLI's own version is the exception, since it names - the tool that wrote the report instead of something read off the machine. - The distribution is checked against a list of known names instead of - being repeated from the machine. - - Hardware that is not on AMD's published - compatibility matrix produces no report at all, and the CLI says why. So - does a WSL machine, for a different reason: this CLI does not inspect the - GPU on WSL yet, so it cannot confirm the hardware is on the compatibility - matrix and says that instead of claiming the architecture could not be - read. - `--send`, which requires `--report`, additionally offers a prefilled mail carrying that report. It still sends nothing: the mail opens already filled in with the content `--report` just printed, addressed to `ROCmCLI@amd.com`, @@ -329,13 +310,34 @@ position. `--report` is what guarantees the content is shown before the mail is offered. - A mail client opens only when you asked and the machine looks like - a desktop you are at; over SSH, with no display, or with `ROCM_NO_BROWSER` - set, the address and the link are printed instead, which is also what - happens on a machine with no mail client. It is not combinable with - `--json`, which exists for scripts, and a script is not a person who can - read a mail before sending it. Note that a mail carries your address, which - the report itself does not. +The `--report` content is deliberately narrow (a schema version, the matched entry, whether +a fix was offered for it, the GPU architecture and which compatibility +matrix snapshot it was checked against, the OS family, distribution and +major version, the ROCm release, the inference engine and its release, the +CLI version), and it carries no host name, user name, file path, or error +text. + +The ROCm release and the inference engine's release are each cut +back to a release, so a build number that would narrow toward one machine +never appears there; the CLI's own version is the exception, since it names +the tool that wrote the report instead of something read off the machine. +The distribution is checked against a list of known names instead of +being repeated from the machine. + +Hardware that is not on AMD's published +compatibility matrix produces no report at all, and the CLI says why. So +does a WSL machine, for a different reason: this CLI does not inspect the +GPU on WSL yet, so it cannot confirm the hardware is on the compatibility +matrix and says that instead of claiming the architecture could not be +read. + +With `--send`, a mail client opens only when you asked and the machine looks like +a desktop you are at; over SSH, with no display, or with `ROCM_NO_BROWSER` +set, the address and the link are printed instead, which is also what +happens on a machine with no mail client. It is not combinable with +`--json`, which exists for scripts, and a script is not a person who can +read a mail before sending it. Note that a mail carries your address, which +the report itself does not. `fix` applies a known fix by the `id:` that `diagnose` reported, not the ranking position noted above, which isn't a stable name. Run it with no id @@ -350,6 +352,8 @@ can make, sometimes because it also needs sudo or a reboot), or DIAGNOSE-ONLY carries this marker today, and it is reserved for a future detect-but-cannot-repair failure). +`rocm fix` takes these flags: + - `--dry-run` shows any fix's plan without changing anything. - `--yes` skips the interactive confirmation once you've reviewed it. - `--device-index` pins the discrete GPU index for `fix-9-igpu-dgpu`, marked @@ -736,6 +740,8 @@ utilization graphs, active serving instances, benchmark results, guided actions, and a chat tab backed by any configured provider. See [Interactive interfaces](#interactive-interfaces) for the tab breakdown. +For `rocm dash`: + - `--demo` runs a deterministic synthetic session with no GPU or daemon needed. It works on all platforms. - `--replay ` replays a recorded NDJSON session. @@ -755,6 +761,8 @@ throughput. It is a local smoke test, **not** an official ROCm or AMD benchmark. multi-turn, long-context tool traffic and isn't comparable to `*-agent-bench` quality harnesses. +Flags and output for `rocm bench load`: + - `--endpoint` is the OpenAI-compatible URL shown by `rocm services list` (a plain host address without `/v1` also works). Only `http://` is accepted; `https://` endpoints are rejected outright, because the load generator has no From 638807b13c41ced724a09bf556d141084ea60d49 Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 8 Oct 2026 22:58:39 -0400 Subject: [PATCH 04/18] docs: rejoin prune lock sentence and complete acceptance-script list Signed-off-by: pmoutsias-amd --- README.md | 4 +--- docs/vllm.md | 7 ++++--- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 76ba5fb97..ca9ab1734 100644 --- a/README.md +++ b/README.md @@ -682,9 +682,7 @@ at the directory. That wait lasts as long as the launch does and has no timeout, so it is usually imperceptible but is not bounded: on an interactive terminal `prune` prints `Waiting for a launch already under way…` while it waits, including under `--dry-run`. That notice goes to stderr and is suppressed when -stderr is not a terminal, so a piped or scripted prune waits silently. - -The same +stderr is not a terminal, so a piped or scripted prune waits silently. The same lock runs in the other direction, so a `rocm serve` started while a `prune` is scanning waits for the prune. diff --git a/docs/vllm.md b/docs/vllm.md index 1d2805c18..eb5c00ea1 100644 --- a/docs/vllm.md +++ b/docs/vllm.md @@ -166,10 +166,11 @@ The acceptance script is Linux or WSL only. It does the following: - Checks `/health` and `/v1/completions`. - Verifies that loaded ROCm libraries come from the managed TheRock SDK wheel directories. +- Rejects external vLLM command overrides. +- Does not allow CPU fallback. -It rejects external vLLM command overrides and does not allow CPU fallback. It -defaults to the active exact runtime key. If you pass `--runtime-id`, use an -exact runtime key or an unambiguous runtime id. +The script defaults to the active exact runtime key. If you pass `--runtime-id`, +use an exact runtime key or an unambiguous runtime id. ### Source build notes From 3d8b1f9882d0ead55f0e8fc96c2eeefad80b880b Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 8 Oct 2026 23:12:18 -0400 Subject: [PATCH 05/18] docs: add install verification and rocm uninstall guidance Signed-off-by: pmoutsias-amd --- README.md | 37 ++++++++++++++++++++++++++++++++++++- 1 file changed, 36 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index ca9ab1734..6a41dc95e 100644 --- a/README.md +++ b/README.md @@ -107,6 +107,18 @@ $env:ROCM_CLI_CHANNEL = "nightly" irm https://raw.githubusercontent.com/ROCm/rocm-cli/main/install.ps1 | iex ``` +### Verify the installation + +Open a new terminal so the updated `PATH` takes effect, then run: + +``` +rocm version +``` + +The output shows the ROCm CLI version with its release tag or branch and commit +hash, the ROCm SDK this machine would use, and the GPU driver version. The SDK +and driver lines read `not detected` when none is found. + ## Build from source Building requires [Rust](https://rustup.rs/); the pinned toolchain in @@ -861,7 +873,30 @@ rocm logs [--service ] [--search TERM ...] rocm uninstall [--yes] [--dry-run] [--keep-binaries] [--keep-config] [--keep-data] [--keep-cache] -``` + [--force-dev-binaries] +``` + +`rocm uninstall` removes the files ROCm CLI manages on this computer. Use +`--dry-run` to preview what it removes, and `--yes` to skip the confirmation. +Each `--keep-*` flag leaves one category in place: + +- `--keep-binaries` keeps the installed ROCm CLI binaries. +- `--keep-config` keeps your saved settings. +- `--keep-data` keeps app data such as logs, services, engines, and the `uv` + package cache. +- `--keep-cache` keeps the caches under the cache directory. It doesn't cover + the `uv` package cache, which `--keep-data` controls. + +When you run `rocm` from a cargo `target` build, `uninstall` skips binary +removal. Pass `--force-dev-binaries` to remove the debug and release binaries +beside it. + +Before removing anything, `uninstall` notes any shared caches it leaves alone. +It also warns when managed service records exist, because it doesn't stop their +background processes, and when remote sessions exist. For each remote session, +run `rocm remote stop ` first. Otherwise the model keeps running on the +remote machine with its endpoint published, and the record that tracked it is +gone. ### Shell completions From 6920612052e146aeffd902fa3f43a2a1947f93d0 Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 8 Oct 2026 23:17:40 -0400 Subject: [PATCH 06/18] docs: clarify verify and uninstall guidance Add expected output and a PATH fallback to the verify section. Name the four uninstall categories, list managed ROCm SDK installs under --keep-data, and split the uninstall warnings into a list that also names the shared caches left in place. Signed-off-by: pmoutsias-amd --- README.md | 39 +++++++++++++++++++++++++-------------- 1 file changed, 25 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 6a41dc95e..8ce8349cd 100644 --- a/README.md +++ b/README.md @@ -111,13 +111,18 @@ irm https://raw.githubusercontent.com/ROCm/rocm-cli/main/install.ps1 | iex Open a new terminal so the updated `PATH` takes effect, then run: -``` +```bash rocm version ``` The output shows the ROCm CLI version with its release tag or branch and commit -hash, the ROCm SDK this machine would use, and the GPU driver version. The SDK -and driver lines read `not detected` when none is found. +hash, the ROCm SDK that ROCm CLI would use on this computer, and the GPU driver +version. A successful install prints the CLI version line. The SDK and driver +lines read `not detected` when none is found, which is expected on a computer +where you haven't installed ROCm or a driver yet. + +If the shell reports `rocm: command not found`, confirm that `~/.local/bin` is on +your `PATH`. ## Build from source @@ -876,14 +881,15 @@ rocm uninstall [--yes] [--dry-run] [--force-dev-binaries] ``` -`rocm uninstall` removes the files ROCm CLI manages on this computer. Use -`--dry-run` to preview what it removes, and `--yes` to skip the confirmation. -Each `--keep-*` flag leaves one category in place: +`rocm uninstall` removes four categories of files that ROCm CLI manages on this +computer: its binaries, configuration, data, and caches. Use `--dry-run` to +preview what it removes, and `--yes` to skip the confirmation. Each `--keep-*` +flag leaves one category in place: - `--keep-binaries` keeps the installed ROCm CLI binaries. - `--keep-config` keeps your saved settings. -- `--keep-data` keeps app data such as logs, services, engines, and the `uv` - package cache. +- `--keep-data` keeps app data such as logs, services, engines, the managed ROCm + SDK installs, and the `uv` package cache. - `--keep-cache` keeps the caches under the cache directory. It doesn't cover the `uv` package cache, which `--keep-data` controls. @@ -891,12 +897,17 @@ When you run `rocm` from a cargo `target` build, `uninstall` skips binary removal. Pass `--force-dev-binaries` to remove the debug and release binaries beside it. -Before removing anything, `uninstall` notes any shared caches it leaves alone. -It also warns when managed service records exist, because it doesn't stop their -background processes, and when remote sessions exist. For each remote session, -run `rocm remote stop ` first. Otherwise the model keeps running on the -remote machine with its endpoint published, and the record that tracked it is -gone. +Before removing anything, `uninstall` prints warnings about two situations: + +- Managed service records exist. `uninstall` doesn't stop their background + processes. +- Remote sessions exist. For each one, run `rocm remote stop ` first. + Otherwise the model keeps running on the remote machine with its endpoint + published, and the record that tracked it is gone. + +It also lists the shared caches it leaves in place: the `uv` package cache and the +Hugging Face model cache, when they live outside the ROCm CLI directories. Delete +them yourself if you no longer need them. ### Shell completions From e764ebabd57bd9213775fb5875d36abc7a396f7a Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 8 Oct 2026 23:26:16 -0400 Subject: [PATCH 07/18] docs: link install pages to uninstall guidance Point the ROCm installation section and the site install page at Logs and cleanup, so readers who finish installing can find how to remove ROCm CLI. Signed-off-by: pmoutsias-amd --- README.md | 3 +++ docs/rocm-docs/install/installation.md | 3 +++ 2 files changed, 6 insertions(+) diff --git a/README.md b/README.md index 8ce8349cd..0959001d5 100644 --- a/README.md +++ b/README.md @@ -395,6 +395,9 @@ rocm update [--apply] [--runtime KEY] [--activate] [--dry-run] by ROCm CLI. It can install ROCm 7.14, 10.0, or 10.1. To install 10.0 or newer, follow the steps under ROCm 10 and newer, below. +To remove ROCm CLI and what it manages, see +[Logs and cleanup](#logs-and-cleanup). + #### Compiler toolchain (--devel) Pass `--devel` to also install the compiler and headers needed to build GPU diff --git a/docs/rocm-docs/install/installation.md b/docs/rocm-docs/install/installation.md index 53cabf3e1..e62863e57 100644 --- a/docs/rocm-docs/install/installation.md +++ b/docs/rocm-docs/install/installation.md @@ -28,5 +28,8 @@ serving is Linux or WSL2 only (see :end-before: "See [CONTRIBUTING.md]" ``` +To remove ROCm CLI and what it manages, see +[Logs and cleanup](../commands.md#logs-and-cleanup). + See [Contributing](../about/contributing.md) for the full development setup, test commands, and commit-signing requirements. From a87de8437223014968b76cd8301367080699e914 Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 8 Oct 2026 23:35:34 -0400 Subject: [PATCH 08/18] docs: give the install page uninstall pointer its own heading Signed-off-by: pmoutsias-amd --- docs/rocm-docs/install/installation.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/rocm-docs/install/installation.md b/docs/rocm-docs/install/installation.md index e62863e57..f3ee20827 100644 --- a/docs/rocm-docs/install/installation.md +++ b/docs/rocm-docs/install/installation.md @@ -28,8 +28,10 @@ serving is Linux or WSL2 only (see :end-before: "See [CONTRIBUTING.md]" ``` -To remove ROCm CLI and what it manages, see -[Logs and cleanup](../commands.md#logs-and-cleanup). - See [Contributing](../about/contributing.md) for the full development setup, test commands, and commit-signing requirements. + +## Uninstall ROCm CLI + +To remove ROCm CLI and what it manages, see +[Logs and cleanup](../commands.md#logs-and-cleanup). From 74d08fb23e02b44a6be7b025a3774015c44225b7 Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 8 Oct 2026 23:35:34 -0400 Subject: [PATCH 09/18] docs: fix command synopses and document model, chat provider, and automation modes Signed-off-by: pmoutsias-amd --- README.md | 55 +++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 41 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 0959001d5..c9fe11777 100644 --- a/README.md +++ b/README.md @@ -291,7 +291,7 @@ for its ROCm build and compiled GPU architectures: ### Diagnose and fix ``` -rocm diagnose [--symptom TEXT] [--top N] [--json] [--distro [NAME]] [--report] +rocm diagnose [--symptom TEXT] [--top N] [--json] [--distro [NAME]] [--report [--send]] rocm diagnose --model [--json] rocm fix [] [--yes] [--dry-run] [--device-index N] ``` @@ -585,12 +585,26 @@ fetches no weights and makes no network call. The verdict is `ready`, engine `rocm serve` would use; a `blocked` one names curated models that would run here instead. -`undetermined` is a real answer and not a failure: it is what you get when the -recipe catalog could not be read, when this machine's GPU memory could not be -measured, or when the model is not one of the curated recipes (`rocm model` -lists those). None of those say anything about whether the model fits, so none -of them are reported as though they did. `rocm serve` still accepts a model -outside the catalog; it cannot tell you in advance how it will go. +`undetermined` is a real answer, not a failure. You get it when any of these is +true: + +- The recipe catalog could not be read. +- This machine's GPU memory could not be measured. +- The model is not one of the curated recipes. [`rocm model`](#curated-models) + lists those. + +None of these says anything about whether the model fits, so none of them is +reported as though it did. `rocm serve` still accepts a model outside the +catalog, but `diagnose` cannot tell you in advance how it will go. + +### Curated models + +``` +rocm model [--verbose] +``` + +`rocm model` (alias `rocm models`) lists the recommended local models, grouped by +the hardware they target. Pass `--verbose` to show detailed recipe diagnostics. ### Model serving @@ -802,14 +816,21 @@ Flags and output for `rocm bench load`: ### Chat ``` -rocm chat [--provider anthropic|openai|...] [--model NAME] [--prompt TEXT] [--tools] +rocm chat [--provider local|openai|anthropic] [--model NAME] [--prompt TEXT] [--tools] [--temperature TEMP] [--top-p PROB] [--max-tokens N] ``` -Chat with an AI provider from the terminal. Reads from stdin when `--prompt` is -omitted. `--temperature`, `--top-p`, and `--max-tokens` are optional sampling -controls forwarded to the request; each is independent, so omit any of them to -use the provider's default. +Chat with an AI provider from the terminal. The command reads from stdin when +you omit `--prompt`. + +- `--provider` selects `local` (a model that `rocm serve` runs on this machine), + `openai`, or `anthropic`. A cloud provider needs setup first: run + `rocm config enable-provider ` and + `rocm config set-provider-key `. See [Configuration](#configuration). +- `--tools` lets an OpenAI-compatible provider request ROCm tool calls. +- `--temperature`, `--top-p`, and `--max-tokens` are optional sampling controls + forwarded to the request. Each is independent, so omit any of them to use the + provider's default. ### ComfyUI @@ -836,7 +857,13 @@ rocm automations enable [--mode observe|propose|contained] rocm automations disable ``` -Optional background checks that can propose or apply changes automatically. +Automations are optional background checks, called watchers, that can propose or +apply changes automatically. `list` shows the available watchers and their IDs. +`enable` turns one on, and `--mode` sets how far it can act: + +- `observe` reports what it finds and changes nothing. +- `propose` suggests a change for you to approve. +- `contained` applies a change on its own, within a contained, reversible scope. ### Configuration @@ -877,7 +904,7 @@ provider settings are left untouched. ### Logs and cleanup ``` -rocm logs [--service ] [--search TERM ...] +rocm logs [--service ] [--search TERM ...] [QUERY ...] rocm uninstall [--yes] [--dry-run] [--keep-binaries] [--keep-config] [--keep-data] [--keep-cache] From 2153b6edd9148c813de1f938c948d7abe774ec22 Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 8 Oct 2026 23:37:44 -0400 Subject: [PATCH 10/18] docs: clarify that rocm chat --provider applies only to one-prompt use Signed-off-by: pmoutsias-amd --- README.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index c9fe11777..793bef7d5 100644 --- a/README.md +++ b/README.md @@ -820,12 +820,15 @@ rocm chat [--provider local|openai|anthropic] [--model NAME] [--prompt TEXT] [-- [--temperature TEMP] [--top-p PROB] [--max-tokens N] ``` -Chat with an AI provider from the terminal. The command reads from stdin when -you omit `--prompt`. - -- `--provider` selects `local` (a model that `rocm serve` runs on this machine), - `openai`, or `anthropic`. A cloud provider needs setup first: run - `rocm config enable-provider ` and +Chat with an AI provider from the terminal. In an interactive terminal, +`rocm chat` without `--prompt` opens the dashboard chat, where you switch +providers with `/provider `. With `--prompt`, or with input piped to +standard input, it sends one prompt and prints the reply. + +- `--provider` selects `local` (a model that `rocm serve` runs on this machine, + the default), `openai`, or `anthropic`. It applies to the one-prompt form; the + dashboard chat ignores it and prints a note. A cloud provider needs setup + first: run `rocm config enable-provider ` and `rocm config set-provider-key `. See [Configuration](#configuration). - `--tools` lets an OpenAI-compatible provider request ROCm tool calls. - `--temperature`, `--top-p`, and `--max-tokens` are optional sampling controls From f5fee374d3c4cbc75fcdc0e13b58fd6c15408e76 Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 8 Oct 2026 23:39:52 -0400 Subject: [PATCH 11/18] docs: correct contained-mode, chat flag, and model verbose descriptions Signed-off-by: pmoutsias-amd --- README.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 793bef7d5..9fc1f7c9c 100644 --- a/README.md +++ b/README.md @@ -604,7 +604,9 @@ rocm model [--verbose] ``` `rocm model` (alias `rocm models`) lists the recommended local models, grouped by -the hardware they target. Pass `--verbose` to show detailed recipe diagnostics. +the hardware they target. Pass `--verbose` to list every recipe, including ones +hidden from the default list, with details such as aliases, data type, minimum +GPU memory, and preferred engines. ### Model serving @@ -826,11 +828,12 @@ providers with `/provider `. With `--prompt`, or with input piped to standard input, it sends one prompt and prints the reply. - `--provider` selects `local` (a model that `rocm serve` runs on this machine, - the default), `openai`, or `anthropic`. It applies to the one-prompt form; the - dashboard chat ignores it and prints a note. A cloud provider needs setup + the default), `openai`, or `anthropic`. A cloud provider needs setup first: run `rocm config enable-provider ` and `rocm config set-provider-key `. See [Configuration](#configuration). - `--tools` lets an OpenAI-compatible provider request ROCm tool calls. +- `--provider`, `--model`, and `--tools` apply to the one-prompt form. The + dashboard chat ignores them. - `--temperature`, `--top-p`, and `--max-tokens` are optional sampling controls forwarded to the request. Each is independent, so omit any of them to use the provider's default. @@ -865,8 +868,11 @@ apply changes automatically. `list` shows the available watchers and their IDs. `enable` turns one on, and `--mode` sets how far it can act: - `observe` reports what it finds and changes nothing. -- `propose` suggests a change for you to approve. -- `contained` applies a change on its own, within a contained, reversible scope. +- `propose` queues a suggested change for you to approve, for watchers that can + make one. +- `contained` lets a watcher act on its own only where that is safe. Today only + `server-recover` does: it restarts a managed service that failed. The other + watchers stay review-gated or record only in this mode. ### Configuration From c92e88a3308d055111d90b3232155aca95d5f89a Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 8 Oct 2026 23:59:17 -0400 Subject: [PATCH 12/18] docs: clarify uninstall cache and data scope, add APU case to diagnose Distinguish the managed uv cache from the shared one, list downloaded models under --keep-data, and include the APU case in the undetermined verdict conditions. Signed-off-by: pmoutsias-amd --- README.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 9fc1f7c9c..3655ce7e3 100644 --- a/README.md +++ b/README.md @@ -589,7 +589,8 @@ run here instead. true: - The recipe catalog could not be read. -- This machine's GPU memory could not be measured. +- This machine's GPU memory could not be measured. This includes an integrated + GPU (APU), where the CLI can't yet read the memory pool the engine uses. - The model is not one of the curated recipes. [`rocm model`](#curated-models) lists those. @@ -927,10 +928,10 @@ flag leaves one category in place: - `--keep-binaries` keeps the installed ROCm CLI binaries. - `--keep-config` keeps your saved settings. -- `--keep-data` keeps app data such as logs, services, engines, the managed ROCm - SDK installs, and the `uv` package cache. +- `--keep-data` keeps app data such as logs, services, engines, downloaded + models, the managed ROCm SDK installs, and the managed `uv` package cache. - `--keep-cache` keeps the caches under the cache directory. It doesn't cover - the `uv` package cache, which `--keep-data` controls. + the managed `uv` package cache, which `--keep-data` controls. When you run `rocm` from a cargo `target` build, `uninstall` skips binary removal. Pass `--force-dev-binaries` to remove the debug and release binaries @@ -944,9 +945,10 @@ Before removing anything, `uninstall` prints warnings about two situations: Otherwise the model keeps running on the remote machine with its endpoint published, and the record that tracked it is gone. -It also lists the shared caches it leaves in place: the `uv` package cache and the -Hugging Face model cache, when they live outside the ROCm CLI directories. Delete -them yourself if you no longer need them. +It also lists the shared caches it leaves in place, when they live outside the +ROCm CLI directories: a `uv` package cache that `UV_CACHE_DIR` or the default +`~/.cache/uv` points to, and the Hugging Face model cache. Delete them yourself +if you no longer need them. ### Shell completions From 033fa1e1f5672e6d65a5365f87c8c934a2df5b5e Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Fri, 9 Oct 2026 09:23:20 -0400 Subject: [PATCH 13/18] docs: address review findings on uninstall, ROCm versions, and synopses Note that --keep-* flags don't protect their files by themselves on the default layout, where config and data share ~/.rocm. State the supported ROCm versions and describe the default as the latest stable release. Correct the storage and diagnose synopses, point the undetermined list at rocm model --verbose, narrow the "never removed" claim to rocm storage, and resync the copied getting-started sentence. Signed-off-by: pmoutsias-amd --- README.md | 28 +++++++++++++++++++--------- docs/engine-plugins.md | 6 ++++-- docs/rocm-docs/getting-started.md | 4 ++-- 3 files changed, 25 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 3655ce7e3..5f86ad7a4 100644 --- a/README.md +++ b/README.md @@ -225,7 +225,8 @@ environment. On machines with an existing ROCm install, `rocm examine` will show it as `legacy_rocm_status: detected_unmanaged`. Running `rocm install sdk` creates a separate managed runtime alongside it. -By default, `rocm install sdk` installs ROCm 7.14. For ROCm 10.0 or newer, add +ROCm CLI supports ROCm 7.14, 10.0, and 10.1. By default, `rocm install sdk` +installs the latest stable release, currently 7.14. For ROCm 10.0 or newer, add `--version` and `--family`, and give `--family` the exact GPU arch (the raw `gfx` code) from `rocm examine`. For example: `--version 10.1.0 --family gfx1200`. @@ -291,7 +292,7 @@ for its ROCm build and compiled GPU architectures: ### Diagnose and fix ``` -rocm diagnose [--symptom TEXT] [--top N] [--json] [--distro [NAME]] [--report [--send]] +rocm diagnose [--symptom TEXT] [--top N] [--json] [--distro [NAME] | --report [--send]] rocm diagnose --model [--json] rocm fix [] [--yes] [--dry-run] [--device-index N] ``` @@ -392,7 +393,8 @@ rocm update [--apply] [--runtime KEY] [--activate] [--dry-run] ``` `install sdk` downloads TheRock ROCm wheels into a Python environment managed -by ROCm CLI. It can install ROCm 7.14, 10.0, or 10.1. To install 10.0 or newer, +by ROCm CLI. ROCm CLI supports ROCm 7.14, 10.0, and 10.1. Without `--version`, +it installs the latest stable release, currently 7.14. To install 10.0 or newer, follow the steps under ROCm 10 and newer, below. To remove ROCm CLI and what it manages, see @@ -530,7 +532,7 @@ updating a few times adds up. `rocm storage` shows where the space went and frees the parts that are safe to remove: ``` -rocm storage [report] [--json] +rocm storage [report [--json]] rocm storage remove-old-installs [--keep N] [--dry-run] [--yes] rocm storage remove-downloads [--dry-run] [--yes] ``` @@ -558,8 +560,10 @@ The report also lists `local server records`: one JSON record plus the engine's log for each `rocm serve --managed` launch, kept after the server stops. `rocm services list --all` lists them, and no `rocm storage` command removes them. It also lists the `uv` package cache, the Hugging Face model cache, and -downloaded models. Those last three are shared with other tools and are never -removed by ROCm CLI. +downloaded models. Those last three are shared with other tools, and no +`rocm storage` command removes them. `rocm uninstall` removes downloaded models +along with the data directory unless you pass `--keep-data`; see +[Logs and cleanup](#logs-and-cleanup). ### Inference engines @@ -591,8 +595,8 @@ true: - The recipe catalog could not be read. - This machine's GPU memory could not be measured. This includes an integrated GPU (APU), where the CLI can't yet read the memory pool the engine uses. -- The model is not one of the curated recipes. [`rocm model`](#curated-models) - lists those. +- The model is not one of the curated recipes. [`rocm model --verbose`](#curated-models) + lists all of them, including recipes hidden from the default list. None of these says anything about whether the model fits, so none of them is reported as though it did. `rocm serve` still accepts a model outside the @@ -924,7 +928,7 @@ rocm uninstall [--yes] [--dry-run] `rocm uninstall` removes four categories of files that ROCm CLI manages on this computer: its binaries, configuration, data, and caches. Use `--dry-run` to preview what it removes, and `--yes` to skip the confirmation. Each `--keep-*` -flag leaves one category in place: +flag skips the removal of one category: - `--keep-binaries` keeps the installed ROCm CLI binaries. - `--keep-config` keeps your saved settings. @@ -933,6 +937,12 @@ flag leaves one category in place: - `--keep-cache` keeps the caches under the cache directory. It doesn't cover the managed `uv` package cache, which `--keep-data` controls. +By default, the configuration and data directories are both `~/.rocm`, and the +cache directory is `~/.rocm/cache`. `uninstall` removes a category's directory +unless you keep that category, so on a default installation a flag doesn't +protect its files by itself. To keep your settings and data, pass `--keep-config` +and `--keep-data` together. Run with `--dry-run` first to confirm what is kept. + When you run `rocm` from a cargo `target` build, `uninstall` skips binary removal. Pass `--force-dev-binaries` to remove the debug and release binaries beside it. diff --git a/docs/engine-plugins.md b/docs/engine-plugins.md index 23a3c4e87..adec65603 100644 --- a/docs/engine-plugins.md +++ b/docs/engine-plugins.md @@ -95,8 +95,10 @@ Installer policy: - External plugins under the rocm-cli data directory are not touched by install or upgrade. - `rocm uninstall` removes the data directory by default. Use - `rocm uninstall --keep-data` when external plugins, managed runtimes, - service records, or model cache entries should be preserved. + `rocm uninstall --keep-data --keep-config` when external plugins, managed + runtimes, service records, or model cache entries should be preserved. On the + default layout the configuration and data directories are the same + directory, so both flags are needed. No fallback engine is selected automatically. If an engine adapter is missing or cannot satisfy the requested device policy, the command must fail until the diff --git a/docs/rocm-docs/getting-started.md b/docs/rocm-docs/getting-started.md index 86bfab3a9..cb5e78004 100644 --- a/docs/rocm-docs/getting-started.md +++ b/docs/rocm-docs/getting-started.md @@ -33,8 +33,8 @@ that approve it without a prompt, and the ROCm 10 and newer requirements. ``` You can also serve any compatible Hugging Face model directly. See -[Model serving](commands.md#model-serving) for the GGUF-vs-safetensors rule, -since which form works depends on the engine your GPU selects. +[Model serving](commands.md#model-serving) for the GGUF versus safetensors rule, +because which form works depends on the engine your GPU selects. ```{include} ../../README.md :start-after: "form works depends on the engine your GPU selects." From 558fd4592b9a6fb80c2b9295ed485550130dbb18 Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Fri, 9 Oct 2026 09:39:19 -0400 Subject: [PATCH 14/18] docs: avoid calling 7.14 the latest stable ROCm release ROCm 10.1 is the latest stable release, so describe the unpinned `rocm install sdk` result as currently ROCm 7.14 and point to the `--version` and `--family` steps for 10.0 and 10.1. Signed-off-by: pmoutsias-amd --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 5f86ad7a4..a1094f78f 100644 --- a/README.md +++ b/README.md @@ -225,10 +225,10 @@ environment. On machines with an existing ROCm install, `rocm examine` will show it as `legacy_rocm_status: detected_unmanaged`. Running `rocm install sdk` creates a separate managed runtime alongside it. -ROCm CLI supports ROCm 7.14, 10.0, and 10.1. By default, `rocm install sdk` -installs the latest stable release, currently 7.14. For ROCm 10.0 or newer, add -`--version` and `--family`, and give `--family` the exact GPU arch (the raw `gfx` -code) from `rocm examine`. For example: `--version 10.1.0 --family gfx1200`. +ROCm CLI supports ROCm 7.14, 10.0, and 10.1. Without `--version`, +`rocm install sdk` currently installs ROCm 7.14. To install ROCm 10.0 or 10.1, +add `--version` and `--family`, and give `--family` the exact GPU arch (the raw +`gfx` code) from `rocm examine`. For example: `--version 10.1.0 --family gfx1200`. Running the command when a managed runtime is already the active default asks first, because the new install takes over as the active default; see @@ -394,8 +394,8 @@ rocm update [--apply] [--runtime KEY] [--activate] [--dry-run] `install sdk` downloads TheRock ROCm wheels into a Python environment managed by ROCm CLI. ROCm CLI supports ROCm 7.14, 10.0, and 10.1. Without `--version`, -it installs the latest stable release, currently 7.14. To install 10.0 or newer, -follow the steps under ROCm 10 and newer, below. +it currently installs ROCm 7.14. To install 10.0 or 10.1, follow the steps under +ROCm 10 and newer, below. To remove ROCm CLI and what it manages, see [Logs and cleanup](#logs-and-cleanup). From 3f564b9c5fd398f5e4d52f9acc97cb537aa19dfc Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Fri, 9 Oct 2026 15:32:47 -0400 Subject: [PATCH 15/18] docs: address review comments on getting started, uninstall, and vLLM - Start the "Configure ROCm" section with `rocm examine` and keyed on `active_runtime_status: ready`; `rocm install sdk` is for a different version or configuration. - Name the command in the "install over an active default" sentence, and update the hand-copied sentence and include anchor in getting-started.md to match. - Add a `rocm uninstall` example and a `--dry-run` note to the Uninstall section of installation.md. - vllm.md: say "torch release" and split the long discovery sentence in three. Signed-off-by: pmoutsias-amd --- README.md | 21 +++++++++++++------ docs/rocm-docs/getting-started.md | 6 +++--- docs/rocm-docs/install/installation.md | 8 ++++++- docs/vllm.md | 29 +++++++++++++------------- 4 files changed, 40 insertions(+), 24 deletions(-) diff --git a/README.md b/README.md index a1094f78f..15ef62c35 100644 --- a/README.md +++ b/README.md @@ -214,24 +214,33 @@ GPU or daemon. ## Configure ROCm and serve a model -Before serving a model, ensure a managed ROCm runtime is configured: +Before serving a model, check whether a managed ROCm runtime is already +configured: + +``` +rocm examine +``` + +When `active_runtime_status` reports `ready`, you can serve a model now. A +machine can also have an existing ROCm install that `rocm examine` reports as +`legacy_rocm_status: detected_unmanaged`, alongside the managed runtime. To +install a runtime, or to get a different version or configuration, run: ``` rocm install sdk ``` This downloads TheRock ROCm wheels and a matching PyTorch stack into a managed -environment. On machines with an existing ROCm install, `rocm examine` will -show it as `legacy_rocm_status: detected_unmanaged`. Running `rocm install sdk` -creates a separate managed runtime alongside it. +environment. Running `rocm install sdk` creates a managed runtime alongside any +legacy install. ROCm CLI supports ROCm 7.14, 10.0, and 10.1. Without `--version`, `rocm install sdk` currently installs ROCm 7.14. To install ROCm 10.0 or 10.1, add `--version` and `--family`, and give `--family` the exact GPU arch (the raw `gfx` code) from `rocm examine`. For example: `--version 10.1.0 --family gfx1200`. -Running the command when a managed runtime is already the active default asks -first, because the new install takes over as the active default; see +Running `rocm install sdk` when a managed runtime is already the active default +asks first, because the new install takes over as the active default; see [ROCm installation](https://github.com/ROCm/rocm-cli/blob/main/README.md#rocm-installation) for that gate, the flags that approve it without a prompt, and the ROCm 10 and newer requirements. diff --git a/docs/rocm-docs/getting-started.md b/docs/rocm-docs/getting-started.md index cb5e78004..06cc84ef3 100644 --- a/docs/rocm-docs/getting-started.md +++ b/docs/rocm-docs/getting-started.md @@ -15,15 +15,15 @@ SPDX-License-Identifier: MIT ```{include} ../../README.md :start-after: "## Configure ROCm and serve a model" -:end-before: "Running the command when a" +:end-before: "Running `rocm install sdk` when a" ``` -Running the command when a managed runtime is already the active default asks -first, because the new install takes over as the active default; see +Running `rocm install sdk` when a managed runtime is already the active default +asks first, because the new install takes over as the active default; see [ROCm installation](commands.md#rocm-installation) for that gate, the flags that approve it without a prompt, and the ROCm 10 and newer requirements. diff --git a/docs/rocm-docs/install/installation.md b/docs/rocm-docs/install/installation.md index f3ee20827..01ea95433 100644 --- a/docs/rocm-docs/install/installation.md +++ b/docs/rocm-docs/install/installation.md @@ -33,5 +33,11 @@ commands, and commit-signing requirements. ## Uninstall ROCm CLI -To remove ROCm CLI and what it manages, see +To remove ROCm CLI and what it manages: + +```bash +rocm uninstall +``` + +Pass `--dry-run` first to preview what it removes. For the full flag list, see [Logs and cleanup](../commands.md#logs-and-cleanup). diff --git a/docs/vllm.md b/docs/vllm.md index eb5c00ea1..1e240ab80 100644 --- a/docs/vllm.md +++ b/docs/vllm.md @@ -36,11 +36,11 @@ as a `torch_alignment:` line. A torch that already executes a GPU kernel against the installed SDK is kept exactly as it is, whichever installer put it there. Otherwise the runtime is -moved to the SDK's *build* of the torch *release* the engine pins: the release -comes from the engine, which was built against it, and the build comes from the -SDK, whose libraries it has to load. A `device_check:` line reports what the -result can actually do, and a realignment also reports what the runtime could do -before it. +moved to the SDK's *build* of the torch *release* that the engine pins: the +release comes from the engine, which was built against it, and the build comes +from the SDK, whose libraries it has to load. A `device_check:` line reports +what the result can actually do, and a realignment also reports what the runtime +could do before it. Set `ROCM_CLI_DISABLE_TORCH_ALIGNMENT` to keep whatever torch is installed and skip the replacement: @@ -63,9 +63,10 @@ runtime that cannot execute a kernel will fail at serving time. ## ROCm 10.x wheel discovery -For most ROCm SDK versions, `rocm engines install vllm` pins a fixed vLLM wheel -and index URL. Any ROCm SDK 10.x version is different: AMD publishes vLLM, -flash-attn, and amd-aiter there under a rotating dev-tag filename (for example +For ROCm SDK 7.x versions, including 7.14, `rocm engines install vllm` pins a +fixed vLLM wheel and index URL. Any ROCm SDK 10.x version is different: AMD +publishes vLLM, flash-attn, and amd-aiter there under a rotating dev-tag +filename (for example `vllm-0.27.1.dev5+rocm10.0.0.gf46a9dfe2.d20260826-cp314-cp314-linux_x86_64.whl`), so there is no fixed filename to pin in the adapter. @@ -80,13 +81,13 @@ This route is selected by `major.minor`, so `10.0.0` and `10.1.0` discover through *different* rows: they pin different vLLM minors and read vLLM from different index URLs, because AMD stages ROCm 10.1's vLLM builds on a separate host from 10.0's production index. Both rows take their torch stack from the -same `whl-next` index, which is where `rocm install sdk` resolves either SDK's -own `+rocmX.Y` torch from, but vLLM's discovery is independent of that: it +same `whl-next` index, which is also where `rocm install sdk` resolves either +SDK's own `+rocmX.Y` torch from. vLLM's discovery is independent of that: it resolves its own torch, torchvision, and torchaudio pins fresh from the row's -own static version prefixes, instead of reusing whatever the SDK install -resolved, which might be a different torch version than this row pins. Patch -and any dev or pre-release suffix are still ignored within a row, because AMD -rotates those constantly. +own static version prefixes, instead of reusing what the SDK install resolved. +The SDK install might have resolved a different torch version than the row pins. +Patch and any dev or pre-release suffix are still ignored within a row, because +AMD rotates those constantly. The install resolves each package's current wheel, including torch, from the row's index with `uv pip install --dry-run --reinstall`, parses the version it From d807a978db1bd13ffaddc7b6fd993c7676ec375a Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Mon, 12 Oct 2026 10:28:56 -0400 Subject: [PATCH 16/18] docs: correct stale install, serve, and cleanup guidance - Say `--family` is needed for ROCm 10 only when ROCm CLI can't resolve an exact GPU arch or you target a different one. - Add the Windows and `ROCM_CLI_INSTALL_DIR` forms to the PATH advice. - Add the missing flags to the `rocm fix`, `rocm serve`, and `rocm engines install` synopses, with short descriptions. - Drop the unsupported "including 7.14" claim from the vLLM page. - Update the `rocm chat` and `rocm diagnose --send` help text, and the `rocm storage` heading and models note, to match current behavior. Signed-off-by: pmoutsias-amd --- README.md | 44 +++++++++++++++++++++++++++++++--------- apps/rocm/src/main.rs | 27 ++++++++++++++---------- apps/rocm/src/storage.rs | 6 +++--- docs/vllm.md | 4 ++-- 4 files changed, 55 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 8ec0094ac..da7eacfa7 100644 --- a/README.md +++ b/README.md @@ -121,8 +121,10 @@ version. A successful install prints the CLI version line. The SDK and driver lines read `not detected` when none is found, which is expected on a computer where you haven't installed ROCm or a driver yet. -If the shell reports `rocm: command not found`, confirm that `~/.local/bin` is on -your `PATH`. +If the shell reports `rocm: command not found`, confirm that the install +directory is on your `PATH`. That is `~/.local/bin` on Linux, +`%USERPROFILE%\.local\bin` on Windows, or the folder you set with +`ROCM_CLI_INSTALL_DIR`. ## Build from source @@ -236,8 +238,9 @@ legacy install. ROCm CLI supports ROCm 7.14, 10.0, and 10.1. Without `--version`, `rocm install sdk` currently installs ROCm 7.14. To install ROCm 10.0 or 10.1, -add `--version` and `--family`, and give `--family` the exact GPU arch (the raw -`gfx` code) from `rocm examine`. For example: `--version 10.1.0 --family gfx1200`. +add `--version`. If ROCm CLI can't tell which exact GPU arch this computer has, +or you want a different one, also add `--family` with the raw `gfx` code from +`rocm examine`. For example: `--version 10.1.0 --family gfx1200`. Running `rocm install sdk` when a managed runtime is already the active default asks first, because the new install takes over as the active default; see @@ -303,7 +306,7 @@ for its ROCm build and compiled GPU architectures: ``` rocm diagnose [--symptom TEXT] [--top N] [--json] [--distro [NAME] | --report [--send]] rocm diagnose --model [--json] -rocm fix [] [--yes] [--dry-run] [--device-index N] +rocm fix [] [--yes] [--dry-run] [--device-index N] [--json] ``` `diagnose` matches this machine against a fixed catalog of known @@ -386,6 +389,8 @@ failure). - `--device-index` pins the discrete GPU index for `fix-9-igpu-dgpu`, marked NEEDS-ARG; without it, that fix only prints the `rocminfo` (Linux) or `hipInfo.exe` (Windows) query needed to find the index and makes no change. +- `--json` prints the catalog as JSON for tooling. It's valid only without a fix + id. ### ROCm installation @@ -464,9 +469,11 @@ active default runtime, not about what a named prefix loses. #### ROCm 10 and newer -ROCm 10 and newer ship from a different source layout. You opt in by passing two -things together: pin the version with `--version`, and name the exact GPU arch, -using the raw `gfx` code instead of a family label: +ROCm 10 and newer ship from a different source layout. You opt in by pinning the +version with `--version`. These versions select their device payload by exact +GPU arch, so when ROCm CLI can't resolve one for this computer (it detects only +a family of GPUs, or none), or you want to target a different GPU, also pass +`--family` with the raw `gfx` code instead of a family label: ``` rocm install sdk --version 10.1.0 --family gfx1200 --dry-run @@ -475,7 +482,8 @@ rocm install sdk --version 10.1.0 --family gfx1200 --dry-run A family label such as `--family gfx120X-all` is rejected for those versions instead of being resolved to a guess, because the ROCm 10 packages publish one payload per exact arch and there is no bucket payload to fall back to. Run -`rocm examine` to see the arch this machine reports. +`rocm examine` to see the arch this machine reports. If `--family` is needed and +missing, the error names the command to re-run. For ROCm 10, `install sdk` asks `uv` to resolve Torch, torchvision, and torchaudio from their published dependency metadata, then validates that every @@ -579,11 +587,15 @@ both `--keep-data` and `--keep-config`; see [Logs and cleanup](#logs-and-cleanup ``` rocm engines list rocm engines install [--runtime-id KEY] [--python-version X.Y] [--reinstall] + [--yes] rocm engines shell [--runtime-id KEY | --env-id ID] [--shell PATH] ``` Supported engines: `lemonade`, `vllm`. +`--yes` approves required system-package installs, such as OpenMPI for vLLM, +without asking. + ### Will a model run here? Ask before downloading anything: @@ -634,10 +646,22 @@ rocm serve [--engine lemonade|vllm] [--host HOST] [--port PORT] [--verbose] [--foreground | --managed] [--no-smoke-test] - [--allow-public-bind] + [--allow-public-bind] [--require-api-key] [--api-key KEY] [--temperature TEMP] [--top-p PROB] [--max-tokens N] + [--gpu-memory-utilization FRACTION] [--tool-call-parser NAME] ``` +`--gpu-memory-utilization` (a fraction above 0.0 and up to 1.0 of each GPU's +total VRAM) and `--tool-call-parser` (for example `hermes`, `llama3_json`, or +`mistral`, which also turns on automatic tool choice) apply to vLLM only. Lower +the memory fraction when a small model shouldn't reserve most of a large card. + +`--require-api-key` makes a loopback endpoint require an API key too, which +matters when something else republishes the port. A public bind generates a key +automatically unless you supply one. Set `ROCM_SERVE_API_KEY` rather than +passing `--api-key`, so the secret stays out of your shell history and the +process table. + `--temperature` (>= 0.0), `--top-p` (0.0-1.0), and `--max-tokens` (> 0) set server-wide sampling defaults for the launched engine. They apply only to `vllm` and `lemonade`; other engines reject them. For vLLM they are folded diff --git a/apps/rocm/src/main.rs b/apps/rocm/src/main.rs index 810b706c0..5311374a9 100644 --- a/apps/rocm/src/main.rs +++ b/apps/rocm/src/main.rs @@ -203,15 +203,16 @@ enum Command { /// architecture check. #[arg(long, conflicts_with = "distro")] report: bool, - /// Also offer the prefilled issue form, so the report can be filed. + /// Also offer a prefilled mail, so the report can be sent. /// - /// Still sends nothing. This opens the form with the same content - /// `--report` printed, already filled in; it reaches the tracker only - /// when you submit it yourself. On a machine with no desktop, or one - /// reached over SSH, the link is printed instead of opened. + /// Still sends nothing. This opens a mail to ROCmCLI@amd.com with the + /// same content `--report` printed, already filled in; it leaves your + /// machine only when you send it yourself. On a machine with no + /// desktop, or one reached over SSH, the link is printed instead of + /// opened. /// /// Requires `--report`, so the content is always shown before the - /// form is offered. Not combinable with `--json`, which is for + /// mail is offered. Not combinable with `--json`, which is for /// scripts, and a script is not a person who can read a form. #[arg(long, requires = "report", conflicts_with = "json")] send: bool, @@ -317,11 +318,14 @@ enum Command { #[arg(long)] allow_mutation: bool, }, - /// [preview] Send a one-shot chat prompt to a configured assistant provider. + /// [preview] Chat with a configured assistant provider. /// - /// Reads the prompt from --prompt or, if omitted, from standard input. Use - /// `rocm config enable-provider` and `rocm config set-provider-key` first to - /// configure a remote provider; the `local` provider talks to a running server. + /// In a terminal with no --prompt, opens the interactive chat (switch + /// providers there with /provider; --provider, --model, and --tools don't + /// apply). With --prompt, or a prompt piped on standard input, sends that one + /// prompt and prints the reply. Use `rocm config enable-provider` and + /// `rocm config set-provider-key` first to configure a remote provider; the + /// `local` provider talks to a running server. #[command(after_help = "EXAMPLES:\n \ rocm chat --prompt \"Explain ROCm in one sentence\"\n \ rocm chat --provider local --model qwen --prompt \"Hello\"\n \ @@ -333,7 +337,8 @@ echo \"Summarize this\" | rocm chat --provider anthropic")] /// Model name to request from the provider, such as qwen. #[arg(long)] model: Option, - /// Prompt to send. If omitted, rocm-cli reads from standard input when possible. + /// Prompt to send. If omitted, a terminal opens the interactive chat and + /// piped input is read from standard input. #[arg(long)] prompt: Option, /// Sampling temperature to send with the request (>= 0.0). Higher is more random. diff --git a/apps/rocm/src/storage.rs b/apps/rocm/src/storage.rs index 078f8672f..b82b56c05 100644 --- a/apps/rocm/src/storage.rs +++ b/apps/rocm/src/storage.rs @@ -509,7 +509,7 @@ pub(crate) fn build_report(paths: &AppPaths, config: &RocmCliConfig) -> Result String { let _ = writeln!(output); let _ = writeln!( output, - "Shared with other tools (never removed by ROCm CLI):" + "Shared with other tools or yours to keep (no `rocm storage` command removes them):" ); for usage in &report.shared_with_other_tools { let _ = writeln!( @@ -1456,7 +1456,7 @@ mod tests { let rendered = render_report(&report); assert!(rendered.contains("in-use version=7.12.0")); assert!(rendered.contains("status: in use")); - assert!(rendered.contains("Shared with other tools (never removed by ROCm CLI):")); + assert!(rendered.contains("Shared with other tools or yours to keep")); assert!(rendered.contains("downloaded models")); let _ = std::fs::remove_dir_all(root); diff --git a/docs/vllm.md b/docs/vllm.md index 1e240ab80..cc7c598c4 100644 --- a/docs/vllm.md +++ b/docs/vllm.md @@ -63,8 +63,8 @@ runtime that cannot execute a kernel will fail at serving time. ## ROCm 10.x wheel discovery -For ROCm SDK 7.x versions, including 7.14, `rocm engines install vllm` pins a -fixed vLLM wheel and index URL. Any ROCm SDK 10.x version is different: AMD +For ROCm SDK 7.x versions, `rocm engines install vllm` pins a fixed vLLM wheel +and index URL. Any ROCm SDK 10.x version is different: AMD publishes vLLM, flash-attn, and amd-aiter there under a rotating dev-tag filename (for example `vllm-0.27.1.dev5+rocm10.0.0.gf46a9dfe2.d20260826-cp314-cp314-linux_x86_64.whl`), From 9915a90744b4a32c8b0eda94e24e36a0c9dcf8df Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Mon, 12 Oct 2026 21:56:48 -0400 Subject: [PATCH 17/18] docs: align vLLM, uninstall, and help wording with the code Revert the rocm storage output wording and its loosened test so this PR stays docs-only; the wording moves to a separate change. Fix the vLLM pin-table scope sentence, the stale --keep-data claim in MANIFEST.md, and the --send help text. Signed-off-by: pmoutsias-amd --- MANIFEST.md | 6 ++++-- apps/rocm/src/main.rs | 2 +- apps/rocm/src/storage.rs | 6 +++--- docs/vllm.md | 5 +++-- 4 files changed, 11 insertions(+), 8 deletions(-) diff --git a/MANIFEST.md b/MANIFEST.md index 0942746ab..169d379da 100644 --- a/MANIFEST.md +++ b/MANIFEST.md @@ -682,8 +682,10 @@ filesystem: a bind mount or `subPath` volume is enough to make Linux refuse the hardlink, and `uv` then silently copies. This cache holds every wheel `uv` downloads — the ROCm SDK and the torch stack included — so it is typically the largest directory rocm-cli manages, on the order of several GB per SDK version -installed. It is removed by `rocm uninstall` unless `--keep-data` is passed, and -its location can be overridden with `ROCM_CLI_UV_CACHE_DIR`. +installed. It is removed by `rocm uninstall`. On the default layout, the +configuration and data directories are both `~/.rocm`, so keeping it takes +`--keep-data` and `--keep-config` together. Its location can be overridden +with `ROCM_CLI_UV_CACHE_DIR`. ### Lemonade Embeddable Runtime diff --git a/apps/rocm/src/main.rs b/apps/rocm/src/main.rs index 5311374a9..779539a4c 100644 --- a/apps/rocm/src/main.rs +++ b/apps/rocm/src/main.rs @@ -213,7 +213,7 @@ enum Command { /// /// Requires `--report`, so the content is always shown before the /// mail is offered. Not combinable with `--json`, which is for - /// scripts, and a script is not a person who can read a form. + /// scripts, and a script is not a person who can read and send a mail. #[arg(long, requires = "report", conflicts_with = "json")] send: bool, }, diff --git a/apps/rocm/src/storage.rs b/apps/rocm/src/storage.rs index b82b56c05..078f8672f 100644 --- a/apps/rocm/src/storage.rs +++ b/apps/rocm/src/storage.rs @@ -509,7 +509,7 @@ pub(crate) fn build_report(paths: &AppPaths, config: &RocmCliConfig) -> Result String { let _ = writeln!(output); let _ = writeln!( output, - "Shared with other tools or yours to keep (no `rocm storage` command removes them):" + "Shared with other tools (never removed by ROCm CLI):" ); for usage in &report.shared_with_other_tools { let _ = writeln!( @@ -1456,7 +1456,7 @@ mod tests { let rendered = render_report(&report); assert!(rendered.contains("in-use version=7.12.0")); assert!(rendered.contains("status: in use")); - assert!(rendered.contains("Shared with other tools or yours to keep")); + assert!(rendered.contains("Shared with other tools (never removed by ROCm CLI):")); assert!(rendered.contains("downloaded models")); let _ = std::fs::remove_dir_all(root); diff --git a/docs/vllm.md b/docs/vllm.md index cc7c598c4..a5adafb15 100644 --- a/docs/vllm.md +++ b/docs/vllm.md @@ -63,8 +63,9 @@ runtime that cannot execute a kernel will fail at serving time. ## ROCm 10.x wheel discovery -For ROCm SDK 7.x versions, `rocm engines install vllm` pins a fixed vLLM wheel -and index URL. Any ROCm SDK 10.x version is different: AMD +For every ROCm SDK version outside a discovery major (today, everything but +10.x), `rocm engines install vllm` pins a fixed vLLM wheel and index URL. Any +ROCm SDK 10.x version is different: AMD publishes vLLM, flash-attn, and amd-aiter there under a rotating dev-tag filename (for example `vllm-0.27.1.dev5+rocm10.0.0.gf46a9dfe2.d20260826-cp314-cp314-linux_x86_64.whl`), From b308ca3a2d9f8452a3aca2b8dc9b8ff7bd83ac50 Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Mon, 19 Oct 2026 13:34:01 -0400 Subject: [PATCH 18/18] docs: tie the serve step to the install it follows "Then serve a model:" followed a paragraph about the activation gate, so "Then" had no clear referent. Signed-off-by: pmoutsia_amdeng --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 9085e5e61..31aab7f86 100644 --- a/README.md +++ b/README.md @@ -250,7 +250,7 @@ asks first, because the new install takes over as the active default; see for that gate, the flags that approve it without a prompt, and the ROCm 10 and newer requirements. -Then serve a model: +After the install finishes, serve a model: ``` rocm serve qwen