Skip to content

test(xtask): close the pwsh contract's edge cases from #658's review - #717

Draft
rominf wants to merge 4 commits into
mainfrom
test/workflow-pwsh-contract-edge-cases
Draft

rominf wants to merge 4 commits into
mainfrom
test/workflow-pwsh-contract-edge-cases

Conversation

@rominf

@rominf rominf commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator

Follows up juhovainio's three non-blocking notes on #658. The shell: pwsh contract in xtask/src/workflow_contract.rs fails a step whose native command can fail without the step noticing. Each note was a case where it misjudged a step: either it flagged a step whose status was read, or it never looked at a step that should be checked.

A $LASTEXITCODE check later in the step. Until now a check counted only in the statement directly after the native command, so putting a Write-Output or an assignment before the check flagged a command whose status was in fact read. A later check now counts if every statement in between leaves $LASTEXITCODE alone. The rule is deliberately narrow. Text inside single quotes is ignored, since it is literal. Outside it, the statement may not contain ( (a $(…) or (…) runs a command, even inside double quotes), { (a script block, or a hashtable whose values run), & (the call operator, &&) or an expanding @" here-string. It must also be a pipeline that starts with a plain value, or an assignment of one, or with one of a short list of output/file cmdlets (Write-Output, Write-Host, Out-File, Add-Content, …), and every later element must be one of those cmdlets. Assigning $LASTEXITCODE is not a read. Anything else counts as possibly resetting the status, including exit, a function call, a loop's closing } and another native command. Reaching the end of the body also counts as a read. GitHub's pwsh wrapper appends if ((Test-Path -LiteralPath variable:\LASTEXITCODE)) { exit $LASTEXITCODE }, so a last native command followed only by a Write-Output is no longer flagged.

Every run: block form. run_block matched only the literal run: | header. A pwsh step written as run: |-, run: > or run: >- fell through to the one-line reading, whose value is the bare indicator, so the step escaped the contract. Every block-scalar header now matches, with any chomping or indentation indicator and an optional trailing comment. A folded > block is read the way YAML folds it: two lines at the block's indent become one line of script, so they are one command. The folding was compared against PyYAML on the same fixtures. Two more ways a step escaped turned up while doing this, and they are fixed in a separate commit: shell: or run: written as the step's first key (- shell: pwsh, - run: |). run_block is shared, so the other workflow checks that use it read these forms too. No workflow in the repository uses any of them, so those checks behave as before.

Case and keywords. PowerShell variable names are case-insensitive, so $LastExitCode now counts as a check. break, continue, begin, process, end, param and trap join PWSH_KEYWORDS. pwsh's own tokenizer confirms that every entry is a keyword.

Verification:

  • Each fix has a test that fails with the fix reverted. Mutation runs on the new rule also caught removing each forbidden character, the scoped-assignment check, chained-assignment re-checking ($a = $b = git …), the end-of-body read, and both the single- and double-quote handling. One mutant survives: allowing a plain value after a pipe. pwsh rejects that at parse time ("Expressions are only allowed as the first element of a pipeline"), so no real step can take that form.
  • Every case on both sides of the later-check rule was run through pwsh 7.4. A native command exited 3, then the in-between statements ran, then the status was printed. Every accepted case still printed 3. Every rejected case lost it, except Write-Output 'x' || git …, which is rejected conservatively.
  • Over all real workflows the contract finds the same 26 pwsh steps and the same three unchecked commands as before. Those are the three allow-listed steps, so nothing is newly flagged and no allow-list entry goes stale.

Out of scope, and still not seen by the contract: a step that inherits pwsh from defaults: run: shell: pwsh (no workflow sets one); a native command whose output is assigned ($out = cargo metadata), which is not classed as native; and the literal-only run: | matching in multiline_run_blocks and preflight_step_shape, which are separate from this contract.

This is test-only (xtask contract tests) and changes nothing a CLI user can observe, so it needs no Gherkin scenario.

  • If this PR fixes a bug, searched tests/e2e-cucumber/expectations.toml for the fixed ticket ID and removed/narrowed any now-stale xfail rows. (No ticket and no xfail rows involved.)
  • If this PR adds a new subcommand or subsystem, its domain implementation lives in its own file per docs/architecture.md. (Not applicable.)
  • Every new or changed user-facing message was read against the code path that runs after it. (The only changed message is the contract's own failure text. It now says to check $LASTEXITCODE before anything that could run another program or reset it, which is the rule the tests exercise.)

PowerShell variable names are case-insensitive, so a check spelled
`$LastExitCode` reads the same status as `$LASTEXITCODE`; the pwsh
contract only matched the upper-case spelling and would flag the native
command before it.

`break`, `continue`, `begin`, `process`, `end`, `param` and `trap`
are pwsh keywords, not programs, but a statement opening with one read as
a native command. They join PWSH_KEYWORDS, all of whose entries pwsh's
tokenizer reports as keywords.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
`run_block` only recognised the literal `run: |` header, so a pwsh step
written with `run: |-`, `run: >` or `run: >-` fell through to the
one-line reading, whose value is the bare indicator, and escaped the
contract whatever its script did.

Every block-scalar header now counts, with any chomping or indentation
indicator and an optional trailing comment. A folded `>` block is read
as YAML folds it before the statements are split: two lines at the
block's indent are one line of script, so they are one command, not a
native command followed by an unchecked one.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
The pwsh contract matched `shell: pwsh` and `run: |` only as sibling
keys. Written as the step's first key, `- shell: pwsh` was never found
as a pwsh step, and `- run: |` read as no script at all; both took the
step out of the contract. A quoted `shell: 'pwsh'` is matched too.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
…empt

The pwsh contract accepted an exit-code check only in the statement right
after a native command, so a `Write-Output` or a plain assignment before
the check flagged a command whose status was in fact read.

A check further on now counts when every statement between leaves
`$LASTEXITCODE` alone. That is defined narrowly: outside single quotes,
no `(`, `{`, `&` or expanding `@"` here-string, and a pipeline of a
plain value or assignment of one, followed only by cmdlets that write
output or files. An assignment to `$LASTEXITCODE` is not a read. Running
off the end of the body is one, since GitHub's wrapper exits with
`$LASTEXITCODE`, so a last native command followed only by a
`Write-Output` is no longer flagged either. Every case in the tests,
on both sides of the rule, was run through pwsh first.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant