[vitek-karas]

Motivation

The CI Outer-Loop Failure Scanner has been failing repeatedly rather than completing its scan. The investigation found 49 consecutive failures after its last successful run on October 9, with two distinct errors:

• The October 9 failure shows the Responses API rejecting tool history after a file edit: 400 Invalid 'input[...].id': 'ctc_call_...'. Expected an ID that begins with 'fc'. This matches github/gh-aw#68667. The upgraded AWF includes the replayed apply_patch function-call ID normalization from github/gh-aw-firewall#8944.
• The November 2 failure fails at the first inference request: 400 Model "gpt-6.1-sol" has no AI credits pricing and no default pricing is configured. gh-aw v1.0.1 compiles the model's catalog pricing into the AWF configuration, so startup no longer depends on runtime discovery supplying that pricing.

The local read-only reproduction with the upgraded stack completed real scan startup and model/tool continuations after file edits, exercising both previously failing paths. This supports the upgrade as a fix for these errors, but does not establish that production CI authentication is valid or that the full workflow will complete.

Summary

Upgrade gh-aw from v0.89.21 to v1.0.1 and regenerate all six agentic workflows. The resulting workflows use AWF v0.29.0 and Copilot CLI 1.0.103, including explicit compiled pricing for gpt-6.1-sol.

Also update the companion setup action and versioned authoring references, and give the optional PAT-selection seed an empty default so omitted inputs compile to valid Actions YAML without changing random selection. The README points to the compiler version recorded in lock-file metadata instead of duplicating a version number.

Why the breaking-change workflow changed

Most of this workflow's source diff is moving and reindenting its existing outcome validator, with a diagnostic-name correction required by the upgraded collector:

• Required migration: the new compiler rejects agent post-steps that inspect safe-output files. Move the validator to safe-outputs.steps, where it still runs before publication. Keep the successful-agent requirement and the existing checks for the source PR, exact raw comment/draft agreement, complete drafts, explicit no-op outcomes, and dry-run behavior.
• Cross-job artifact handling: the validator now runs in a separate job, so download the breaking-change drafts there and read the raw comment intents from the downloaded /tmp/gh-aw/safeoutputs.jsonl. Draft uploading remains in the agent job. Pattern-based downloading permits legitimate no-op runs with no draft artifact.
• Canonical diagnostics: reject report_missing_tool, report_missing_data, and report_incomplete, including when a diagnostic accompanies an otherwise-valid comment. The pinned collector normalizes legacy names before writing the file consumed by the validator, so aliases are not needed here. Update the prompt to use the canonical tool name and commit compound regression fixtures for both source and compiled validators.
• Compiler diagnostics and version alignment: add the explicit acknowledgement for the existing trigger and a per-PR concurrency discriminator, update the artifact actions, and explicitly name pwsh in the helper instructions to avoid a command-permission warning. These smaller edits are not themselves fixes for the scanner failure.

Validation

• gh aw compile --schedule-seed dotnet/runtime --validate --validate-images — passed for all six workflows, with zero compiler warnings.
• gh aw lint .github/workflows/*.lock.yml .github/workflows/validate-pat-pool.yml — actionlint passed for all seven files.
• Repeated gh aw compile --schedule-seed dotnet/runtime — passed and produced byte-identical generated output.
• node --test .github/workflows/evals/breaking-change-doc.test.mjs — passed 43 checks: 21 fixture cases against each actual inline validator plus source/compiled-script equality. Running the same suite against the original published scripts produced exactly four expected failures: a valid comment plus either canonical missing-tool/data diagnostic, in both source and compiled scripts; the other 39 checks passed. The fixtures are committed.
• npm test --prefix .github/workflows/evals — passed all 71 Node tests, including existing scanner tests and the new documentation-validator fixtures.
• bash <session-artifacts>/repro-v1.0.1/controller.sh — bounded read-only local scanner reproduction passed using AWF v0.29.0, Copilot CLI 1.0.103, and gpt-6.1-sol. It fetched 25 completed builds for pipeline 154, selected source build 1658360 and follow-up 1659071, read an 835-record timeline and a failed task's 1,762-line log, and completed model/tool continuations after file edits. No GitHub writes, workflow dispatches, or repository builds were performed. The harness is not committed.
• git diff --check — passed.

The local reproduction verifies startup and initial scan work using local authentication. It does not validate production CI credentials, the full scan, MCP integration, or publication. The committed documentation tests exercise the validators, not the live agent or collector. No .NET product builds or tests were run for these workflow-only changes.

Note

This PR description and the changes were prepared with GitHub Copilot.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
14 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[PureWeen]

Holistic Review

Motivation: Updating the compiler and regenerating the workflows is reasonable.

Approach: Moving outcome validation into safe-outputs.steps is appropriate, but the validator also needs to follow the upgraded diagnostic output contract.

Summary: ⚠️ Needs Changes. One publication-gate regression is described inline; this is a code-correctness recommendation, not an approval or request-changes vote.

Detailed Findings

⚠️ Publication validation — canonical diagnostic names are not rejected

The breaking-change documentation validator still recognizes the old diagnostic names. Please update the filter and cover compound comment-plus-diagnostic outputs before merging.

Note

This review was generated by GitHub Copilot.

[vitek-karas]

Reviewed published head d8d68a2e423. No additional findings beyond the existing unresolved diagnostic-filter issue were identified.

The existing issue is independently confirmed by 3/3 reviewers plus the repository specialist: an otherwise-valid add_comment combined with report_missing_tool or report_missing_data is not rejected by the published validator. The existing thread already describes the required correction; no duplicate inline comment has been added.

Coverage and prior feedback: The requested compound cases have been exercised in ad hoc local validation, but their fixtures are not committed in this PR. The existing review thread remains unresolved. Renewed human review should follow an updated PR containing the correction and addressing the fixture request.

Methodology: 3 independent reviewers with adversarial consensus + repo domain specialist.

Scope: code review only. CI/check status and merge readiness were not evaluated; use the repository's CI-analysis workflow for that assessment.

Note

This review was generated with GitHub Copilot.