Review-workflow templates pin a model ID, so every model release is a hand edit in every target — use the family alias and let the action pin carry the bump #67

Activity

1. j4th

   added

   enhancementNew feature or requestNew feature or request

   source:you-are-hearEvidence from the you-are-hear run (GitHub axis)Evidence from the you-are-hear run (GitHub axis)

   on Sep 23, 2026
2. j4th commented on Sep 24, 2026

   

   j4th

   on Sep 24, 2026

   OwnerAuthor

   More actions

   One more item for the proposal, found while applying it in you-are-hear (PR #85 there).

   templates/claude.yml passes no --effort: its header says "the workhorse model at its default effort", and claude_args carries only --model and --fallback-model (template lines 9 and 86–87). Under an ID that was stable. Under an alias it isn't, because the default is per model. code.claude.com/docs/en/model-config § Adjust effort level says: "The model's default effort: high on every model that supports effort, except that Opus 5.5 defaults to medium…" So once a target's action pin reaches a Claude Code whose opus means Opus 5.5 (v1.0.232, Claude Code 2.1.280, onward), the mention bot drops from high to medium with no change in the diff.

   The fix is to name it: --effort high in the template's claude_args, with the header saying why. That keeps what the bot runs at today on Opus 5, and matches orchestration.md's own rule that an omitted effort is not neutral. claude-review.yml already passes --effort on every branch, so it isn't affected. you-are-hear's version is in PR #85, with the orchestration.md § Generation notes line from proposal item 3.
3. j4th commented on Sep 28, 2026

   

   j4th

   on Sep 28, 2026

   OwnerAuthor

   More actions

   A separate defect in the same template: the "Assert the review posted" step (constraint 4), from echosphere, 2026-09-28

   Posting here because this is the open issue on .claude/skills/blueprint/references/templates/claude-review.yml. The assertion came with harvest 3 (#52, closed). Echosphere's copy of the workflow had never taken it, and it found out on j4th/echosphere#55: the ready flip's review was skipped by the action's workflow validation (the PR edited the workflow), posted nothing, and passed green. Adding the step there in j4th/echosphere#56 turned up three defects in the template's version, each on main as it stands.

   1. The count breaks on any PR with more than one page of comments, and a posted review reads as missing. gh api --paginate --jq '[…] | length' runs the jq program once per page, so it prints one number per page. Reproduced on a real PR whose review did post, with per_page=1 standing in for the default 30:

   n=$'0\n1\n1\n0'
   [: 0\n1\n1\n0: integer expression expected   → takes the "not posted" branch: false red, false notice
   

   Past 30 comments this happens at the default page size. The fix is to slurp across pages and count once:

   comments="$(gh api "repos/$REPO/issues/$PR_NUMBER/comments?per_page=100" --paginate)" || { echo "::error::could not list comments"; exit 1; }
   n="$(jq -s --arg start "$STARTED_AT" '[.[][] | select(…)] | length' <<<"$comments")"

   2. With track_progress on, which the template sets, the tracker comment satisfies the assertion by itself. The action posts its tracker ("Claude finished @…'s task … View job") from the review app on a run that starts a session; each of the three echosphere reviews checked has one. So select(.user.login == REVIEW_LOGIN) | select(.created_at >= STARTED_AT) | length >= 1 passes even when the session never ran gh pr comment, which is the exact failure family the step exists for. Excluding the tracker is not the fix: a summary can land inside it (gh pr comment --edit-last edits the app's latest comment), as it did on one of echosphere's reviews. What works is matching the summary's own marker. The template's prompt makes the summary's first line a verdict, so the check is (.body | test("APPROVE|REQUEST CHANGES|NEEDS DISCUSSION")), and updated_at rather than created_at, since an edit in place is a landing too.

   3. if: always() together with concurrency: cancel-in-progress: true (both in the template) reports a superseded run as a failure. always() runs the step even on cancellation, so a run cancelled by a newer one posts "Auto-review did not post a summary". if: ${{ !cancelled() }} keeps the verdict for success and failure only.

   Also worth taking: a notice that says why. With steps.review.outputs.execution_file empty, no model session started. That is the workflow-validation skip, which the template's constraint 1 describes, or an auth or setup failure. Re-triggering with the label cannot fix a PR that changes the workflow, so echosphere's notice names that case separately.

   Echosphere's step, checked against real comment data from four PRs before landing, is in j4th/echosphere#56 (001bada). The three reviewed PRs pass; the skipped one, a start time after the review, and an unreadable comment list each fail.
4. j4th commented on Sep 29, 2026

   

   j4th

   on Sep 29, 2026

   OwnerAuthor

   More actions

   From crease, 2026-09-28: the skip the assert step should key on, and which release moves which alias

   Read with gh api …/contents/…?ref=<tag> at crease's pin, be7b93b (v1.0.183), and at v1.0.236.

   1. The workflow-validation skip has its own output. Key the why-notice on it.

   • What happens. In src/github/token.ts (identical at v1.0.183 and v1.0.236), a token exchange that fails with "workflow validation failed" is turned into a skip. It logs "Action skipped due to workflow validation error. This is expected when adding Claude Code workflows to new repositories or on PRs with workflow changes. …". src/entrypoints/run.ts then sets the output skipped_due_to_workflow_validation_mismatch=true and the step ends green.
   • What that means for the assert step. Its "no session started" notice can read that output directly, instead of inferring the skip from an empty execution_file. A target's harvest PR, which edits claude-review.yml, is exactly the PR that takes this path.
   • A newer option. v1.0.236 also adds a conclusion output ("Execution status of Claude Code ('success' or 'failure')"). v1.0.183 has none.

   2. Which action release moves which alias. Each release's claudeCodeVersion in src/entrypoints/run.ts:

   action	Claude Code	opus	sonnet
   v1.0.183	2.1.220	Opus 5	Sonnet 5
   v1.0.231	2.1.278	Opus 5	Sonnet 5
   v1.0.232	2.1.280	Opus 5.5	Sonnet 5
   v1.0.235	2.1.283	Opus 5.5	Sonnet 5
   v1.0.236	2.1.284	Opus 5.5	Sonnet 5.5

   • Two branches' --effort flags become load-bearing at v1.0.236.
     • Fast branch: in Claude Code, Sonnet 5.5 defaults to medium, so that branch's explicit --effort matters.
     • Deep branch: 2.1.284 stops ultracode forcing xhigh, so the deep branch's --effort xhigh is no longer redundant.
   • Cooldown timing. Under a 7-day cooldown, v1.0.232 becomes eligible after 2026-09-30T16:51Z and v1.0.236 after 2026-10-05T18:05Z.
   • The current Dependabot group PR. crease's group PR currently resolves to v1.0.231, which is still Opus 5.

   3. crease is taking the echosphere step (001bada) with the three fixes in the comment above. crease's summary already opens with APPROVE / APPROVE WITH NITS / REQUEST CHANGES / NEEDS DISCUSSION, so the marker regex carries over unchanged.
5. j4th commented on Sep 29, 2026

   

   j4th

   on Sep 29, 2026

   OwnerAuthor

   More actions

   Applied to crease, 2026-09-28: crease-data/crease#281 (open, not yet merged)

   A correction to crease's comment above (item 1). skipped_due_to_workflow_validation_mismatch is not a declared output of the action. It is set on the internal steps.run and read only by the action's own token-revoke step. The action's outputs: block lists:

   • at v1.0.183 (be7b93b): execution_file, branch_name, github_token, structured_output, session_id;
   • at v1.0.231 and v1.0.236: the same, plus conclusion.

   A caller cannot key on the skip value. crease's assert step keys the skip notice on an empty execution_file plus git diff --quiet "$BASE_SHA" HEAD -- .github/workflows/claude-review.yml, the file the action validates. An unchanged file names an auth or setup failure instead (a7d9cfe).

   Applied as proposed otherwise: echosphere's 001bada with the three fixes — --paginate then jq -s, the verdict marker on updated_at >= start, and !cancelled(). The step body is fixture-tested by extracting it from the YAML and running it on recorded crease comment data plus synthetic cases (.claude/workflows/tests/review-assert-fixture.sh, red first in 506878a; 11 cases + 3 structural at the PR head, the non-JSON answer added at review in 4e807ab).

   One more measured fact for the template. With track_progress, the action ends its tracker comment with the session's final text. On crease-data/crease#274 the only bot comment is that tracker, and it carries the whole review, verdict included. The verdict regex counts it as landed, which is right: the review is readable on the PR. crease-data/crease#276 is the live negative: a PR editing claude-review.yml, a green run whose log says "Exiting due to workflow validation skip", and zero bot comments.

   Effort. The deep branch's --effort xhigh is load-bearing from Claude Code 2.1.284 (ultracode no longer implies it). The model-config page now agrees with the changelog that the ultracode toggle "leaves the effort level unchanged".
6. j4th commented on Sep 30, 2026

   

   j4th

   on Sep 30, 2026

   Last edited by j4th

   OwnerAuthor

   More actions

   Applied to crease — the harvest-5 final check (2026-09-29)

   More template changes from three targets' runs since this issue was filed, all applied in crease-data/crease#281 (open) to crease's filled workflows and to its copy of the kit's blueprint/references/templates/ pair. Two measurements confirm what the proposal assumed.

   Template changes

   1. Auto-review one-way-door markdown; skip only plain docs (j4th/echosphere#53). The template's paths-ignore: '**/*.md' skips every markdown-only PR, including one that changes only a rule, skill, command or agent under .claude/, or an ADR. pr-review.md calls those one-way doors, "not automatically light". echosphere had to substitute a direct reviewer workflow. paths-ignore cannot re-include anything: GitHub takes the ! prefix only in paths, and there "A matching positive pattern after a negative match will include the path again" (docs.github.com, workflow syntax § on.<push|pull_request>.<paths|paths-ignore>, read 2026-09-29). So:

       paths:
         - '**'
         - '!docs/cbk/**'
         - '!**/*.md'
         - '!.gitignore'
         - '!.gitattributes'
         - '!.editorconfig'
         - '!.github/dependabot.yml'
         # [add the project's other prose-only paths, each with a leading `!`, above the re-includes]
         - '.claude/**'
         - 'docs/adr/**'

   The prompt's "What to skip" now says markdown under .claude/ or docs/adr/ is reviewed like code. The order is the semantics, so crease pins it with a fixture: review-trigger-fixture.py evaluates a workflow's filter the way GitHub does, following the docs' cheat sheet (**/README.md matches README.md), over twelve PR shapes. It was red on the old filter. A re-include moved above !**/*.md fails it. It refuses to pass when it finds no workflow. It runs in the verification block's kit sub-block, so the kit guards its own template after the mint. Cascade artifacts under docs/cbk/, the rule's other one-way door, stay skipped as the bot's hard skip always had them; the comment says so, and a human reviews those. Cost: more review runs on rule and ADR PRs, which spends subscription quota and no money. Crease commits e28f987 (red), a923d44, 00f4466, 9b0ea75, 272fe7b.

   2. The review reports only commands it actually ran (j4th/you-are-hear#87). On j4th/you-are-hear#82 the bot's summary said mise run check was refused by its Bash permissions, but it never issued the command: the run's permission_denials hold 8 entries, and none is check. The summary instructions now say: "Report only commands you actually ran: a command you never issued is never reported as run, passed or refused." Crease commit 1a75bc9.

   3. A gate added to the prompt needs a matching --allowedTools entry (j4th/you-are-hear#87). That prompt offered mise run hooks and mise run levels as gates, and the allowlist permitted neither. It also offered cd packages/yah_core && dart test against Bash(dart test:*). Claude Code splits a compound command, and "A rule must match each subcommand independently" (code.claude.com/docs/en/permissions § Compound commands). A YAML comment above claude_args, outside the block scalar so it stays out of the prompt, now says the two lists are edited together. The same page says a cd inside the working directory counts as read-only and passes on its own, so a compound gate can pass. The comment still prefers one command per gate and asks for a compound one to be confirmed on a real run. Crease commits 9532676, c295ce3.

   4. How to size the degrade clause's N (#58 addendum item 11's leftover; the reopened #58's residue R12). --max-turns is spent per tool call: echosphere measured five Read calls in one response costing five turns (Claude Code 2.1.274, claude -p --output-format stream-json). At one Read a file, keep about a third of the cap for gh calls, inline comments and the summary: N = 40 against --max-turns 60, and N and the cap move together. The measurement is from 2.1.274. The template tells a target to re-measure on the release its pinned action installs, and crease's fill notes that its pin installs 2.1.220, where 40 is at worst conservative. crease's filled workflow never had the clause at all, and now carries it at 40. Crease commits 713cc5c, 24c4f3b.

   5. The action bump re-checks the turn cap and N, not only effort (j4th/echosphere#50). This issue's proposal names the bot's effort settings as what the action-bump PR re-checks. The same PR is where --max-turns and the degrade N are re-checked, because a new Claude Code can spend turns differently. It is in crease's orchestration.md § Generation notes. Crease commit fb34f05.

   6. The bot's Claude Code facts come from its pinned release (j4th/you-are-hear#90). The action SHA pins the bot's Claude Code as well as its model. That session reported the workflow-size guideline as "medium — under 15 agents", the value before Claude Code 2.1.271, while the operator ran 2.1.282 ("under 10"). The cause is hedged there ("most likely"). tooling.md § Automated review now says a harness fact the bot asserts is checked against the operator's version before acting on it. Crease commit 42ecca0.

   7. The template's summary reads CI; it no longer says CI is unreadable. The template grants checks: read in both its permissions block and additional_permissions, but its summary instructions still said "CI status is usually NOT readable from your token … otherwise say nothing about CI". Now: read CI with gh pr checks and reference a failed check; if it fails "Resource not accessible by integration", the grant is missing, so say so in one line and never as a finding. Crease commit b99956f.

   8. Here-strings, shell: bash, a named runner image. The templates' printf … | grep -q label checks, their missing defaults: run: shell: bash, and runs-on: ubuntu-latest are filed as #70, with measurements, since the same fixes cover the CI templates.

   9. tooling.md's rule says "changes", not "introduces" (j4th/echosphere#56, 69d3df8). The template's constraint 1 already says the action exits green without running on "any PR that adds or changes" the workflow. But tooling.md § Automated review still reads "The workflow cannot review the PR that introduces it; verify on the next one." A later PR that edits the workflow gets no review either. crease-data/crease#281 is one such PR, where the assert step's red notice (confirmation 2 below) is the proof. crease's rule reads "The workflow cannot review any PR that changes it — claude-code-action runs only a workflow identical to the default branch's copy and otherwise skips, green and silent … so such a PR gets no review, and the workflow's last step, Assert the review posted, goes red on it with the reason; verify on the next PR". Crease commit a7d9cfe.

   Two confirmations

   • init.model names the resolved model, not the alias. This answers the proposal's open question. With --model opus, the SDK options line logs "model": "opus" and the init message logs "model": "claude-opus-5" (action v1.0.183 / Claude Code 2.1.220; crease review run 36349822530 on crease-data/crease#275). So the record-model step reads the resolved id from execution_file. The job-summary half is still unverified.
   • The "Assert the review posted" step's negative path, live. On crease-data/crease#281, a PR that edits claude-review.yml, the action's validation skip left no review. The step went red (run 36522803360) and posted: "Auto-review: no summary posted. The review was skipped by claude-code-action's workflow validation … Re-triggering cannot fix that". That is the notice it keys on an empty execution_file plus a diff of the workflow.

   Template file	Items	crease commits
   blueprint/references/templates/claude-review.yml	1, 2, 3, 4, 7	a923d44, 272fe7b, 1a75bc9, 9532676, c295ce3, 713cc5c, 24c4f3b, b99956f
   .claude/workflows/tests/review-trigger-fixture.py (new), § Verification's kit sub-block	1	e28f987, 00f4466, 9b0ea75
   blueprint/references/templates/tooling.md	1 (the review-bot paragraph names the paths shape)	a923d44
   .claude/rules/orchestration.md, .claude/rules/tooling.md	5, 6	fb34f05, 42ecca0
7. j4th

   added a commit that references this issue on Oct 2, 2026

   feat: harvest 5 — v1.0.0: guards that resolve paths, the block as a g…

   ...

   7dbd6b2
8. j4th

   closed this as completedin #75on Oct 2, 2026