Skip to content

fix(review): stop tool-free cross-model reviews when Claude invokes advisor #158

Closed

Description

@BaseInfinity

Goal

Reject forbidden tool use immediately in the tool-free Claude/Fable review leg and preserve an honest infrastructure-failure result.

Problem

The configured cross-model reviewer was launched with tool-disabling flags but emitted a built-in advisor tool-use event. The runner did not recognize the real nested stream shape, consumed the wall timeout, and could allow later clean output to overwrite the rejection.

Preserved checkpoint (2026-08-22)

  • Base commit: b59b3471d1f0072e15182c432cdceee1092cfd3b
  • Preserved candidate tree object: f43b24de8058b9a97e8ac009b7718ccb66361b3b
  • Candidate diff: 3 files, 52 insertions, 3 deletions.
  • Focused proof and canonical 11/11 proof were green before the final gate.
  • The final gate returned NOT CERTIFIED.
  • The former /private/tmp/codex-sdlc-issue158 linked worktree is no longer valid; its Git metadata is prunable. Do not treat that path as resumable.

Current blocking findings

  1. Detect real nested Claude message.content[] tool-use blocks, including server_tool_use for advisor.
  2. Once a forbidden event is observed, a later parsed clean result must not overwrite the rejection.
  3. Rejection classification must take precedence over timeout classification.

Acceptance criteria

  • Focused RED fixtures cover nested tool use, result overwrite, and rejection-before-timeout precedence.
  • Any forbidden tool-use event terminates the full reviewer process tree immediately.
  • The attempt is classified as invalid reviewer infrastructure, never CLEAN/CERTIFIED.
  • The structured failure summary and bound log are retained.
  • An unchanged candidate's ordinary proof is not invalidated solely by reviewer infrastructure failure.
  • The configured subscription fallback remains available; no downgrade or fabricated receipt is introduced.
  • Focused tests and canonical proof pass before merge.

Next slice

Create a fresh linked worktree from current origin/main, deliberately reconstruct/rebase the preserved candidate, add the three RED regressions above, implement one bounded correction, and run one completion gate. After integration, resume #109 v4.

Activity

  1. added
    bugSomething isn't working
    priority: P0Stop-the-line critical path; work before lower-priority roadmap items
    on Aug 17, 2026
  2. BaseInfinity commented on Aug 17, 2026

    @BaseInfinity
    OwnerAuthor

    Implementation status: RED/GREEN complete, adapter integration 115/115, canonical proof 11/11. The live completion gate did not exercise the advisor rejection because Fable first returned a quota-exhausted stream-json envelope with API status 429 but process exit 0. The runner mislabeled that as INVALID_VERDICT and did not enter the approved Opus fallback. #158 remains frozen and unchanged while the pre-existing #143 fallback contract is repaired.

  3. BaseInfinity commented on Aug 18, 2026

    @BaseInfinity
    OwnerAuthor

    Tracker checkpoint after the post-#159 completion gate (2026-08-16 23:29 PDT): candidate tree f43b24de8058b9a97e8ac009b7718ccb66361b3b remains staged and unchanged. Focused proof and the canonical 11/11 proof were green. The joint Sol High + Fable High gate returned NOT CERTIFIED. Blocking P1s: the detector inspects top-level fields instead of real nested message.content[] tool-use blocks, and a rejected event can still be overwritten by a parsed clean result. The gate also identified rejection-versus-timeout precedence as a bounded P2. Next action is one corrective TDD slice in /private/tmp/codex-sdlc-issue158, then focused proof, one fresh canonical proof, and one bounded completion gate. No commit, push, or PR exists for this candidate.

  4. BaseInfinity commented on Aug 22, 2026

    @BaseInfinity
    OwnerAuthor

    Cold-session correction (2026-08-22): the candidate tree object f43b24de8058b9a97e8ac009b7718ccb66361b3b and its three-file diff are still recoverable, but /private/tmp/codex-sdlc-issue158 is no longer a valid linked worktree and its Git metadata is prunable.

    The issue body now records the recovery boundary and exact next TDD slice. Do not resume by editing that old path. Reconstruct/rebase deliberately in a fresh linked worktree, preserving the prior green proof and NOT CERTIFIED findings as evidence rather than claiming the candidate is ready.

  5. BaseInfinity commented on Aug 22, 2026

    @BaseInfinity
    OwnerAuthor

    Fresh-worktree completion checkpoint (2026-08-22):

    • Worktree: /private/tmp/codex-sdlc-issue158-v2
    • Branch: fix/issue-158-reject-review-tools-v2
    • Base: 500384289ffc8ce3bf73fe78200eee1aa967ad5b (origin/main)
    • Frozen candidate tree: 24b71b8df61fddba8142725a5f052d5b4166013a
    • Patch: 3 files, +132/-6
    • RED/GREEN fixtures now cover the captured real assistant.message.content[] -> server_tool_use(advisor) shape, a buffered clean-result overwrite attempt, and rejection-before-timeout precedence.
    • Focused supervisor proof passed; adapter proof passed 115/115.
    • Canonical stamped proof passed 11/11.

    The live Sol High + Fable High completion gate then failed closed exactly as intended. Claude Code 2.1.221 launched Fable High with --tools "", --disallowedTools advisor, safe mode, no slash commands, no MCP servers, and strict MCP config. Fable still emitted a nested server_tool_use block for advisor. The supervisor terminated it with exit 143 in 77.8s, recorded rejected_event.reason = forbidden reviewer tool use: advisor, retained terminal_state = INVALID_VERDICT, did not time out, did not retry, did not fall back, and wrote no certifying receipt.

    This validates the candidate behavior but blocks delivery under the required Fable gate. Anthropic documentation says an empty tool set or bare-name deny removes built-in tools from availability, yet the Fable server-side advisor event still bypassed those controls. No commit, push, or PR was created. Do not waive or relabel this as quota/model unavailability.

  6. BaseInfinity commented on Aug 22, 2026

    @BaseInfinity
    OwnerAuthor

    Owner-authorized one-time bootstrap exception (2026-08-22): this exception applies only to the frozen #158 candidate because #158 repairs the same meta-review harness whose required Fable leg is currently invoking forbidden server-side advisor despite tool-denial flags. It does not waive tests, findings, or future Fable review; it does not add a reusable bypass or relabel the failed Fable run as certified. Delivery requires the existing green canonical proof plus fresh exact-candidate Sol High and Opus 4.8 replacement review evidence. The commit will identify the exception explicitly. After #158, the normal configured review policy resumes; any further exception requires a new owner decision.

  7. BaseInfinity commented on Aug 22, 2026

    @BaseInfinity
    OwnerAuthor

    Integrated — one-time owner exception consumed

    PR #165 advanced main from 500384289ffc8ce3bf73fe78200eee1aa967ad5b to the exact reviewed commit af037604a6f8d8ac2b9a3279b176aebed61aefb5. GitHub reports the PR merged and this issue closed.

    Evidence:

    • candidate tree: a94df554b156bec706f65c274a301a2736f4e9fe
    • patch SHA-256: 3771b360acd74e64ea21260d97e64f32f11f476bc74211ec4920d7ba509aed6d
    • focused availability-probe RED reproduced the unsafe fallback; focused GREEN passed
    • canonical proof: 11/11 passed
    • adapter suite: 115/115 passed
    • final Sol High review: no actionable P0/P1/P2 findings
    • required PR windows-npm check: passed before integration

    The prior Fable and Opus attempts remain invalid because each invoked forbidden server-side advisor; neither was relabeled as certified. The owner-authorized exception recorded above was consumed only for #158. Normal fail-closed review policy resumes after this merge.

    Next release handoff: return to #109, then follow the remaining 1.0.0 milestone blockers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority: P0Stop-the-line critical path; work before lower-priority roadmap items

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions