Skip to content

Require tool-free Fable to return the final review instead of deferring to repository inspection #175

Closed
#177
Closed
Require tool-free Fable to return the final review instead of deferring to repository inspection#175
#177
Labels
bugSomething isn't workingpriority: P0Stop-the-line critical path; work before lower-priority roadmap items
Milestone
1.0.0 — Bounded autonomous delivery

Description

@BaseInfinity
BaseInfinity
opened on Aug 23, 2026

Problem

The tool-free Fable lane can complete successfully at the provider/process level yet return only an interim intention to inspect the repository instead of the required review JSON.

The #174 exact-candidate review proved the remaining prompt-contract hole after #167 and #172:

  • preflight selected Fable High normally and tools remained disabled;
  • the supplied prompt included the frozen patch and exact CERTIFIED / NOT CERTIFIED contract;
  • Fable ended its only turn with prose equivalent to “I’ll review this patch… let me first see whether the base repository is available”;
  • no final JSON review or verdict was emitted;
  • the gate correctly classified INVALID_VERDICT, wrote a non-certifying partial receipt, cancelled the still-running Sol peer, and recommended no retry.

The validator is correct. The prompt must make clear that the supplied patch/context is the complete review environment and that repository inspection is unavailable and unnecessary.

Acceptance criteria

  • Initial and reconciliation Fable prompts explicitly state that all available review evidence is already supplied, no repository/tools will be available, and the reviewer must complete the review in the current response.
  • Require exactly one final JSON object matching the schema; prohibit preambles, progress narration, deferral, requests for more context, repository-inspection intent, and Markdown wrappers.
  • Keep exact uppercase CERTIFIED / NOT CERTIFIED literals and the existing contradiction rules.
  • Do not normalize or salvage interim prose; it remains non-retryable INVALID_VERDICT / invalid_execution with a non-certifying partial receipt.
  • Preserve the completed peer result only when it actually completed; do not relabel a cancelled/incomplete peer as completed.
  • Add RED/GREEN prompt assertions for both initial and reconciliation paths plus an execution regression where a tool-free interim response fails closed without retry.
  • Keep Fable High tool-free, subscription-authenticated, and candidate-bound; do not re-enable tools or broaden fallback.
  • Synchronize ROADMAP and the authoritative 1.0 milestone in this slice.
  • Require one canonical 11/11 proof and normal Sol High + Fable High review with no exception.

Evidence source

#174 candidate tree 2636c7da25fcd9a19735112f938e050981012a6c, patch SHA-256 20b901eb0a8b066467d4c75ee6d3dc96a9333155c83d71062d37a1a4a9cd7618:

  • canonical proof: 11/11 PASS;
  • review receipt: partial, certifying: false, INVALID_VERDICT, invalid_execution, non-transient, retry not recommended;
  • Fable preferred route completed one provider turn but emitted only repository-inspection deferral prose;
  • Sol was cancelled before completion, so no valid peer result existed to preserve.

Non-goals

Activity

  1. BaseInfinity
    added this to the 1.0.0 — Bounded autonomous delivery milestone on Aug 23, 2026
  2. BaseInfinity
    added
    bugSomething isn't working
    priority: P0Stop-the-line critical path; work before lower-priority roadmap items
    on Aug 23, 2026

  3. BaseInfinity commented on Aug 23, 2026

    @BaseInfinity
    BaseInfinity
    on Aug 23, 2026
    OwnerAuthor

    Start checkpoint — decomposed current-main slice

    Confidence: HIGH.

    • Worktree: /Users/stefanayala/codex-sdlc-issue175
    • Branch: fix/issue-175-final-fable-response
    • Base: origin/main at b7c8d4590a483afa0be61f1bb9ed4408d3b520cd
    • GOALS.md: absent; GitHub milestone plus ROADMAP own scope/order.
    • Dirty root checkout: out of scope and untouched.
    • Frozen Block model-policy migration completion when preserved guidance still authorizes autonomous xhigh #174 candidate: tree 2636c7da25fcd9a19735112f938e050981012a6c, canonical 11/11, non-certifying invalid-execution receipt; do not mutate, retry, or deliver it.
    • Current implementation effort: Sol High per the live maintainer decision.

    Narrow contract

    • The supplied binding, proof, patch, and peer JSON are the complete review environment for tool-free Fable.
    • Initial and reconciliation prompts must require the final schema-valid JSON in the current response and forbid preamble/progress/inspection intent/deferral/context requests/Markdown wrappers.
    • Interim prose remains strict INVALID_VERDICT / invalid_execution, non-transient and non-retryable; no normalization.
    • A cancelled Sol peer remains incomplete; preserve a peer result only if it actually completed.
    • Keep tools disabled, subscription auth, candidate binding, and exact uppercase verdict literals unchanged.
    • Synchronize ROADMAP/milestone, then one canonical 11/11 proof and normal Sol High + Fable High review. No exception.

    TDD plan

    1. Add prompt assertions for both Fable initial and reconciliation paths plus an actual interim-prose execution regression; run only the dual-review adapter test RED.
    2. Add the smallest shared Fable final-response contract to both prompt builders; run focused GREEN.
    3. Self-review, focused proof, roadmap synchronization, canonical proof, required dual review, and candidate-bound delivery.

  4. BaseInfinity commented on Aug 23, 2026

    @BaseInfinity
    BaseInfinity
    on Aug 23, 2026
    OwnerAuthor

    TDD RED — complete-evidence / immediate-final-response prompt contract

    Focused command:

    ADAPTER_TEST_ONLY=dual-review-final-response bash tests/test-adapter.sh

    Result: expected 0 pass / 1 fail.

    A direct inspection of the exported initial and reconciliation prompt builders confirmed these missing clauses:

    • initial-complete-evidence
    • reconciliation-complete-evidence
    • immediate-final-json

    The new execution fixture also mirrors the #174 terminal behavior: Fable emits only interim repository-inspection prose while Sol hangs. Existing production behavior already classifies that as non-certifying, non-transient, non-retryable invalid_execution and leaves both reviewers incomplete; the intended production change is therefore prompt-only and must not weaken the validator or retry policy.

  5. BaseInfinity commented on Aug 23, 2026

    @BaseInfinity
    BaseInfinity
    on Aug 23, 2026
    OwnerAuthor

    Focused GREEN — immediate final tool-free review contract

    The narrow production change adds two Fable/cross-model-only clauses to both initial and reconciliation prompts:

    • the supplied binding, proof summary, patch, and (for reconciliation) peer JSON are the complete available review evidence; repository access/tools are unavailable, and the reviewer must not defer, request context, or announce future inspection;
    • output exactly one final JSON object now, with no preamble, progress narration, inspection intent, request, deferral, Markdown wrapper, or surrounding prose.

    Sol's repository-backed prompt is unchanged. Tool disablement, subscription auth, exact uppercase verdicts, strict parsing/contradiction checks, retry classification, candidate binding, and fallback rules are unchanged.

    Focused proof:

    • prompt/behavior RED: 0/1;
    • prompt/behavior GREEN: 1/1;
    • full adapter: 115/115;
    • packaging: 39/39;
    • roadmap: 15/15;
    • syntax plus git diff --check: GREEN.

    The new interim-prose fixture reproduces the #174 shape and proves a non-certifying invalid_execution receipt, no retry recommendation, no salvaged review, and cancelled/incomplete Sol.

    ROADMAP matches the authoritative milestone: 10 open / 15 closed, with #175 -> #174 -> #86 -> #109 first. #79 remains the exact-candidate real Windows Codex Desktop + CLI gate before #88 public artifact verification.

    Next: freeze the tree, run one canonical stamped 11/11 proof, then normal Sol High + Fable High review with no exception.

  6. BaseInfinity commented on Aug 23, 2026

    @BaseInfinity
    BaseInfinity
    on Aug 23, 2026
    OwnerAuthor

    Canonical candidate proof — 11/11

    Frozen candidate:

    • base/head: b7c8d4590a483afa0be61f1bb9ed4408d3b520cd
    • staged tree: b6f059a71a1ebd6133bd53714b2d684efaa6e734
    • patch SHA-256: 0cc29ded77485d0c7e3b4b9eb6b0af7485e91ab4ed121eb5c601143f0de9fb08
    • unstaged changes: none

    Canonical stamped command:

    node .codex/hooks/git-guard.cjs prove --reviewed --check "node scripts/run-proof-suite.cjs"

    Result: 11/11 PASS.

    • diff: 19ms
    • adapter: 302.2s (115/115)
    • packaging: 14.5s (39/39)
    • npm: 75.4s
    • skill: 1.5s
    • release: 408ms
    • roadmap: 2.1s (15/15)
    • benchmark: 1.2s
    • e2e: 19ms
    • setup: 250.8s
    • update: 583.9s

    Next gate is required normal Sol High + Fable High review over this unchanged candidate; no exception is available or requested.

  7. BaseInfinity commented on Aug 23, 2026

    @BaseInfinity
    BaseInfinity
    on Aug 23, 2026
    OwnerAuthor

    Initial joint review — certified with one bounded P2 correction

    Exact candidate b6f059a71a1ebd6133bd53714b2d684efaa6e734 completed both reviewers normally:

    • Sol High: CERTIFIED, confidence 98, no findings.
    • Fable High: CERTIFIED, confidence 80, one P2 and three P3s.
    • Joint: CERTIFIED, no reconciliation, reviewer-verdict outcome, no retry.

    The P2 is accurate enough to correct now: prompt text reduces deferral likelihood, while the actual guarantee is the existing strict validator plus non-certifying invalid-execution receipt. ROADMAP should not blur those facts.

    One bounded correction round will:

    1. reword ROADMAP to distinguish the prompt obligation from runtime fail-closed enforcement;
    2. centralize the shared final-JSON contract and reduce brittle exact-string duplication in tests;
    3. add narrated-prose-plus-valid-JSON rejection coverage;
    4. make the focused adapter filter exit status explicit.

    This changes the candidate, so the current proof/review will not be used for delivery. After narrow delta proof, run one fresh canonical 11/11 proof and a fresh normal Sol High + Fable High gate. No exception.

  8. BaseInfinity commented on Aug 23, 2026

    @BaseInfinity
    BaseInfinity
    on Aug 23, 2026
    OwnerAuthor

    Corrected final canonical proof passed on the exact staged candidate.

    Candidate identity:

    • base/HEAD: b7c8d45
    • tree: 4f5b8a27bda15f8833df39674afe192ff8812aef
    • staged patch SHA-256: 86888abc2ea106ddb5a2ded9ceeb5f05fb8bea96f5419232d3877486f0d94500

    Bounded correction after the first review:

    • centralized the cross-model tool-free final-response prompt contract;
    • added a narrated-JSON Fable fixture proving prose plus JSON remains invalid_execution, non-transient, and non-retryable while the Sol peer is cancelled;
    • made focused-test exit status explicit;
    • corrected ROADMAP wording so it does not claim prompt text is runtime enforcement.

    Fresh canonical proof:

    • PASS git diff --check (21ms)
    • PASS tests/test-adapter.sh (311.6s)
    • PASS tests/test-packaging.sh (14.4s)
    • PASS tests/test-npm.sh (74.7s)
    • PASS tests/test-skill.sh (1.6s)
    • PASS tests/test-release.sh (397ms)
    • PASS tests/test-roadmap.sh (2.0s)
    • PASS tests/test-benchmark.sh (597ms)
    • PASS tests/test-e2e.sh (21ms)
    • PASS tests/test-setup.sh (250.9s)
    • PASS tests/test-update.sh (584.2s)

    Result: 11/11 PASS. The earlier proof/review is superseded; fresh required Sol High + Fable High review follows on this exact candidate.

  9. BaseInfinity commented on Aug 23, 2026

    @BaseInfinity
    BaseInfinity
    on Aug 23, 2026
    OwnerAuthor

    Fresh required review on corrected tree 4f5b8a27bda15f8833df39674afe192ff8812aef completed normally but did not certify.

    Review outcome:

    • Sol High: NOT CERTIFIED (98)
    • Fable High: NOT CERTIFIED (88)
    • classification: reviewer_verdict
    • infrastructure failure: false
    • transient: false
    • retry recommended: false
    • no delivery, commit, push, or exception

    After one normal reconciliation round, both reviewers agreed on:

    • P1: the newly added ADAPTER_TEST_ONLY environment selector could be inherited by the canonical proof and create a false-green focused adapter run.
    • P2: the new invalid-Fable loop did not delete the previous receipt before asserting a fresh receipt.
    • P3 observations were non-blocking.

    Second and final bounded correction:

    • Added release-integrity coverage first. RED: tests/test-release.sh was 19/20 because the canonical adapter entrypoint honored an inherited focus selector.
    • Removed the focused environment selector entirely. GREEN: release tests 20/20.
    • Delete the receipt before every interim/narrated invalid-output case.
    • Full adapter regression: 115/115 PASS.

    The non-certifying review receipt is preserved. It is not being retried unchanged; the exact candidate has changed and now requires a fresh canonical proof and fresh normal dual review.

  10. BaseInfinity commented on Aug 23, 2026

    @BaseInfinity
    BaseInfinity
    on Aug 23, 2026
    OwnerAuthor

    Final corrected candidate is frozen and its one canonical proof passed.

    Candidate identity:

    • base/HEAD: b7c8d45
    • tree: ef23ee78047d9a37dc59fa3bceaf33da24a0d418
    • staged patch SHA-256: 7edf112624e46f38f411aa1fb866e79cca452b0f2c9109f898813d8d88280b46
    • no unstaged changes

    Final canonical proof:

    • PASS git diff --check (20ms)
    • PASS tests/test-adapter.sh, full 115/115 (309.9s)
    • PASS tests/test-packaging.sh (14.6s)
    • PASS tests/test-npm.sh (74.3s)
    • PASS tests/test-skill.sh (1.6s)
    • PASS tests/test-release.sh, 20/20 including canonical focus-selector integrity (397ms)
    • PASS tests/test-roadmap.sh (2.1s)
    • PASS tests/test-benchmark.sh (567ms)
    • PASS tests/test-e2e.sh (19ms)
    • PASS tests/test-setup.sh (249.8s)
    • PASS tests/test-update.sh (582.7s)

    Result: 11/11 PASS. The proof receipt is fresh and reviewed. A fresh normal Sol High + Fable High review now runs on this exact candidate. No exceptions and no further correction loop.

  11. BaseInfinity commented on Aug 23, 2026

    @BaseInfinity
    BaseInfinity
    on Aug 23, 2026
    OwnerAuthor

    Final normal dual review certified the exact candidate.

    Receipt binding:

    • base/HEAD: b7c8d45
    • candidate tree: ef23ee78047d9a37dc59fa3bceaf33da24a0d418
    • patch SHA-256: 7edf112624e46f38f411aa1fb866e79cca452b0f2c9109f898813d8d88280b46
    • status/certifying/joint verdict: certified / true / CERTIFIED

    Required reviewers:

    • Sol High: CERTIFIED, confidence 96, no findings
    • Fable High: CERTIFIED, confidence 82
    • both completed normally; classification reviewer_verdict
    • no infrastructure failure, fallback, retry, reconciliation, or exception

    Fable reported one non-blocking P2 about the literal-grep implementation of the canonical focus-selector regression plus three P3 maintainability observations. The bounded correction cap is exhausted, so the certified tree is not being mutated. The P2 is recorded as #176 in the 1.1.0 harness-hardening milestone: #176

    Afterhours check: 2026-08-23 16:51 PDT, Sunday. Remote delivery is allowed. Candidate-bound delivery follows.

  12. BaseInfinity
    closed this as completedin #177on Aug 23, 2026

  13. BaseInfinity commented on Aug 23, 2026

    @BaseInfinity
    BaseInfinity
    on Aug 23, 2026
    OwnerAuthor

    Post-merge verification complete.

    #175 is complete. The release train resumes #174 from a fresh current-main worktree; the previous non-certifying #174 candidate remains frozen as evidence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority: P0Stop-the-line critical path; work before lower-priority roadmap items

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions