[jayleaton]

Rules. Before merge, run what this change needs on its exact head (docs/CI.md, "What to run"): for Go, make lint test scoped with LINT_DIRS/TEST_PKGS (scripts/affected.sh prints them) or make check-fast, plus the checks of what you touched. Then make ci-local RAN="<what you ran>" posts local-ci and fills "Local verification" with that statement and the head SHA. Never edit that section by hand. The Local verification check, make pr-merge and the staging gate refuse the PR while it is missing, names another head, or differs from what local-ci attested. A new push or rebase needs a new attestation. A reported failure (FAILED=) is final for that head: fix it in a new commit (docs/CI.md, "Local verification").

Problem

Every PR ran its checks on GitHub-hosted runners (ci-ok), so verification cost billed minutes on every push and could not run at all on a repo whose Actions budget is blocked. Railway deployed straight from GitHub on every push, with nothing proving the commit was verified, and production moved by branch promotion (dev → main).

Lockpick first replaced that with a local runner. It still blocked everything: in Lockpick #1210 a 4-file PR took about 15 min and 16 PRs queued for about 4 hours. JJ, 31 Oct: "ci:local just tells the PR that you have sanitized that CI should have passed." The template ships that final model for a Go + make + Railway stack.

Fix

flowchart LR
  ran["run what the change needs<br/>(make lint test, scoped)"] --> push[push]
  push --> attest["make ci-local RAN=…<br/>runs nothing · seconds"]
  attest --> status[local-ci on the exact SHA]
  attest --> body[PR body: Local verification]
  body -- "pull_request: edited" --> gh["CI: Secret scan + claim + Local verification"]
  gh --> merge["make pr-merge<br/>all green PRs at once"]
  merge --> staging["Deploy staging (push to main):<br/>attested only → Railway staging"]
  merge -.-> check["--check-deploy &lt;squash&gt;"]
  staging --> prod["Draft release → publish → Deploy production (gate)"]

Loading

• Attestation (Lockpick #1210): make ci-local RAN="<what you ran>" (scripts/cilocal, Go, standard library only) runs nothing.
  • It refuses a dirty, unpushed or already-failed head, posts local-ci on the exact SHA (result and section digest), and fills the section with the head, who and where, and the statement.
  • FAILED= records a failure, which is final for that head; COMMIT= attests a main commit.
  • It takes seconds: there is no step plan, runner, reuse ledger, Davit routing, slot, memory wait or failure triage any more (−3,361 / +2,049 lines).
• One rule (scripts/cilocal/verify, a test boundary at 99.4% coverage): the section names the head, says what ran, and matches the status digest, and the head never failed. The Local verification job, make pr-merge and the staging gate (cilocal attest, Deploy staging's plan) all use it.
• Expected, not red (Lockpick #1197): a Local verification claim job names the checking job Awaiting make ci-local until the head is claimed, so an unattested PR's required check stays Expected. A claimed head with a bad section is red.
• Merging: make pr-merge PR="…" merges every green PR at once and never waits.
  • It retargets stacked children and squashes with --match-head-commit, then prints ARGS="--check-deploy <squash sha>", a one-read check of Deploy staging plus /healthz. --check-staging checks main's head.
  • CI jobs GitHub never started (Actions blocked by billing) still count as not run, so local-ci alone merges there.
• Post-merge report (Lockpick #1196): comments on the merged PR when Deploy staging fails on main. A verify test fails when a push-to-main workflow is missing from its list.
• Deploys (earlier commits, as in Lockpick):
  • Deploy staging, Staging catch-up, Draft release and Deploy production (with a gate), with .github/scripts/deploy-railway.sh. Each environment holds its own RAILWAY_TOKEN.
  • Until RAILWAY_DEPLOY is true the deploys dry-run.
  • deploy/platform.yaml defaults to repo: "" (Railway's auto-deploy off), and main is the only deployed branch.
• Kept: no deploy or UI tests; the test-boundary check stays in make lint; deploy-railway.sh omits --environment when a token is present.
• Docs:
  • docs/CI.md: "What runs", "What to run", "Local verification", "Merging".
  • AGENTS.md: "Verifying" and "Pull requests".
  • The PR template, CONTEXT.md and docs/RELEASES.md.
  • The github-prs skill (.claude/skills/ and .agents/skills/, identical), linked from AGENTS.md and CLAUDE.md.

Evidence

• Ran on head 9068dd7, in a Davit box of the exact head (8 stale image files pruned first):
  • go test -cover ./scripts/cilocal/verify: 99.4%.
  • go vet, full make lint (0 issues), make lint-layering, the test probes, the coverage floors, make actionlint and make deploy-check: all pass.
• Attested: make ci-local RAN="…" took 8 s. It posted local-ci and filled the section, and that edit's run passed Secret scan, Local verification claim and Local verification.
• Hand-edited section refused: I changed one word in the section's Ran line.
  • make pr-merge PR=134 ARGS=--dry-run said skip - is not verified locally: the Local verification section is not the one local-ci attested (edited by hand?).
  • The edit's CI run failed Local verification with the same error. Re-attesting restored the section.
• Unattested head: shown on the sparkrack port's PR before its attestation; see that PR.
• Earlier commits: the deploy pipeline's actionlint, its dry runs and the production gate's read-only run are as described in the previous revision of this PR.

Merge Danger

Door: two-way for code. Reverting restores the old local runner. One-way only if someone sets RAILWAY_DEPLOY=true and publishes a release.

Blast Radius: platform

Every later PR of the template, and of the repos cloned from it, is verified by the agent and attested with make ci-local RAN=…, not run by it. The attestation is only as honest as the statement; Lockpick has the same risk.

Settings needed, not made here:

• Rulesets:
  • main requires Secret scan and Local verification (GitHub Actions).
  • A tag ruleset on refs/tags/v*.
• Environments:
  • staging: main only, secret RAILWAY_TOKEN.
  • production: main and v*, secret RAILWAY_TOKEN.
• Variables: PRODUCTION_RELEASERS, STAGING_WEB_URL, PRODUCTION_WEB_URL, the optional admin URLs, RAILWAY_DEPLOY, and the optional RAILWAY_PROJECT_ID, CI_RUNNER and RELEASE_RUNNER.

🤖 Generated with Claude Code

Local verification

Filled by make ci-local, 2026-10-31T12:07:41Z. Never edit this section by hand.

Head 34e7ffeff2d416536134b4e2355538d45b842d6f · tree 97b32790bc96 · passed
Attested by unknown on dev-box

Ran: make ci; make build; make actionlint; make check-fast; make secret-scan (no leaks, refused only the .gitleaks.toml change); Randy's 5-location probe: the same synthetic token now fails in all 5

[jayleaton]

CHANGES — reviewed bf0b5d3 (63 files, +6,469/-1,044). Three inline findings:

1. Doug (complex), P1: run the hosted verifier and scanner/config from trusted code; PR-controlled Go can bypass both required checks, including from a fork.
2. Danny (simple), P1: make a report rejected for tree drift permanently unpublishable.
3. Doug (complex), P2: provide a safe first-attestation/full-tree path for a new template repository with no attested ancestors, including a root commit.

Evidence: all 13 selected ci-local steps passed locally, zero reuse, in 3m33s. Invoked make ci-local itself: correctly refused detached HEAD; then executed every selected item through make ci-local ARGS="--local --only secrets,build,lint,generated,tests,migrations,smoke,deploy,mobile --keep-going", which posts nothing. Separate make lint, make test and make build passed; 12 test packages passed, all six coverage floors passed, and cilocal/verify has 100% statement coverage. Four additional focused reproduction tests passed; separate PR-controlled init() probes made both hosted subcommands return 0 without verification or scanning.

Hand-edited sections, wrong-head reports/new heads without a status, and a missing executable were refused in focused checks. Secret scan and Local verification remain hosted, triggered only by PR edits; staging/release gates and bench remain workflows. The nine-item body/ledger/digest format follows the Lockpick model. A writer can still forge local-ci by design; the fork/code-execution bypass is a separate trust-boundary problem.

Fresh-project evidence: exercised the current new-pattern-project helper against this head with --skip-github --verify. Renamed lint passed; tests failed at the existing unchanged packages/auth/tokens_test.go mixed-case BetterStack://Home fixture after the helper rewrites only its lowercase counterpart. Renamed build passed separately. That bootstrap failure also exists in the base inputs and is not counted as a new inline finding. Independently, the clean new-repository/no-status fixture reproduced the new first-attestation refusal described inline.

All heavy checks used ~/Documents/Projects/spark-work/heavy.sh; no VMs. The review clone and reproducible evidence are under ~/Documents/Projects/spark-work/zig-dsv41-scratch/randy-gfs134{,-evidence}. No PR branch pushes, merges, or local-ci status/body updates were made. The PR head remained unchanged at final verification.

[jayleaton]

Pushed 2f1cca7, on top of the attest-only rework (9068dd7), with fixes for all three review findings and regression tests in scripts/cilocal/verify, still at 100% coverage. make ci, build, actionlint and check-fast pass, and make ci-local attested the head. Note that ci.yml now runs on pull_request_target, which reads the workflow from main, so this PR gets no hosted checks until it merges. The rename failure Randy saw on fresh projects (packages/auth/tokens_test.go BetterStack://Home) predates this PR and is not fixed here.

[jayleaton]

CHANGES — reviewed 2f1cca7. One remaining P1 is inline: Danny (simple), pin gitleaks' ignore policy to trusted input.

The target workflow never checks out or executes head code. All jobs explicitly use read-only permissions; no repository secrets/write token reach the scanner. But copied head .gitleaksignore is interpreted as scanner policy: the synthetic-secret control exits 1, then the same head plus an ignore fingerprint exits 0.

Forks fail closed. A/B probes show old fork acceptance/root refusal and safe new behavior. The old dirty-report publication safety assertion fails; the new --publish call refuses without posting. The workflow guard rejects actual bf0b5d3 YAML and accepts this head. The rewritten tests are not portable onto the old APIs; comparisons used executable reproductions. Note the design change: ci-local now records a manual signoff and runs no checks; the saved-report runner is removed.

Validation through heavy.sh: lint, actionlint, build, 36 rule tests/163 including subtests, 100% statement coverage. All three commands replayed against real #134 passed locally, including the 62-file scan.

Bootstrap: local-ci-only dry-run is not hosted proof. Live validation before merging here is possible in a disposable repository with the candidate on its default branch and hostile branch/fork PRs. Local replay cannot prove GitHub event/ref wiring. No pushes, merges, workflow dispatches or status/body updates.

[jayleaton]

CHANGES — reviewed ce1ba40. The suppression-input fix works, but I do not accept the proposed blanket path allowlist; one P1 is inline (Danny, simple).

Seven detection probes failed closed: positive control, .gitleaksignore, inline gitleaks:allow, GITLEAKS_CONFIG, GITLEAKS_CONFIG_TOML, alternate config filenames, and a head-supplied extension config. In contrast, the identical synthetic PAT passed in all five excluded locations: *_test.go, testdata/, root/nested .env.example, and scripts/bench-devloop.sh.

No additional active config-discovery route was found. The trusted policy only uses useDefault=true. Future [extend].path policy must remain outside scanned content: upstream resolves it from the working directory, not the config's directory (pinned documentation).

Lint, actionlint, build and 39 top-level tests/166 including subtests passed; rules coverage is 100%. All computation used heavy.sh.

The self-flag is expected: make secret-scan found no leaks, then exited nonzero solely because this PR adds policy. Its local-ci statement honestly records that. Current pr-merge --dry-run would merge on local-ci alone because no hosted job ran; this is a bootstrap/manual-review exception, not approval of the policy. A real hosted Secret scan failure still blocks ordinary pr-merge and is not overridden by local-ci. Tighten the policy before accepting that exception. No push or merge.

[jayleaton]

MERGE — focused final review of 34e7ffe. No remaining findings.

All 5 location probes and all 7 suppression probes now fail. The AND behavior is verified with the real scanner: the exact fake fixture passes in the bench script, fails elsewhere, and a one-character variation fails both inside and outside the script. The two explicit bench values are visibly synthetic; the exception is scoped to generic-api-key.

TestScannerPolicyAllowlistsValuesOnly fails with ce1ba40's config and passes with this config. Lint, build and all 41 top-level tests/168 including subtests passed, with 100% rules coverage. secret-scan --all scanned 469 files and found no leaks. All computation used heavy.sh.

Only five files changed since ce1ba40: policy, docs, regression test/helper, and scanner error-message wording. No other runtime behavior changed. No push or merge.