diff --git a/.agents/skills/github-prs/SKILL.md b/.agents/skills/github-prs/SKILL.md new file mode 100644 index 0000000..111cca4 --- /dev/null +++ b/.agents/skills/github-prs/SKILL.md @@ -0,0 +1,105 @@ +--- +name: github-prs +description: How every agent verifies and attests locally (`make ci-local RAN="…"`), opens, stacks and merges pull requests in this repo, fast. Use before any `git commit`, `gh pr create`, `gh pr merge`, retargeting or rebasing a stacked PR, or re-running a check, and whenever you are about to wait on CI, write a merge script, or merge PRs "one at a time". +--- + +# Pull requests: run what you need, attest, stack, merge at once + +Run the checks your change needs (lint, tests, build), then attest with `make ci-local RAN="…"` and merge right away. `make ci-local` runs nothing: it tells the PR you checked, and what. GitHub runs no verification on `main`, and on a PR only two small jobs that check the attestation. Stack PRs and merge each the moment it is attested; after the merge, check its staging deploy. + +The rules on what may merge are in `AGENTS.md` ("Verifying", "Pull requests", "Environments: staging is always current"); the detail is in `docs/CI.md` ("What runs", "What to run", "Local verification", "Merging on local verification"). This skill is how you do it. + +## 1. Before every commit + +- **Never skip the hooks.** `make hooks` installs the pre-push hook (`make check-fast`); never push with `--no-verify` except in an emergency, and say so. +- Run what your change needs (the table is `docs/CI.md`, "What to run"): `make lint LINT_DIRS=` and `make test TEST_PKGS=` for the Go packages it can affect (`scripts/affected.sh` prints them; `make check-fast` is the seconds-long pass), `make check-generated` for sqlc, templ or mock inputs, `make check-migrations` for a migration, `make deploy-check` for `deploy/`, `make actionlint` for workflows, `make release-smoke` for a Dockerfile or what it builds, `make ci` for a change to the tooling. Tests only at the test boundaries (`AGENTS.md`, "Test boundaries"); never a test in `deploy/` or the UI. +- A change adds no lint or format findings, warnings included. +- Add the paths you changed; never `git add -A` in an old worktree. + +## 2. Attest: `make ci-local RAN="…"` + +After **every** push or rebase, once you have run what the change needs: + +```bash +git push # pushed head, clean worktree +make ci-local RAN="make lint LINT_DIRS=./apps/web/... ; make test TEST_PKGS=./apps/web/internal/services" +``` + +- It runs nothing and takes seconds. It refuses a dirty worktree, a detached HEAD (except `COMMIT=` of a `main` commit), a head that is not the pushed branch head, or a head that already has a failed `local-ci`. +- It posts `local-ci` on that exact SHA and fills the PR body's **Local verification** section (head, who, and your statement). **Never edit the section by hand:** the status carries its digest. +- **One attestation, one commit.** Any push, amend or rebase needs a new one. No PR yet? Open it (section 3) and run it again. +- **Something failed?** Fix it before attesting. If you attest a failure (`make ci-local FAILED=""`), that head is final: fix forward with a new commit. Code this PR touched: fix it here. Anything else: its own PR off `main` with a test that fails before the fix, then rebase. Add the prevention. +- Say exactly what ran; a reviewer reads the scope from it. + +## 3. Open and stack + +- `gh pr create --base ` with `.github/pull_request_template.md`, open and ready, no labels: problem, fix, and the Local verification section. Conventional commit title; the body ends with the attribution line. +- **Base per layer.** Needs nothing unmerged: base `main`. Needs an open PR: base that PR's branch. Independent work gets its own PR on `main`. +- One concern per PR; push follow-ups to the same branch. **Never close and reopen a PR.** + +## 4. What PR CI and the merge script refuse + +PR CI runs only `Secret scan` and `Local verification`, the two checks the main ruleset requires, on the `CI_RUNNER` runner. It runs only on a PR body edit, and until `make ci-local` claims the head (posts `local-ci` or fills the section for it) the check runs as `Awaiting make ci-local`, so the required one shows **Expected**, not red; red is a real problem. `Local verification` and `make pr-merge` share one rule (`scripts/cilocal/verify`, `LocalVerificationProblems`): + +| Refused because | Fix | +|---|---| +| no `local-ci` status on the head (never attested, or a new push) | run what it needs, `make ci-local RAN="…"` | +| `local-ci` is `failure`, or failed on this head before | fix forward with a new commit, push, attest | +| no Local verification section, it names another head or no "Ran:" | `make ci-local RAN="…"` | +| the section is not the attested one (edited by hand) | `make ci-local RAN="…"` again restores it | +| `Secret scan` found a credential | remove it (rotate it if real), push, attest | +| it conflicts with `main` | rebase onto `main`, push, attest | + +`make ci-local` posts the status before it edits the body, and that edit starts the PR run, so you never re-run `Local verification` by hand; when the body was already current, `make ci-local` re-runs the run itself. + +## 5. Merge + +```bash +make pr-merge PR=" [ ...]" # merges every green PR now, reports the rest +make pr-merge PR= ARGS=--dry-run # says what it would do +``` + +- **Green** means `local-ci` and the section attest the exact head, GitHub can merge it without conflicts, and the checks passed. It squash-merges with `--match-head-commit `, so only the attested head lands; behind `main` is fine. Where Actions cannot run (no CI job ran, or GitHub never started them), it merges on `local-ci` alone and says so. +- Anything else (draft, unattested, pending, failing, conflicting, stacked) is reported with its next step; the script never waits. Only a conflict needs rebase, push, attest. +- **After each merge, check its deploy** (a follow-up, never a gate; merge the next PR now): it prints `make pr-merge ARGS="--check-deploy "`. Run it once "Deploy staging" has run for that commit. Failed or refused: top priority, fix forward. Cancelled by a newer main deploy: `make pr-merge ARGS=--check-staging` checks main's head. A dry run (`RAILWAY_DEPLOY` not `true`) deployed nothing. After a burst, `ARGS=--then-check-staging` follows main's deploy (bounded). +- **Stacked children** are retargeted to `main` before their parent merges. Then `git rebase --onto origin/main `, `git push --force-with-lease`, attest. +- Integration branches (`feat/`): `make pr-merge PR= ARGS=--any-base`. +- Never `gh pr merge --delete-branch`: it closes every PR stacked on the branch and deletes any worktree holding it. Never write your own merge or queue script; change `scripts/cilocal` (rules and tests in `scripts/cilocal/verify`). +- **Admin bypass** only when the developer authorized it and a GitHub check is red for a reason outside the code, and only after `make ci-local RAN="…"` attested the head: `gh pr merge --squash --admin --match-head-commit `, then a PR comment "Authorized by , merged by " with the attestation. + +## 6. Checks that fail outside the code + +- **Lost runner** ("lost communication with the server", "received a shutdown signal", "The operation was canceled.", a timeout with no log): `gh run rerun --failed`, once, by hand. Nothing re-runs it automatically. +- **Actions blocked** (billing or a spending limit: "The job was not started because…"): those jobs count as not run; `make pr-merge` merges on `local-ci` alone (`docs/CI.md`, "When Actions cannot run"). +- A real failure belongs to the PR's owner; the rest of the burst doesn't wait for it. +- One read per question (`gh pr checks `, `gh run view --json jobs`), never a polling loop. + +## 7. Staging deploys itself + +`main` runs no CI. "Deploy staging" deploys a push to Railway staging only when `make ci-local` attested it (`cilocal attest`): the squash of an attested PR head, or a commit with its own `local-ci`. Two verified PRs merged together are not themselves verified; a clash surfaces on staging and is fixed forward. The hourly "Staging catch-up" deploys `main` whenever staging is behind and `main` is attested. Never wait for a deploy or for another PR to settle: merge each PR when its own head is attested and conflict-free, and stack the next at once; check its deploy afterwards (section 5); "Post-merge report" comments on the PR when a push workflow on its commit fails. A failed deploy is fixed forward in parallel and blocks no unrelated merge. + +An unattested `main` is the top priority: both workflows fail and say why. Fix it forward: `git worktree add --detach `, run what the commits since the last attested one need (`make ci` when there is none, as in a new project's first attestation), then `make ci-local COMMIT= RAN="…"` there. It refuses a dirty tree, a commit off `origin/main` or one that already failed, and posts `local-ci` on the commit; then dispatch "Deploy staging". + +## 8. Waiting without losing the work + +- **Never end a turn while your own wait, watcher or helper still runs.** Register it (`t3_background_wait`) or wait for it. +- **Never `pgrep -f`, `pkill -f` or kill a process found by name**; wait on a PID captured at spawn or the background task's ID. +- A background command from a T3 tool call dies with that tool's scope and has a two-hour limit. A merge never needs one. + +## Rules and what enforces them + +| Rule | Enforced by | +|---|---| +| Every PR is attested on its exact head | `Local verification` (`ci.yml`) and `verify.Decide`, through `verify.LocalVerificationProblems`; a skipped `Secret scan` is never green | +| The section is the script's, never hand-edited | the section digest in the `local-ci` status | +| Merge only the verified head, never a conflict | `--match-head-commit` and the conflict refusal in `make pr-merge` | +| Staging deploys only an attested PR head or commit | `cilocal attest` in Deploy staging's plan, `verify.Attestation` and its test | +| Merge every green PR at once; stacks survive | `make pr-merge` (no wait path, retargets children), `scripts/cilocal/verify/merge_test.go` | +| A push workflow that fails on main is reported | `Post-merge report`; a verify test fails when a push-to-main workflow is missing from its list | +| No `--no-verify`, no reopen, no `pgrep -f` | this skill only; add a check if an agent repeats one | + +## Done means + +- Every PR you were asked to land is merged, or reported with its blocker and next step. +- Every PR you pushed to is attested on its current head, and each merge's staging deploy was checked. +- No PR was closed and reopened, and no hooks were skipped. diff --git a/.claude/skills/github-prs/SKILL.md b/.claude/skills/github-prs/SKILL.md new file mode 100644 index 0000000..111cca4 --- /dev/null +++ b/.claude/skills/github-prs/SKILL.md @@ -0,0 +1,105 @@ +--- +name: github-prs +description: How every agent verifies and attests locally (`make ci-local RAN="…"`), opens, stacks and merges pull requests in this repo, fast. Use before any `git commit`, `gh pr create`, `gh pr merge`, retargeting or rebasing a stacked PR, or re-running a check, and whenever you are about to wait on CI, write a merge script, or merge PRs "one at a time". +--- + +# Pull requests: run what you need, attest, stack, merge at once + +Run the checks your change needs (lint, tests, build), then attest with `make ci-local RAN="…"` and merge right away. `make ci-local` runs nothing: it tells the PR you checked, and what. GitHub runs no verification on `main`, and on a PR only two small jobs that check the attestation. Stack PRs and merge each the moment it is attested; after the merge, check its staging deploy. + +The rules on what may merge are in `AGENTS.md` ("Verifying", "Pull requests", "Environments: staging is always current"); the detail is in `docs/CI.md` ("What runs", "What to run", "Local verification", "Merging on local verification"). This skill is how you do it. + +## 1. Before every commit + +- **Never skip the hooks.** `make hooks` installs the pre-push hook (`make check-fast`); never push with `--no-verify` except in an emergency, and say so. +- Run what your change needs (the table is `docs/CI.md`, "What to run"): `make lint LINT_DIRS=` and `make test TEST_PKGS=` for the Go packages it can affect (`scripts/affected.sh` prints them; `make check-fast` is the seconds-long pass), `make check-generated` for sqlc, templ or mock inputs, `make check-migrations` for a migration, `make deploy-check` for `deploy/`, `make actionlint` for workflows, `make release-smoke` for a Dockerfile or what it builds, `make ci` for a change to the tooling. Tests only at the test boundaries (`AGENTS.md`, "Test boundaries"); never a test in `deploy/` or the UI. +- A change adds no lint or format findings, warnings included. +- Add the paths you changed; never `git add -A` in an old worktree. + +## 2. Attest: `make ci-local RAN="…"` + +After **every** push or rebase, once you have run what the change needs: + +```bash +git push # pushed head, clean worktree +make ci-local RAN="make lint LINT_DIRS=./apps/web/... ; make test TEST_PKGS=./apps/web/internal/services" +``` + +- It runs nothing and takes seconds. It refuses a dirty worktree, a detached HEAD (except `COMMIT=` of a `main` commit), a head that is not the pushed branch head, or a head that already has a failed `local-ci`. +- It posts `local-ci` on that exact SHA and fills the PR body's **Local verification** section (head, who, and your statement). **Never edit the section by hand:** the status carries its digest. +- **One attestation, one commit.** Any push, amend or rebase needs a new one. No PR yet? Open it (section 3) and run it again. +- **Something failed?** Fix it before attesting. If you attest a failure (`make ci-local FAILED=""`), that head is final: fix forward with a new commit. Code this PR touched: fix it here. Anything else: its own PR off `main` with a test that fails before the fix, then rebase. Add the prevention. +- Say exactly what ran; a reviewer reads the scope from it. + +## 3. Open and stack + +- `gh pr create --base ` with `.github/pull_request_template.md`, open and ready, no labels: problem, fix, and the Local verification section. Conventional commit title; the body ends with the attribution line. +- **Base per layer.** Needs nothing unmerged: base `main`. Needs an open PR: base that PR's branch. Independent work gets its own PR on `main`. +- One concern per PR; push follow-ups to the same branch. **Never close and reopen a PR.** + +## 4. What PR CI and the merge script refuse + +PR CI runs only `Secret scan` and `Local verification`, the two checks the main ruleset requires, on the `CI_RUNNER` runner. It runs only on a PR body edit, and until `make ci-local` claims the head (posts `local-ci` or fills the section for it) the check runs as `Awaiting make ci-local`, so the required one shows **Expected**, not red; red is a real problem. `Local verification` and `make pr-merge` share one rule (`scripts/cilocal/verify`, `LocalVerificationProblems`): + +| Refused because | Fix | +|---|---| +| no `local-ci` status on the head (never attested, or a new push) | run what it needs, `make ci-local RAN="…"` | +| `local-ci` is `failure`, or failed on this head before | fix forward with a new commit, push, attest | +| no Local verification section, it names another head or no "Ran:" | `make ci-local RAN="…"` | +| the section is not the attested one (edited by hand) | `make ci-local RAN="…"` again restores it | +| `Secret scan` found a credential | remove it (rotate it if real), push, attest | +| it conflicts with `main` | rebase onto `main`, push, attest | + +`make ci-local` posts the status before it edits the body, and that edit starts the PR run, so you never re-run `Local verification` by hand; when the body was already current, `make ci-local` re-runs the run itself. + +## 5. Merge + +```bash +make pr-merge PR=" [ ...]" # merges every green PR now, reports the rest +make pr-merge PR= ARGS=--dry-run # says what it would do +``` + +- **Green** means `local-ci` and the section attest the exact head, GitHub can merge it without conflicts, and the checks passed. It squash-merges with `--match-head-commit `, so only the attested head lands; behind `main` is fine. Where Actions cannot run (no CI job ran, or GitHub never started them), it merges on `local-ci` alone and says so. +- Anything else (draft, unattested, pending, failing, conflicting, stacked) is reported with its next step; the script never waits. Only a conflict needs rebase, push, attest. +- **After each merge, check its deploy** (a follow-up, never a gate; merge the next PR now): it prints `make pr-merge ARGS="--check-deploy "`. Run it once "Deploy staging" has run for that commit. Failed or refused: top priority, fix forward. Cancelled by a newer main deploy: `make pr-merge ARGS=--check-staging` checks main's head. A dry run (`RAILWAY_DEPLOY` not `true`) deployed nothing. After a burst, `ARGS=--then-check-staging` follows main's deploy (bounded). +- **Stacked children** are retargeted to `main` before their parent merges. Then `git rebase --onto origin/main `, `git push --force-with-lease`, attest. +- Integration branches (`feat/`): `make pr-merge PR= ARGS=--any-base`. +- Never `gh pr merge --delete-branch`: it closes every PR stacked on the branch and deletes any worktree holding it. Never write your own merge or queue script; change `scripts/cilocal` (rules and tests in `scripts/cilocal/verify`). +- **Admin bypass** only when the developer authorized it and a GitHub check is red for a reason outside the code, and only after `make ci-local RAN="…"` attested the head: `gh pr merge --squash --admin --match-head-commit `, then a PR comment "Authorized by , merged by " with the attestation. + +## 6. Checks that fail outside the code + +- **Lost runner** ("lost communication with the server", "received a shutdown signal", "The operation was canceled.", a timeout with no log): `gh run rerun --failed`, once, by hand. Nothing re-runs it automatically. +- **Actions blocked** (billing or a spending limit: "The job was not started because…"): those jobs count as not run; `make pr-merge` merges on `local-ci` alone (`docs/CI.md`, "When Actions cannot run"). +- A real failure belongs to the PR's owner; the rest of the burst doesn't wait for it. +- One read per question (`gh pr checks `, `gh run view --json jobs`), never a polling loop. + +## 7. Staging deploys itself + +`main` runs no CI. "Deploy staging" deploys a push to Railway staging only when `make ci-local` attested it (`cilocal attest`): the squash of an attested PR head, or a commit with its own `local-ci`. Two verified PRs merged together are not themselves verified; a clash surfaces on staging and is fixed forward. The hourly "Staging catch-up" deploys `main` whenever staging is behind and `main` is attested. Never wait for a deploy or for another PR to settle: merge each PR when its own head is attested and conflict-free, and stack the next at once; check its deploy afterwards (section 5); "Post-merge report" comments on the PR when a push workflow on its commit fails. A failed deploy is fixed forward in parallel and blocks no unrelated merge. + +An unattested `main` is the top priority: both workflows fail and say why. Fix it forward: `git worktree add --detach `, run what the commits since the last attested one need (`make ci` when there is none, as in a new project's first attestation), then `make ci-local COMMIT= RAN="…"` there. It refuses a dirty tree, a commit off `origin/main` or one that already failed, and posts `local-ci` on the commit; then dispatch "Deploy staging". + +## 8. Waiting without losing the work + +- **Never end a turn while your own wait, watcher or helper still runs.** Register it (`t3_background_wait`) or wait for it. +- **Never `pgrep -f`, `pkill -f` or kill a process found by name**; wait on a PID captured at spawn or the background task's ID. +- A background command from a T3 tool call dies with that tool's scope and has a two-hour limit. A merge never needs one. + +## Rules and what enforces them + +| Rule | Enforced by | +|---|---| +| Every PR is attested on its exact head | `Local verification` (`ci.yml`) and `verify.Decide`, through `verify.LocalVerificationProblems`; a skipped `Secret scan` is never green | +| The section is the script's, never hand-edited | the section digest in the `local-ci` status | +| Merge only the verified head, never a conflict | `--match-head-commit` and the conflict refusal in `make pr-merge` | +| Staging deploys only an attested PR head or commit | `cilocal attest` in Deploy staging's plan, `verify.Attestation` and its test | +| Merge every green PR at once; stacks survive | `make pr-merge` (no wait path, retargets children), `scripts/cilocal/verify/merge_test.go` | +| A push workflow that fails on main is reported | `Post-merge report`; a verify test fails when a push-to-main workflow is missing from its list | +| No `--no-verify`, no reopen, no `pgrep -f` | this skill only; add a check if an agent repeats one | + +## Done means + +- Every PR you were asked to land is merged, or reported with its blocker and next step. +- Every PR you pushed to is attested on its current head, and each merge's staging deploy was checked. +- No PR was closed and reopened, and no hooks were skipped. diff --git a/.github/actions/check/action.yml b/.github/actions/check/action.yml deleted file mode 100644 index 86dee5f..0000000 --- a/.github/actions/check/action.yml +++ /dev/null @@ -1,121 +0,0 @@ -# The checks one CI job runs, shared by the single job (mode "all") and the -# two fan-out jobs ("db": lint + DB-backed tests, "other": the rest + mobile). -# Expects: checkout, module cache, Go toolchain and the affected outputs -# (scripts/affected.sh) to be done by the caller. The steps here only set up -# tools and caches; scripts/ci-check.sh runs the checks themselves as parallel -# tasks (lint, migration order, generated code, tests, release smoke, deploy/, -# mobile), then -# the probes. -name: check -description: Lint, test and (optionally) mobile checks for the affected packages -inputs: - mode: { description: "all | db | other", required: true } - packages: { description: "affected import paths (or ./...)", required: true } - dirs: { description: "affected package dirs for golangci-lint", required: true } - full: { description: "full run (shared inputs changed)", required: true } - sqlc: { description: "rerun sqlc", required: true } - templ: { description: "rerun templ", required: true } - generate: { description: "packages to go generate", required: true } - mobile: { description: "run the mobile app checks", required: true } - deploy: { description: "run the deploy/ module checks", required: true } - smoke: { description: "run the release smoke (scripts/release-smoke.sh)", required: true } - save: { description: "save the build cache", required: true } -runs: - using: composite - steps: - - name: Packages to test - if: inputs.packages != '' - id: shard - shell: bash - run: ./scripts/ci-shard.sh "${{ inputs.mode }}" - env: - PACKAGES: ${{ inputs.packages }} - - # Boots while caches restore and lint runs; only DB-backed test binaries - # wait for it (see scripts/ci-check.sh). tmpfs and no fsync: test data is - # throwaway. The fan-out "db" job starts it itself, first thing (ci.yml). - - name: Start Postgres - if: (steps.shard.outputs.db == 'true' || inputs.smoke == 'true') && inputs.mode != 'db' - shell: bash - run: | - nohup docker run -d --name pg -p 5432:5432 -e POSTGRES_PASSWORD=postgres \ - --tmpfs /var/lib/postgresql postgres:18-alpine \ - -c fsync=off -c synchronous_commit=off -c full_page_writes=off -c max_connections=200 \ - > "$RUNNER_TEMP/pg.log" 2>&1 & - - # Build + lint cache, keyed by commit: saved by the single CI job on pushes - # to dev or main and restored by prefix everywhere else (incl. opt-in fan-out). - - uses: actions/cache/restore@v6 - if: inputs.packages != '' || inputs.deploy == 'true' || inputs.smoke == 'true' - with: - path: | - ~/.cache/go-build - ~/.cache/golangci-lint - key: v7-go-${{ runner.os }}-${{ github.sha }} - restore-keys: v7-go-${{ runner.os }}- - - - uses: sqlc-dev/setup-sqlc@v5 - if: inputs.mode != 'other' && inputs.sqlc == 'true' - with: - sqlc-version: '1.31.1' - - # Only the binary: ci-check.sh runs it (root module and deploy/ at once). - # Its own cache rides in the build cache above. - - uses: golangci/golangci-lint-action@v9 - if: (inputs.mode != 'other' && inputs.packages != '') || inputs.deploy == 'true' - with: - version: v2.15.0 - install-only: true - - # ---- mobile app: Node from the runner's tool cache, pnpm via corepack - # (the packageManager pin), node_modules cached by lockfile ---- - # (The contract test, pnpm test:contract, needs Go + Docker and runs locally.) - - uses: actions/setup-node@v7 - if: inputs.mobile == 'true' - with: - node-version: 22 - - name: pnpm - if: inputs.mobile == 'true' - shell: bash - run: corepack enable - env: - COREPACK_ENABLE_DOWNLOAD_PROMPT: '0' - - uses: actions/cache/restore@v6 - if: inputs.mobile == 'true' - id: node-modules - with: - path: apps/mobile-app/node_modules - key: v1-mobile-nm-${{ runner.os }}-node22-${{ hashFiles('apps/mobile-app/pnpm-lock.yaml') }} - - - name: Checks - shell: bash - run: ./scripts/ci-check.sh - env: - MODE: ${{ inputs.mode }} - DIRS: ${{ inputs.packages != '' && inputs.dirs || '' }} - TESTS: ${{ steps.shard.outputs.packages }} - DB: ${{ steps.shard.outputs.db }} - FULL: ${{ inputs.full }} - SQLC: ${{ inputs.sqlc }} - TEMPL: ${{ inputs.templ }} - GENERATE: ${{ inputs.generate }} - MOBILE: ${{ inputs.mobile }} - DEPLOY: ${{ inputs.deploy }} - SMOKE: ${{ inputs.smoke }} - EVENT: ${{ github.event_name }} - TARGET: ${{ github.base_ref || github.ref_name }} - COREPACK_ENABLE_DOWNLOAD_PROMPT: '0' - - - uses: actions/cache/save@v6 - if: inputs.save == 'true' && inputs.packages != '' && inputs.mode == 'all' - with: - path: | - ~/.cache/go-build - ~/.cache/golangci-lint - key: v7-go-${{ runner.os }}-${{ github.sha }} - # Any branch saves a missing node_modules: there is one per lockfile. - - uses: actions/cache/save@v6 - if: inputs.mobile == 'true' && steps.node-modules.outputs.cache-hit != 'true' - with: - path: apps/mobile-app/node_modules - key: v1-mobile-nm-${{ runner.os }}-node22-${{ hashFiles('apps/mobile-app/pnpm-lock.yaml') }} diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000..8b43457 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,17 @@ +> **Rules.** Before merge, run what this change needs on its exact head (docs/CI.md, "What to run"): for Go, `make lint test` scoped with `LINT_DIRS`/`TEST_PKGS` (`scripts/affected.sh` prints them) or `make check-fast`, plus the checks of what you touched. Then `make ci-local RAN=""` posts `local-ci` and fills "Local verification" with that statement and the head SHA. Never edit that section by hand. The `Local verification` check, `make pr-merge` and the staging gate refuse the PR while it is missing, names another head, or differs from what `local-ci` attested. A new push or rebase needs a new attestation. A reported failure (`FAILED=`) is final for that head: fix it in a new commit (docs/CI.md, "Local verification"). + +## Problem + + + +## Fix + + + +## Local verification + + + +Not attested yet: run what the change needs, then `make ci-local RAN=""` on the pushed head. + + diff --git a/.github/scripts/deploy-railway.sh b/.github/scripts/deploy-railway.sh new file mode 100644 index 0000000..a22b5b4 --- /dev/null +++ b/.github/scripts/deploy-railway.sh @@ -0,0 +1,265 @@ +#!/bin/sh +# Deploys the checked-out commit to one Railway environment and proves it +# healthy (docs/RELEASES.md). Used by "Deploy staging" and "Deploy production"; +# the workflow checks out the commit and holds the token. +# +# sh .github/scripts/deploy-railway.sh +# +# The services and their order come from deploy/platform.yaml through +# deploy/cmd/deploy-services: web, then admin when admin is on, then the +# backup cron when backups are on. Each is uploaded with `railway up` (Railway +# builds its Dockerfile from this checkout; the deployment message names the +# commit) and waited for before the next one starts. There is no separate +# migrate step: each service's Railway pre-deploy command migrates its +# database (/server migrate for web, /admin migrate for admin), and /healthz +# gates the deployment, so a failed migration fails that service's deployment +# and stops the rollout. Once every service is up, the public /healthz +# endpoints must answer 200 (retried for a few minutes). +# +# RAILWAY_TOKEN Railway project token scoped to . A project +# token already names its project and environment, so no +# --environment flag is passed (with one, the CLI cannot +# look environments up and answers "Environment not +# found"). Without a token (a person signed in with +# `railway login`, in a linked checkout) --environment +# is added, and --project when +# RAILWAY_PROJECT_ID is set. +# WEB_URL public web origin; /healthz must answer 200 +# ADMIN_URL optional public admin origin; checked when set +# DEPLOY_SERVICES optional override of the deploy-services list +# DEPLOY_TIMEOUT seconds to wait for each deployment (default 1800) +# READY_ATTEMPTS, READY_INTERVAL readiness retries (default 6 x 30s) +# DRY_RUN=1 print every railway and curl command instead of running +# it: what the workflows do until RAILWAY_DEPLOY is true +# +# Never pass a variable named RAILWAY_ENV: it switches the Railway CLI to +# Railway's internal API (docs/railway-go-iac-issues.md). +set -eu + +usage="usage: sh .github/scripts/deploy-railway.sh " +environment=${1:?$usage} +sha=${2:?$usage} +dry=false +[ "${DRY_RUN:-}" = 1 ] && dry=true + +root=$(cd "$(dirname "$0")/../.." && pwd) +cd "$root" + +if [ "$dry" = true ]; then + echo "DRY RUN: printing the railway and curl commands instead of running them." +fi + +# The upload is this checkout, so it must be the commit being deployed. +head=$(git rev-parse HEAD) +if [ "$head" != "$sha" ]; then + if [ "$dry" = true ]; then + echo "::warning::the checkout is $head, not $sha; a real deploy refuses this" + else + echo "::error::the checkout is $head, not $sha; nothing was deployed" + exit 1 + fi +fi + +if [ -z "${RAILWAY_TOKEN:-}" ] && [ -n "${GITHUB_ACTIONS:-}" ] && [ "$dry" = false ]; then + upper=$(echo "$environment" | tr '[:lower:]' '[:upper:]') + echo "::error::RAILWAY_TOKEN is empty. Add a Railway project token scoped to $environment as the RAILWAY_TOKEN secret of the $environment GitHub environment (docs/RELEASES.md); nothing was deployed" + exit 1 +fi + +target="" +if [ -z "${RAILWAY_TOKEN:-}" ] && [ "$dry" = false ]; then + target="--environment $environment" + [ -n "${RAILWAY_PROJECT_ID:-}" ] && target="$target --project $RAILWAY_PROJECT_ID" +fi + +services=${DEPLOY_SERVICES:-$(cd deploy && go run ./cmd/deploy-services -config platform.yaml)} +if [ -z "$services" ]; then + echo "::error::deploy-services listed no services for $environment" + exit 1 +fi + +timeout=${DEPLOY_TIMEOUT:-1800} +poll=15 + +# show CMD...: the command as it would run. +show() { + printf '+' + for arg in "$@"; do + case $arg in + *[!A-Za-z0-9_./:=@-]* | '') printf " '%s'" "$arg" ;; + *) printf ' %s' "$arg" ;; + esac + done + printf '\n' +} + +# newest_id SERVICE: the id of the service's newest deployment ("" when none +# or unreadable). +newest_id() { + # shellcheck disable=SC2086 + railway deployment list $target --service "$1" --json 2>/dev/null | jq -r '.[0].id // ""' 2>/dev/null || true +} + +# upload SERVICE: starts the deployment and prints the CLI's output. An +# upload can fail transiently at Railway's edge, so one answering a 5xx is +# retried twice with a pause; any other failure stops at once. +upload() { + attempt=1 + while :; do + # shellcheck disable=SC2086 + if out=$(railway up $target --service "$1" --detach --verbose -m "deploy $sha" 2>&1); then + echo "$out" + return 0 + fi + echo "$out" >&2 + if [ "$attempt" -lt 3 ] && echo "$out" | grep -q 'status code 5[0-9][0-9]'; then + echo "Upload for $1 failed, retrying in $((attempt * 20))s" >&2 + sleep $((attempt * 20)) + attempt=$((attempt + 1)) + continue + fi + echo "::error::upload for $1 failed" >&2 + return 1 + done +} + +# wait_for SERVICE ID: polls the deployment until it settles. +wait_for() { + svc=$1 + id=$2 + waited=0 + while [ "$waited" -lt "$timeout" ]; do + # A read that fails or is not JSON says nothing about the deployment, so + # the next poll tries again. + # shellcheck disable=SC2086 + status=$(railway deployment list $target --service "$svc" --json 2>/dev/null | + jq -r --arg id "$id" '(map(select(.id == $id))[0].status) // "UNKNOWN"' 2>/dev/null) || status=UNREAD + case $status in + SUCCESS) + echo "$svc $id SUCCESS" + return 0 + ;; + SLEEPING) + # The backup cron sleeps between runs once deployed. + if [ "$svc" = backup ]; then + echo "$svc $id SLEEPING (a cron between runs)" + return 0 + fi + ;; + FAILED | CRASHED | REMOVED | SKIPPED) + echo "::error::$svc deployment $id in $environment ended $status (railway logs --service $svc --deployment $id)" + return 1 + ;; + UNREAD) echo "::warning::could not read $svc's deployment status; retrying" ;; + esac + sleep "$poll" + waited=$((waited + poll)) + done + echo "::error::$svc deployment $id in $environment did not finish within ${timeout}s" + return 1 +} + +# deploy SERVICE: upload, find the new deployment's id, wait for it. +deploy() { + svc=$1 + case $svc in + web) echo "== $svc: Railway runs /server migrate as its pre-deploy command; a failed migration fails this deployment" ;; + admin) echo "== $svc: Railway runs /admin migrate as its pre-deploy command; a failed migration fails this deployment" ;; + *) echo "== $svc" ;; + esac + if [ "$dry" = true ]; then + # shellcheck disable=SC2086 + show railway up $target --service "$svc" --detach --verbose -m "deploy $sha" + # shellcheck disable=SC2086 + show railway deployment list $target --service "$svc" --json + echo " (polled every ${poll}s until the new deployment is SUCCESS$([ "$svc" = backup ] && echo ' or SLEEPING'), for up to ${timeout}s)" + return 0 + fi + before=$(newest_id "$svc") + out=$(upload "$svc") || return 1 + # The CLI names the deployment in its output (id=, or "id" in JSON). + id=$(echo "$out" | grep -oE 'id=[0-9a-f-]{36}' | head -1 | cut -d= -f2) || true + [ -n "$id" ] || id=$(echo "$out" | grep -oE '"(deployment)?[iI]d" *: *"[0-9a-f-]{36}"' | head -1 | grep -oE '[0-9a-f-]{36}') || true + if [ -z "$id" ]; then + # Otherwise the deployment is the newest one that was not there before. + tries=0 + while [ "$tries" -lt 20 ]; do + id=$(newest_id "$svc") + [ -n "$id" ] && [ "$id" != "$before" ] && break + id="" + tries=$((tries + 1)) + sleep 3 + done + fi + if [ -z "$id" ]; then + echo "$out" + echo "::error::cannot find the deployment railway up started for $svc" + return 1 + fi + echo "$svc started $id" + wait_for "$svc" "$id" +} + +# ready URL: GET URL until it answers 200 with {"data":{"status":"ok"}} +# (the /healthz body), retried because readiness can lag a successful +# deployment. +ready() { + if [ "$dry" = true ]; then + show curl -s -m 10 -w '\n%{http_code}' "$1" + echo " (retried up to ${READY_ATTEMPTS:-6} times, ${READY_INTERVAL:-30}s apart, until 200 and data.status ok)" + return 0 + fi + attempts=${READY_ATTEMPTS:-6} + interval=${READY_INTERVAL:-30} + attempt=1 + while :; do + answer=$(curl -s -m 10 -w '\n%{http_code}' "$1") || true + code=$(printf '%s\n' "$answer" | tail -n 1) + status=$(printf '%s\n' "$answer" | sed '$d' | jq -r '.data.status // empty' 2>/dev/null) || true + if [ "$code" = 200 ] && [ "$status" = ok ]; then + echo "$1 200 ok" + return 0 + fi + [ "$code" = 200 ] && code="200 with data.status '${status}'" + if [ "$attempt" -ge "$attempts" ]; then + echo "::error::$1 answered $code after $attempts attempts" + return 1 + fi + echo "$1 answered $code; retrying in ${interval}s ($attempt/$attempts)" + attempt=$((attempt + 1)) + sleep "$interval" + done +} + +upper=$(echo "$environment" | tr '[:lower:]' '[:upper:]') +echo "Deploying $(git log --oneline -1 "$sha" 2>/dev/null || echo "$sha") to Railway $environment" +echo "Services, in order: $services" +if [ "$dry" = false ]; then + echo "Railway CLI: $(railway --version)" +fi + +for svc in $services; do + deploy "$svc" || { + echo "::error::$environment deploy stopped at $svc; the services after it were not deployed" + exit 1 + } +done + +echo "== readiness" +if [ -n "${WEB_URL:-}" ]; then + ready "${WEB_URL%/}/healthz" +elif [ "$dry" = true ]; then + echo "::warning::WEB_URL is empty (the ${upper}_WEB_URL repository variable); a real deploy fails here" + ready "<${upper}_WEB_URL>/healthz" +else + echo "::error::WEB_URL is empty: set the ${upper}_WEB_URL repository variable to $environment's web origin" + exit 1 +fi +if [ -n "${ADMIN_URL:-}" ]; then + ready "${ADMIN_URL%/}/healthz" +else + case " $services " in + *" admin "*) echo "::warning::ADMIN_URL is empty (the ${upper}_ADMIN_URL repository variable); admin's /healthz not checked" ;; + esac +fi +echo "$environment runs $sha" diff --git a/.github/scripts/production-gate.sh b/.github/scripts/production-gate.sh new file mode 100644 index 0000000..3bfd355 --- /dev/null +++ b/.github/scripts/production-gate.sh @@ -0,0 +1,147 @@ +#!/bin/sh +# Decides whether a release may go to production (docs/RELEASES.md). Every +# check runs and every failure is reported, then the script exits 1 if any +# failed, so the person releasing sees the whole picture. Anything it cannot +# prove counts as a failure (fail closed). It reads GitHub and staging's +# /healthz only; it never deploys. +# +# TAG release tag, vX.Y.Z +# SHA commit the tag points at (resolved by the workflow) +# ACTOR GitHub user who published or dispatched +# RELEASERS PRODUCTION_RELEASERS repo variable: logins allowed to release, +# separated by commas or spaces +# STAGING_WEB_URL staging's web origin (repo variable) +# GH_TOKEN, GITHUB_REPOSITORY +# +# Needs a checkout with the full history of origin/main. Read-only from a +# laptop signed in with gh (RELEASERS, STAGING_WEB_URL and the repository are +# read with gh when unset; see release-lib.sh): +# +# git fetch origin main --tags +# TAG=v1.2.3 SHA=$(git rev-parse 'v1.2.3^{commit}') ACTOR=$(gh api user -q .login) \ +# sh .github/scripts/production-gate.sh +set -u + +: "${TAG:?set TAG (vX.Y.Z)}" "${SHA:?set SHA (the commit TAG points at)}" "${ACTOR:?set ACTOR (a GitHub login)}" + +# shellcheck source=.github/scripts/release-lib.sh +. "$(dirname "$0")/release-lib.sh" +R=$GITHUB_REPOSITORY + +failures=0 +report() { + echo "$1" + [ -n "${GITHUB_STEP_SUMMARY:-}" ] && echo "- $1" >>"$GITHUB_STEP_SUMMARY" + return 0 +} +pass() { report "PASS: $1"; } +fail() { + report "FAIL: $1" + # The annotation repeats the line; only Actions shows it. + [ -n "${GITHUB_ACTIONS:-}" ] && echo "::error::$1" + failures=$((failures + 1)) +} + +[ -n "${GITHUB_STEP_SUMMARY:-}" ] && printf '## Production gate for %s (%s)\n\n' "$TAG" "$SHA" >>"$GITHUB_STEP_SUMMARY" + +# 1. The deploy environments must exist and admit only the refs that may +# deploy: production main and v* tags, staging main. GitHub creates a missing +# environment on first use with no restrictions, so without this check the +# first deploy would run in an unconfigured one. +check_environment() { + environment=$1 + want=$2 + if env_json=$(gh api "repos/$R/environments/$environment" 2>/dev/null) && + policies=$(gh api "repos/$R/environments/$environment/deployment-branch-policies" \ + -q '[.branch_policies[] | "\(.type):\(.name)"] | sort | join(" ")' 2>/dev/null); then + if [ "$(echo "$env_json" | jq -r '.deployment_branch_policy.custom_branch_policies // false')" = true ] && + [ "$policies" = "$want" ]; then + pass "The $environment environment admits only $want" + else + fail "The $environment environment must admit only $want (found: ${policies:-none}). Run sh .github/scripts/release-settings.sh (docs/RELEASES.md, one-time setup)." + fi + else + fail "The $environment environment does not exist or cannot be read. Run sh .github/scripts/release-settings.sh (docs/RELEASES.md, one-time setup)." + fi +} +check_environment production "branch:main tag:v*" +check_environment staging "branch:main" + +# 1b. Only releasers may create, move or delete a v* tag. The production +# environment admits v* tags, and a tag runs the workflow file at its own +# commit, so without this ruleset anyone who can push could tag a commit whose +# workflow skips this gate and read the production token. +protected=false +for id in $(gh api "repos/$R/rulesets?includes_parents=false" -q '.[] | select(.target == "tag" and .enforcement == "active") | .id' 2>/dev/null); do + if gh api "repos/$R/rulesets/$id" -q '[.conditions.ref_name.include[]] | index("refs/tags/v*") != null' 2>/dev/null | grep -qx true && + [ "$(gh api "repos/$R/rulesets/$id" -q '[.rules[].type | select(. == "creation" or . == "update" or . == "deletion")] | unique | length' 2>/dev/null)" = 3 ]; then + protected=true + fi +done +if [ "$protected" = true ]; then + pass "An active tag ruleset restricts creating, moving and deleting v* tags" +else + fail "No active tag ruleset restricts creating, moving and deleting v* tags. Run sh .github/scripts/release-settings.sh (docs/RELEASES.md, one-time setup)." +fi + +# 2. Only allowed people release. +if [ -z "${RELEASERS:-}" ]; then + fail "The PRODUCTION_RELEASERS repository variable is not set." +elif is_releaser "$ACTOR"; then + pass "$ACTOR may release" +else + fail "$ACTOR is not in PRODUCTION_RELEASERS." +fi + +# 3. A published, final release for a vX.Y.Z tag. +if echo "$TAG" | grep -qE '^v[0-9]+\.[0-9]+\.[0-9]+$'; then + state=$(gh release view "$TAG" -R "$R" --json isDraft,isPrerelease -q '"\(.isDraft) \(.isPrerelease)"' 2>/dev/null || echo missing) + case $state in + "false false") pass "$TAG is a published release" ;; + missing) fail "No release exists for $TAG." ;; + *) fail "$TAG is a draft or a pre-release (draft, pre-release: $state)." ;; + esac +else + fail "$TAG is not a vX.Y.Z tag." +fi + +# 4. The commit is on main: no branch builds, no hotfixes past staging. +if git merge-base --is-ancestor "$SHA" origin/main 2>/dev/null; then + pass "$SHA is on main" +else + fail "$SHA is not on main. Production only takes commits merged to main." +fi + +# 5. Staging ran this exact commit and passed its readiness check. A dry run +# (RAILWAY_DEPLOY not true) deploys nothing, so it does not count. +proved=$(staging_ran "$SHA") +if [ -n "$proved" ]; then + pass "Staging ran $SHA and passed readiness (run $proved)" +else + fail "No successful Deploy staging run, with its \"$STAGING_JOB\" job succeeded, deployed $SHA. Production only takes a commit staging has run." +fi + +# 6. Staging still runs this commit or a later main commit, so production never +# leads staging. +current=$(staging_commit) +if [ -z "$current" ]; then + fail "Cannot tell which commit staging runs: no successful Deploy staging run whose \"$STAGING_JOB\" job succeeded." +elif git merge-base --is-ancestor "$SHA" "$current" 2>/dev/null; then + pass "Staging runs $current, which includes $SHA" +else + fail "Staging runs $current, which does not include $SHA. Production may lag staging, never lead it." +fi + +# 7. Staging is healthy now. +code=$(staging_ready) +if [ "$code" = 200 ]; then + pass "Staging is ready ($STAGING_READY answered 200)" +else + fail "$(staging_not_ready "$code")" +fi + +if [ "$failures" -gt 0 ]; then + echo "${GITHUB_ACTIONS:+::error::}Production gate refused $TAG: $failures check(s) failed." + exit 1 +fi +echo "Production gate passed for $TAG ($SHA)." diff --git a/.github/scripts/release-lib.sh b/.github/scripts/release-lib.sh new file mode 100644 index 0000000..5eaf343 --- /dev/null +++ b/.github/scripts/release-lib.sh @@ -0,0 +1,80 @@ +#!/bin/sh +# Shared checks for "Deploy production" and "Draft release" (docs/RELEASES.md). +# Source it; it only reads GitHub (gh) and staging's /healthz. +# +# In a workflow it needs GH_TOKEN, GITHUB_REPOSITORY, and the repository +# variables passed in as RELEASERS (PRODUCTION_RELEASERS) and STAGING_WEB_URL. +# On a laptop, signed in with `gh auth login`, the repository and both +# variables are read with gh when unset (reading variables needs write access +# to the repository): +# +# . .github/scripts/release-lib.sh +# staging_commit; staging_ready; is_releaser "$(gh api user -q .login)" && echo yes + +if [ -z "${GITHUB_ACTIONS:-}" ]; then + : "${GITHUB_REPOSITORY:=$(gh repo view --json nameWithOwner -q .nameWithOwner)}" + : "${RELEASERS=$(gh variable get PRODUCTION_RELEASERS -R "$GITHUB_REPOSITORY" 2>/dev/null || true)}" + : "${STAGING_WEB_URL=$(gh variable get STAGING_WEB_URL -R "$GITHUB_REPOSITORY" 2>/dev/null || true)}" +fi + +STAGING_WORKFLOW=deploy-staging.yml +STAGING_JOB='Deploy main to Railway staging' +STAGING_READY=${STAGING_WEB_URL:+${STAGING_WEB_URL%/}/healthz} + +# is_releaser LOGIN: LOGIN is listed in PRODUCTION_RELEASERS (the repository +# variable, passed in as RELEASERS; logins separated by commas or spaces). +is_releaser() { + [ -n "${RELEASERS:-}" ] && echo "$RELEASERS" | tr ',' ' ' | tr -s ' ' '\n' | grep -qxF "$1" +} + +# staging_run_ok RUN: a successful "Deploy staging" run proves staging ran a +# commit only when its deploy job (which ends with the readiness check) +# succeeded. The run title names the deployed commit. A dry run (RAILWAY_DEPLOY +# not true) skips that job, so it proves nothing. +staging_run_ok() { + [ "$(gh api "repos/$GITHUB_REPOSITORY/actions/runs/$1/jobs" \ + -q "[.jobs[] | select(.name == \"$STAGING_JOB\" and .conclusion == \"success\")] | length" 2>/dev/null)" = 1 ] +} + +# staging_ran SHA: prints the id of a run that deployed SHA to staging. +staging_ran() { + for run in $(gh api "repos/$GITHUB_REPOSITORY/actions/workflows/$STAGING_WORKFLOW/runs?status=success&head_sha=$1&per_page=20" \ + -q ".workflow_runs[] | select(.display_title == \"Deploy staging $1\") | .id" 2>/dev/null); do + if staging_run_ok "$run"; then + echo "$run" + return 0 + fi + done + return 1 +} + +# staging_commit: prints the commit staging runs now (its latest proven deploy). +staging_commit() { + for line in $(gh api "repos/$GITHUB_REPOSITORY/actions/workflows/$STAGING_WORKFLOW/runs?status=success&per_page=20" \ + -q '.workflow_runs[] | "\(.id):\(.display_title | ltrimstr("Deploy staging "))"' 2>/dev/null); do + if staging_run_ok "${line%%:*}"; then + echo "${line#*:}" + return 0 + fi + done + return 1 +} + +# staging_ready: prints the HTTP status of staging's /healthz (000 when it +# does not answer), or "unset" when STAGING_WEB_URL is empty. +staging_ready() { + if [ -z "$STAGING_READY" ]; then + echo unset + return 0 + fi + curl -s -o /dev/null -m 20 -w '%{http_code}' "$STAGING_READY" || true +} + +# staging_not_ready CODE: why staging_ready's CODE is not a pass. +staging_not_ready() { + if [ "$1" = unset ]; then + echo "The STAGING_WEB_URL repository variable is not set, so staging's /healthz cannot be checked." + else + echo "Staging is not ready ($STAGING_READY answered $1)." + fi +} diff --git a/.github/scripts/release-settings.sh b/.github/scripts/release-settings.sh new file mode 100644 index 0000000..9d70424 --- /dev/null +++ b/.github/scripts/release-settings.sh @@ -0,0 +1,52 @@ +#!/bin/sh +# One-time setup, run by hand (never by a workflow): applies the repository +# settings that keep the Railway tokens away from pushed workflows +# (docs/RELEASES.md): the staging and production deployment environments and +# the refs each admits, and the tag ruleset that lets only administrators +# create, move or delete a v* tag. production-gate.sh refuses every release +# until these hold. Idempotent; run it with an administrator's gh login. It +# never reads or writes a secret: add each environment's +# RAILWAY_TOKEN secret yourself. +# +# sh .github/scripts/release-settings.sh [owner/repo] # default: this checkout's GitHub repo +set -eu +repo=${1:-$(gh repo view --json nameWithOwner -q .nameWithOwner)} +R=repos/$repo + +# environment NAME REF...: NAME admits exactly the given refs (branch:main, tag:v*). +environment() { + name=$1 + shift + gh api -X PUT "$R/environments/$name" --silent --input - <<'JSON' +{"deployment_branch_policy":{"protected_branches":false,"custom_branch_policies":true}} +JSON + existing=$(gh api "$R/environments/$name/deployment-branch-policies" -q '.branch_policies[] | "\(.id) \(.type):\(.name)"') + echo "$existing" | while read -r id ref; do + [ -n "$id" ] || continue + case " $* " in *" $ref "*) ;; *) gh api -X DELETE "$R/environments/$name/deployment-branch-policies/$id" --silent ;; esac + done + for ref in "$@"; do + echo "$existing" | awk -v ref="$ref" '$2 == ref { found = 1 } END { exit !found }' && continue + gh api -X POST "$R/environments/$name/deployment-branch-policies" -f "type=${ref%%:*}" -f "name=${ref#*:}" --silent + done + echo "$name admits: $(gh api "$R/environments/$name/deployment-branch-policies" -q '[.branch_policies[] | "\(.type):\(.name)"] | sort | join(" ")')" +} + +environment production branch:main tag:v* +environment staging branch:main + +ruleset='{ + "name": "Release tags", + "target": "tag", + "enforcement": "active", + "conditions": { "ref_name": { "include": ["refs/tags/v*"], "exclude": [] } }, + "rules": [{ "type": "creation" }, { "type": "update" }, { "type": "deletion" }, { "type": "non_fast_forward" }], + "bypass_actors": [{ "actor_id": 5, "actor_type": "RepositoryRole", "bypass_mode": "always" }] +}' +id=$(gh api "$R/rulesets" -q '.[] | select(.name == "Release tags") | .id') +if [ -n "$id" ]; then + echo "$ruleset" | gh api -X PUT "$R/rulesets/$id" --silent --input - +else + echo "$ruleset" | gh api -X POST "$R/rulesets" --silent --input - +fi +echo "Tag ruleset 'Release tags' protects refs/tags/v* in $repo (bypass: the Admin role)." diff --git a/.github/workflows/bench-devloop.yml b/.github/workflows/bench-devloop.yml index 50fbb71..c9319c2 100644 --- a/.github/workflows/bench-devloop.yml +++ b/.github/workflows/bench-devloop.yml @@ -4,15 +4,14 @@ # the same machine (separate runs land on different machines; their noise # hides changes of tens of percent). The report is the run's summary page. # Run it by hand (Actions > Bench dev loop), or push a branch to bench/ -# to compare it with the branch work lands on (dev, or main without one: -# scripts/branches.sh). +# to compare it with the branch work lands on (main: scripts/branches.sh). name: Bench dev loop on: workflow_dispatch: inputs: base: - description: Ref to compare against (A/B of the test run on the same machine; empty = origin/dev, else origin/main) + description: Ref to compare against (A/B of the test run on the same machine; empty = origin/main) default: '' rounds: description: Alternating A/B rounds (the median is compared) @@ -36,7 +35,7 @@ jobs: BENCH_BASE: ${{ inputs.base }} BENCH_ROUNDS: ${{ inputs.rounds || '3' }} steps: - # Same Postgres as CI's db shard. + # A throwaway Postgres on tmpfs, as make test's. - name: Start Postgres run: | docker run -d --name pg -p 5432:5432 -e POSTGRES_PASSWORD=postgres \ @@ -45,8 +44,9 @@ jobs: - uses: actions/checkout@v7 with: fetch-depth: 0 # the A/B base ref - # Read-only: the module cache CI saves. Build and lint caches start - # empty on purpose; the bench measures cold and warm itself. + # Read-only module cache (nothing saves it since PR checks moved to the + # agent's machine; a miss downloads). Build and lint caches start empty on + # purpose; the bench measures cold and warm itself. - uses: actions/cache/restore@v6 with: path: ~/go/pkg/mod diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index fadea85..d5836aa 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,432 +1,129 @@ -# ci-ok, the single required check, is a commit status, not a job: jobs post -# it (pending, then success/failure) on the commit they actually tested, so a -# skipped or cancelled job can never satisfy it and a draft (which starts no -# jobs) never gets it. Jobs bill rounded up to whole minutes, so CI is ONE job: -# it checks the affected packages (all of them on pushes to dev and main, the -# merge queue and manual runs) plus the mobile app and deploy/ module when -# those change. Sharding is -# an opt-in fallback only: a "ci:fanout" label on the PR or the fanout input -# of a manual run. +# Pull requests only: the agent runs what its change needs before merge and +# attests it with `make ci-local RAN=""` (docs/CI.md, "Local +# verification"), and main runs no CI. This keeps the two small checks the +# main ruleset requires: the secret scan, and the check that make ci-local's +# attestation holds for the head. A push to main is gated separately: Deploy +# staging deploys only a commit make ci-local attested (deploy-staging.yml). +# +# Trusted code only (docs/CI.md, "What runs"): pull_request_target runs this +# file and checks out the code as they are on the base branch, never the pull +# request's. The jobs read the head as data, through the API and git's +# objects, and never run it, so a branch or a fork that rewrites +# scripts/cilocal, .gitleaks.toml or this workflow changes nothing they do +# until it merges. A fork's head fails Local verification. Never add a +# checkout of the head, or an expression inside a run script +# (verify.TrustProblems, tested in scripts/cilocal/verify). +# +# Runner: the CI_RUNNER repository variable, a JSON runs-on value such as +# '["self-hosted","linux"]'; GitHub-hosted ubuntu-latest without it. name: CI -on: - pull_request: - types: [opened, synchronize, reopened, ready_for_review] - workflow_dispatch: - inputs: - fanout: - description: Split the checks across two parallel jobs (faster wall time, more billed minutes) - type: boolean - default: false - # The deployed branches (scripts/branches.sh): dev deploys staging, main - # production. A repo without dev just never pushes it. PRs run whatever - # branch they target. - push: - branches: [dev, main] - merge_group: - # Refreshes the caches PRs restore (pushes that reuse a PR's result do not); - # runs on the default branch, skipped when it has not moved since the last - # saved cache. - schedule: - - cron: '17 */4 * * *' - -# PR runs: draft and real runs use separate groups, so a skipped draft run -# (e.g. a push landing a second before "ready for review") never cancels the -# real run, and a newer push cancels the older real run; a cancelled run's -# ci-ok still runs and fails, so an old SHA never keeps a green required check. -# Everything else (pushes to dev/main, merge queue, manual runs) gets a group -# of its own: a shared group made pushes queue behind each other, and GitHub -# cancels a pending run when a newer one joins its group. -concurrency: - group: ${{ github.event_name == 'pull_request' && format('ci-{0}-{1}-{2}', github.workflow, github.ref, github.event.pull_request.draft && 'draft' || 'run') || format('ci-{0}-{1}', github.workflow, github.run_id) }} - cancel-in-progress: ${{ github.event_name == 'pull_request' }} - +# Every job reads only, unless it asks for more itself. permissions: contents: read - pull-requests: read # paths-filter lists PR files through the API - statuses: write # the ci-ok commit status - actions: read # the opt-in fan-out finaliser reads the other job's result; scheduled runs list caches -env: - TEST_DATABASE_URL: postgres://postgres:postgres@127.0.0.1:5432/postgres?sslmode=disable - # A stalled cache download falls back to a cache miss after this long instead - # of hanging (default 10 min); restores take 2-12 s. - SEGMENT_DOWNLOAD_TIMEOUT_MINS: 1 +# A newer edit of a pull request cancels the run it supersedes. +concurrency: + group: ci-pr-${{ github.event.pull_request.number }} + cancel-in-progress: true + +on: + pull_request_target: + # Edits only: make ci-local posts local-ci, then fills the PR body, and + # that edit starts the run that checks both (docs/CI.md, "Local + # verification"). A push or a new PR starts nothing, and an edit before + # make ci-local claims the head runs Local verification under another + # name, so until it attests the required checks wait as "Expected", + # never red; a job skipped by `if` would count as passed. + types: [edited] jobs: - # Does all the work (unless fan-out was asked for). Drafts start no jobs: - # ready for review (or any push to a ready PR) runs CI. - ci: - if: github.event_name != 'pull_request' || !github.event.pull_request.draft - runs-on: ubuntu-latest - # The ~2x-average timeouts sit on the steps that do the work (see - # "Timeouts" below): the job cannot know at start whether a run is full or - # cold. This is only a backstop just above the worst case (all packages, - # cold: ~5.5 min job, Check step capped at 11). - timeout-minutes: 13 - outputs: - fanout: ${{ steps.plan.outputs.fanout }} - timeout: ${{ steps.timeouts.outputs.fanout }} - poll: ${{ steps.timeouts.outputs.poll }} - target: ${{ steps.target.outputs.sha }} - merge: ${{ steps.target.outputs.merge }} - tree: ${{ steps.target.outputs.tree }} - packages: ${{ steps.affected.outputs.packages }} - dirs: ${{ steps.affected.outputs.dirs }} - full: ${{ steps.affected.outputs.full }} - sqlc: ${{ steps.affected.outputs.sqlc }} - templ: ${{ steps.affected.outputs.templ }} - generate: ${{ steps.affected.outputs.generate }} - smoke: ${{ steps.affected.outputs.smoke }} - mobile: ${{ steps.scope.outputs.mobile }} - deploy: ${{ steps.scope.outputs.deploy }} + secrets: + name: Secret scan + # The pull request's changed files, docs included (docs/CI.md, "Secret + # scan"), read from the head's objects and scanned with the base's + # scanner and policy (.gitleaks.toml, .gitleaksignore); a head that edits + # that policy fails for review. It does not depend on the agent's machine. + runs-on: ${{ fromJSON(vars.CI_RUNNER || '"ubuntu-latest"') }} + timeout-minutes: 10 + permissions: + contents: read + pull-requests: read steps: - # PR events list changed files through the API (no checkout needed). - - uses: dorny/paths-filter@v4 - if: github.event_name == 'pull_request' - id: go-pr - with: - predicate-quantifier: every - filters: | - go: - - '**' - - '!docs/**' - - '!apps/mobile-app/**' - - '!deploy/**' # its own module: the deploy checks - - '!*.md' - - '!.gitignore' - - '!.dockerignore' - - uses: dorny/paths-filter@v4 - if: github.event_name == 'pull_request' - id: mobile-pr - with: - filters: &mobile-filter | - mobile: - - 'apps/mobile-app/**' - # The mobile route test reads the Go route table. - - 'apps/web/internal/config/routes.go' - # How CI runs it. - - '.github/workflows/ci.yml' - - '.github/actions/check/**' - - 'scripts/ci-check.sh' - - uses: dorny/paths-filter@v4 - if: github.event_name == 'pull_request' - id: deploy-pr - with: - filters: &deploy-filter | - deploy: - - 'deploy/**' - - '.env.example' # make deploy-vars reads it - - 'Makefile' - - '.golangci.yml' - - 'tools/**' # the golangci-lint pin - - '.github/workflows/ci.yml' - - '.github/actions/check/**' - - 'scripts/ci-check.sh' - - # Depth 2: a PR checkout is the merge commit (HEAD^1 = base, HEAD^2 = head). - uses: actions/checkout@v7 + # The tool imports only the standard library: no module downloads. + - uses: actions/setup-go@v6 with: - fetch-depth: 2 - - # A push to dev or main whose tree is identical to a PR merge commit that - # already passed CI is not tested again: ci-ok is posted from that - # result. Safe by construction: the PR's ci-ok must be a success posted by - # this workflow (github-actions[bot]) recording exactly this tree (see - # "ci-ok result"); if the branch moved in between (tree differs) or the result - # is missing or not success, the run checks everything as usual. - # A scheduled run is skipped when the cache for this commit already exists. - - name: Reuse or skip - id: reuse - if: github.event_name == 'push' || github.event_name == 'schedule' + go-version-file: go.mod + cache: false + - name: No secrets in the changed files env: GH_TOKEN: ${{ github.token }} - EVENT: ${{ github.event_name }} - MESSAGE: ${{ github.event.head_commit.message }} - run: ./scripts/ci-reuse.sh - - - uses: dorny/paths-filter@v4 - if: github.event_name == 'push' - id: mobile-push - with: - filters: *mobile-filter - - uses: dorny/paths-filter@v4 - if: github.event_name == 'push' - id: deploy-push - with: - filters: *deploy-filter - - - name: Scope - id: scope - env: - EVENT: ${{ github.event_name }} - GO_PR: ${{ steps.go-pr.outputs.go }} - MOBILE: ${{ steps.mobile-pr.outputs.mobile || steps.mobile-push.outputs.mobile }} - DEPLOY: ${{ steps.deploy-pr.outputs.deploy || steps.deploy-push.outputs.deploy }} - SKIP: ${{ steps.reuse.outputs.skip }} - run: | - go=true mobile=$MOBILE deploy=$DEPLOY - # PRs and pushes check mobile/ and deploy/ only when they (or how CI - # runs them) changed; the merge queue, manual and scheduled runs always. - if [ "$EVENT" = pull_request ]; then go=$GO_PR; fi - case $EVENT in merge_group | workflow_dispatch | schedule) mobile=true deploy=true ;; esac - if [ "$SKIP" = true ]; then go=false mobile=false deploy=false; fi - save=false - # Pushes only trigger on the deployed branches (dev, main). - case $EVENT in workflow_dispatch | schedule | push) save=true ;; esac - echo "go=$go" >> "$GITHUB_OUTPUT"; echo "mobile=${mobile:-false}" >> "$GITHUB_OUTPUT" - echo "deploy=${deploy:-false}" >> "$GITHUB_OUTPUT"; echo "save=$save" >> "$GITHUB_OUTPUT" - echo "skip=${SKIP:-false}" >> "$GITHUB_OUTPUT" - - # The commit ci-ok is posted on: the PR head of the merge commit actually - # checked out (the event's head can be stale when a push and "ready for - # review" land together), else the PR's current head, else github.sha. - - name: Target commit - id: target - env: - GH_TOKEN: ${{ github.token }} - EVENT: ${{ github.event_name }} PR: ${{ github.event.pull_request.number }} - run: | - if [ "$EVENT" != pull_request ]; then - echo "sha=$GITHUB_SHA" >> "$GITHUB_OUTPUT"; echo "merge=$GITHUB_SHA" >> "$GITHUB_OUTPUT" - else - echo "sha=$(git rev-parse HEAD^2)" >> "$GITHUB_OUTPUT"; echo "merge=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT" - fi - # The tree tested (the merge commit's), recorded in ci-ok so a push - # to dev or main with exactly this tree can reuse the result. - echo "tree=$(git rev-parse HEAD^{tree})" >> "$GITHUB_OUTPUT" - - - name: ci-ok pending - if: (steps.scope.outputs.go == 'true' || steps.scope.outputs.mobile == 'true' || steps.scope.outputs.deploy == 'true') - env: - GH_TOKEN: ${{ github.token }} - SHA: ${{ steps.target.outputs.sha }} - run: | - gh api "repos/$GITHUB_REPOSITORY/statuses/$SHA" -f state=pending -f context=ci-ok \ - -f description="running" -f target_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" >/dev/null - - # Modules (and the go.mod toolchain) keyed by go.sum; zips and dependency - # testdata are trimmed before saving (see the fan-out job). - - uses: actions/cache/restore@v6 - if: steps.scope.outputs.go == 'true' - id: mod-cache - timeout-minutes: 1 - with: - path: ~/go/pkg/mod - key: v7-gomod-${{ runner.os }}-${{ hashFiles('go.sum') }} - restore-keys: v7-gomod-${{ runner.os }}- - - # No setup-go: the runner's go switches to the go.mod version itself - # (GOTOOLCHAIN=auto); the toolchain lives in the module cache. - - name: Go toolchain - if: steps.scope.outputs.go == 'true' - # 1-5 s; ~10 s when the toolchain has to be downloaded. - timeout-minutes: 1 - run: go version - - # PRs check only what their changes can affect; shared inputs, pushes to - # dev/main, the merge queue and manual runs check everything. - - name: Affected packages - if: steps.scope.outputs.go == 'true' - id: affected - run: ./scripts/affected.sh HEAD^1 - env: - AFFECTED_FULL: ${{ github.event_name != 'pull_request' && '1' || '' }} - - # Opt-in only: one job is cheapest (jobs bill rounded up to minutes). - - name: Plan - if: steps.affected.outputs.packages != '' - id: plan - env: - FANOUT: ${{ contains(github.event.pull_request.labels.*.name, 'ci:fanout') || inputs.fanout == true }} - run: echo "fanout=$FANOUT" >> "$GITHUB_OUTPUT" - - # Timeouts (minutes) for the Check step (the work), ~2x its measured - # average for the case, by whether the build cache is warm (a lookup-only - # probe, no download) and whether all packages are checked: - # (scripts/ci-check.sh runs the checks in parallel; measured 2026-10-22): - # affected set: warm 13-24 s -> 3; cold ~2.5 min -> 6 - # all packages: warm 36-66 s (with mobile + deploy) -> 5; cold 228 s -> 11 - # mobile/deploy only (no Go packages): ~45 s -> 2 - # opt-in fan-out jobs: warm 59-97 s -> 3; cold 185-233 s -> 8 - # release smoke (in parallel; measured 2026-10-23): 37 s with cold - # release builds, +11 s on the all-packages Check -> +1 either way - - uses: actions/cache/restore@v6 - if: steps.affected.outputs.packages != '' - id: build-cache - timeout-minutes: 1 - with: - lookup-only: true - path: ~/.cache/go-build - key: v7-go-${{ runner.os }}-${{ github.sha }} - restore-keys: v7-go-${{ runner.os }}- - - name: Timeouts - id: timeouts - env: - MATCHED: ${{ steps.build-cache.outputs.cache-matched-key }} - PACKAGES: ${{ steps.affected.outputs.packages }} - FULL: ${{ steps.affected.outputs.full }} - SMOKE: ${{ steps.affected.outputs.smoke }} - run: | - smoke_minutes=1 # the release smoke's release builds and boots - if [ -n "$MATCHED" ]; then - fanout=3; check=3; [ "$FULL" = true ] && check=5 - else - fanout=8; check=6; [ "$FULL" = true ] && check=11 - fi - [ -z "$PACKAGES" ] && check=2 - if [ "$SMOKE" = true ]; then check=$((check + smoke_minutes)) fanout=$((fanout + smoke_minutes)); fi - echo "check=$check" >> "$GITHUB_OUTPUT"; echo "fanout=$fanout" >> "$GITHUB_OUTPUT" - echo "poll=$((fanout + 1))" >> "$GITHUB_OUTPUT" - - - name: Check - if: steps.plan.outputs.fanout != 'true' && (steps.scope.outputs.go == 'true' || steps.scope.outputs.mobile == 'true' || steps.scope.outputs.deploy == 'true') - timeout-minutes: ${{ fromJSON(steps.timeouts.outputs.check || '3') }} - uses: ./.github/actions/check + HEAD_SHA: ${{ github.event.pull_request.head.sha }} + run: go run ./scripts/cilocal secret-scan --pr "$PR" --head "$HEAD_SHA" + + claim: + name: Local verification claim + # Whether make ci-local has claimed this head: posted local-ci on it, or + # filled the section for it (verify.Claimed; a fork's head always is, so + # it fails). Until then the next job runs under another name, so the + # required check waits as "Expected". + runs-on: ${{ fromJSON(vars.CI_RUNNER || '"ubuntu-latest"') }} + timeout-minutes: 5 + permissions: + contents: read + pull-requests: read + statuses: read + outputs: + claimed: ${{ steps.claim.outputs.claimed }} + steps: + - uses: actions/checkout@v7 with: - mode: all - packages: ${{ steps.affected.outputs.packages }} - dirs: ${{ steps.affected.outputs.dirs }} - full: ${{ steps.affected.outputs.full }} - sqlc: ${{ steps.affected.outputs.sqlc }} - templ: ${{ steps.affected.outputs.templ }} - generate: ${{ steps.affected.outputs.generate }} - smoke: ${{ steps.affected.outputs.smoke }} - mobile: ${{ steps.scope.outputs.mobile }} - deploy: ${{ steps.scope.outputs.deploy }} - save: ${{ steps.scope.outputs.save }} - - # This job (lint) loads every package and its test imports, so it saves - # the module cache. Cache-path "!" exclusions did not apply, so trim - # explicitly: zips (~125 MB incl. the 71 MB toolchain zip) and testdata. - - name: Trim module cache - if: steps.scope.outputs.save == 'true' && steps.mod-cache.outcome == 'success' && steps.mod-cache.outputs.cache-hit != 'true' - run: | - chmod -R u+w ~/go/pkg/mod - find ~/go/pkg/mod/cache/download -name '*.zip' -delete - find ~/go/pkg/mod -mindepth 2 -type d -name testdata -prune -exec rm -rf {} + - - uses: actions/cache/save@v6 - if: steps.scope.outputs.save == 'true' && steps.mod-cache.outcome == 'success' && steps.mod-cache.outputs.cache-hit != 'true' + sparse-checkout: scripts/cilocal # and the root files (cone mode) + # The tool imports only the standard library: no module downloads. + - uses: actions/setup-go@v6 with: - path: ~/go/pkg/mod - key: v7-gomod-${{ runner.os }}-${{ hashFiles('go.sum') }} - - # This job's own result (docs-only PRs land here with nothing to check: - # success). With fan-out, the db job posts it instead. - # Last real step: every check ran before it (job.status covers them). The - # job/run can still show "in progress" a few seconds longer (post-step - # cleanup), or minutes when GitHub is slow to process Actions results. - - name: ci-ok result - if: always() && !cancelled() && steps.plan.outputs.fanout != 'true' && steps.target.outputs.sha != '' && steps.scope.outputs.skip != 'true' + go-version-file: go.mod + cache: false + - id: claim + name: Has make ci-local attested this head env: GH_TOKEN: ${{ github.token }} - SHA: ${{ steps.target.outputs.sha }} - TREE: ${{ steps.target.outputs.tree }} - STATUS: ${{ job.status }} - run: | - state=failure; [ "$STATUS" = success ] && state=success - gh api "repos/$GITHUB_REPOSITORY/statuses/$SHA" -f state=$state -f context=ci-ok \ - -f description="$state tree=$TREE" -f target_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" >/dev/null - - # A run cancelled by a newer push can still report job.status success - # (seen on a superseded PR run), so a cancelled run only ever posts failure. - - name: ci-ok cancelled - if: cancelled() && steps.plan.outputs.fanout != 'true' && steps.target.outputs.sha != '' && steps.scope.outputs.skip != 'true' - env: - GH_TOKEN: ${{ github.token }} - SHA: ${{ steps.target.outputs.sha }} - run: | - gh api "repos/$GITHUB_REPOSITORY/statuses/$SHA" -f state=failure -f context=ci-ok \ - -f description="cancelled" -f target_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" >/dev/null - - # Opt-in fallback ("ci:fanout" label, or the fanout input of a manual run): - # two parallel jobs for faster wall time at more billed minutes. "db" = lint - # + DB-backed tests, posts ci-ok once "other" (the rest + mobile/deploy) is done. - fanout: - name: ${{ matrix.mode == 'db' && 'lint + test (db)' || 'test (other)' }} - needs: ci - if: needs.ci.outputs.fanout == 'true' - strategy: - matrix: - mode: [db, other] - runs-on: ubuntu-latest - timeout-minutes: 12 # backstop; the Check step has the measured timeout + REPO: ${{ github.repository }} + PR: ${{ github.event.pull_request.number }} + HEAD_SHA: ${{ github.event.pull_request.head.sha }} + HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name }} + run: go run ./scripts/cilocal check --claim + + local: + # The agent ran the PR's checks on its machine (docs/CI.md, "Local + # verification"). For a claimed head this is the required check, and + # fails unless the local-ci status on the head and the PR body's section + # attest them. For any other it passes under another name and posts + # nothing the ruleset reads. + name: ${{ needs.claim.outputs.claimed == 'true' && 'Local verification' || 'Awaiting make ci-local' }} + needs: claim + runs-on: ${{ fromJSON(vars.CI_RUNNER || '"ubuntu-latest"') }} + timeout-minutes: 5 + permissions: + contents: read + pull-requests: read + statuses: read steps: - # First thing, so it is up before the DB tests run. - - name: Start Postgres - if: matrix.mode == 'db' - run: | - nohup docker run -d --name pg -p 5432:5432 -e POSTGRES_PASSWORD=postgres \ - --tmpfs /var/lib/postgresql postgres:18-alpine \ - -c fsync=off -c synchronous_commit=off -c full_page_writes=off -c max_connections=200 \ - > "$RUNNER_TEMP/pg.log" 2>&1 & - - uses: actions/checkout@v7 with: - ref: ${{ needs.ci.outputs.merge }} - fetch-depth: 2 - - uses: actions/cache/restore@v6 - timeout-minutes: 1 - id: mod-cache + sparse-checkout: scripts/cilocal # and the root files (cone mode) + - uses: actions/setup-go@v6 with: - path: ~/go/pkg/mod - key: v7-gomod-${{ runner.os }}-${{ hashFiles('go.sum') }} - restore-keys: v7-gomod-${{ runner.os }}- - - name: Go toolchain - # 1-5 s; ~10 s when the toolchain has to be downloaded. - timeout-minutes: 1 - run: go version - - - name: Check - timeout-minutes: ${{ fromJSON(needs.ci.outputs.timeout || '8') }} - uses: ./.github/actions/check - with: - mode: ${{ matrix.mode }} - packages: ${{ needs.ci.outputs.packages }} - dirs: ${{ needs.ci.outputs.dirs }} - full: ${{ needs.ci.outputs.full }} - sqlc: ${{ needs.ci.outputs.sqlc }} - templ: ${{ needs.ci.outputs.templ }} - generate: ${{ needs.ci.outputs.generate }} - smoke: ${{ matrix.mode == 'other' && needs.ci.outputs.smoke || 'false' }} - mobile: ${{ matrix.mode == 'other' && needs.ci.outputs.mobile || 'false' }} - deploy: ${{ matrix.mode == 'other' && needs.ci.outputs.deploy || 'false' }} - save: 'false' - - # Waits for "other", then posts ci-ok for both. Two jobs are named - # "test (other)" (this matrix leg; the full-run job): the one that ran is - # the one not skipped. Also runs when this job - # failed or the run was cancelled (posts failure: never green untested). - - name: ci-ok result - if: always() && !cancelled() && matrix.mode == 'db' - timeout-minutes: ${{ fromJSON(needs.ci.outputs.poll || '9') }} - env: - GH_TOKEN: ${{ github.token }} - SHA: ${{ needs.ci.outputs.target }} - TREE: ${{ needs.ci.outputs.tree }} - STATUS: ${{ job.status }} - POLL_MIN: ${{ needs.ci.outputs.poll || '9' }} - run: | - other="" - for _ in $(seq 1 $(( (${POLL_MIN:-9}) * 30 ))); do - other=$(gh api "repos/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID/attempts/$GITHUB_RUN_ATTEMPT/jobs" \ - --jq '[.jobs[] | select(.name == "test (other)" and .conclusion != "skipped")] | if length == 1 and .[0].status == "completed" then .[0].conclusion else "" end') - [ -n "$other" ] && break - sleep 2 - done - echo "db=$STATUS other=${other:-unfinished}" - state=failure; [ "$STATUS" = success ] && [ "$other" = success ] && state=success - gh api "repos/$GITHUB_REPOSITORY/statuses/$SHA" -f state=$state -f context=ci-ok \ - -f description="$state tree=$TREE" -f target_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" >/dev/null - - # A run cancelled by a newer push can still report job.status success - # (seen on a superseded PR run), so a cancelled run only ever posts failure. - - name: ci-ok cancelled - if: cancelled() && matrix.mode == 'db' + go-version-file: go.mod + cache: false + - name: make ci-local attested this head env: GH_TOKEN: ${{ github.token }} - SHA: ${{ needs.ci.outputs.target }} - run: | - gh api "repos/$GITHUB_REPOSITORY/statuses/$SHA" -f state=failure -f context=ci-ok \ - -f description="cancelled" -f target_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" >/dev/null + REPO: ${{ github.repository }} + PR: ${{ github.event.pull_request.number }} + HEAD_SHA: ${{ github.event.pull_request.head.sha }} + HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name }} + CLAIMED: ${{ needs.claim.outputs.claimed }} + run: go run ./scripts/cilocal check diff --git a/.github/workflows/deploy-production.yml b/.github/workflows/deploy-production.yml new file mode 100644 index 0000000..579a051 --- /dev/null +++ b/.github/workflows/deploy-production.yml @@ -0,0 +1,160 @@ +name: Deploy production +run-name: Deploy production ${{ github.event.release.tag_name || inputs.tag }} + +# Production is released, never pushed (docs/RELEASES.md). Publishing a +# GitHub release for a vX.Y.Z tag starts this workflow. The gate +# (.github/scripts/production-gate.sh) proves the tag's commit is on main, +# that staging ran it and passed readiness, that staging still runs it or a +# later main commit and is healthy now, and that the publisher is in +# PRODUCTION_RELEASERS. Publishing the release (or dispatching a redeploy) is +# the explicit yes every production deploy needs. The deploy job runs in the +# `production` environment, which holds its own RAILWAY_TOKEN (scoped to production) and admits +# only main and v* tags (.github/scripts/release-settings.sh). A dispatch +# redeploys, or rolls back to, an existing release through the same gate. +# Railway builds every service from the release's checkout +# (.github/scripts/deploy-railway.sh). +# +# Until the RAILWAY_DEPLOY repository variable is `true`, the deploy job is +# skipped and "Dry run" prints the commands it would run instead. +# +# Runner: the RELEASE_RUNNER repository variable, else CI_RUNNER (ci.yml). +on: + release: + types: [published] + workflow_dispatch: + inputs: + tag: + description: Existing release tag to deploy (vX.Y.Z); use an older one to roll back + required: true + type: string + +# Never two production deploys at once, and never cancel one mid-flight. A run +# queued behind another waits; GitHub keeps only the newest waiting run. +concurrency: + group: deploy-production + cancel-in-progress: false + +permissions: + contents: read + +jobs: + gate: + name: Check the release may go to production + # Pre-releases are not deployed. + if: github.event_name == 'workflow_dispatch' || !github.event.release.prerelease + runs-on: ${{ fromJSON(vars.RELEASE_RUNNER || vars.CI_RUNNER || '"ubuntu-latest"') }} + timeout-minutes: 10 + permissions: + contents: read + # Deploy staging's runs and jobs: the proof that staging ran the commit. + actions: read + outputs: + tag: ${{ steps.resolve.outputs.tag }} + sha: ${{ steps.resolve.outputs.sha }} + env: + GH_TOKEN: ${{ github.token }} + TAG: ${{ github.event.release.tag_name || inputs.tag }} + steps: + # A dispatch runs the workflow file from the ref it was started on, so + # only main's copy (with this gate in it) may deploy. + - name: Refuse a dispatch from anywhere but main + if: github.event_name == 'workflow_dispatch' && github.ref != 'refs/heads/main' + run: | + echo "::error::Start Deploy production from main, not $GITHUB_REF. Pick the release with the tag input." + exit 1 + + - name: Checkout main with history + uses: actions/checkout@v7 + with: + ref: main + fetch-depth: 0 + + - name: Resolve the tag + id: resolve + run: | + if ! echo "$TAG" | grep -qE '^v[0-9]+\.[0-9]+\.[0-9]+$'; then + echo "::error::'$TAG' is not a vX.Y.Z release tag. Production deploys only release tags, never branches." + exit 1 + fi + git fetch --no-tags origin "+refs/tags/$TAG:refs/tags/$TAG" + sha=$(git rev-parse --verify "refs/tags/$TAG^{commit}") + echo "$TAG is $sha" + echo "tag=$TAG" >>"$GITHUB_OUTPUT" + echo "sha=$sha" >>"$GITHUB_OUTPUT" + + - name: Gate + env: + SHA: ${{ steps.resolve.outputs.sha }} + ACTOR: ${{ github.triggering_actor }} + RELEASERS: ${{ vars.PRODUCTION_RELEASERS }} + STAGING_WEB_URL: ${{ vars.STAGING_WEB_URL }} + run: sh .github/scripts/production-gate.sh + + deploy: + name: Deploy release to Railway production + needs: gate + if: vars.RAILWAY_DEPLOY == 'true' + runs-on: ${{ fromJSON(vars.RELEASE_RUNNER || vars.CI_RUNNER || '"ubuntu-latest"') }} + timeout-minutes: 90 + environment: + name: production + url: ${{ vars.PRODUCTION_WEB_URL }} + env: + TAG: ${{ needs.gate.outputs.tag }} + SHA: ${{ needs.gate.outputs.sha }} + steps: + - name: Checkout the release + uses: actions/checkout@v7 + with: + ref: ${{ needs.gate.outputs.sha }} + + # The deploy can start well after the gate (queued behind another run). + # If the tag was moved meanwhile, what was checked is not what would ship. + - name: Refuse a moved tag + run: | + git fetch --no-tags origin "+refs/tags/$TAG:refs/tags/$TAG" + now=$(git rev-parse --verify "refs/tags/$TAG^{commit}") + if [ "$now" != "$SHA" ]; then + echo "::error::$TAG moved from $SHA to $now after the gate ran. Start a new deploy." + exit 1 + fi + + # deploy-services reads deploy/platform.yaml. + - uses: actions/setup-go@v6 + with: + go-version-file: deploy/go.mod + cache-dependency-path: deploy/go.sum + - name: Install the Railway CLI + run: npm install --global @railway/cli@5.71.0 + - name: Deploy + env: + RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }} + WEB_URL: ${{ vars.PRODUCTION_WEB_URL }} + ADMIN_URL: ${{ vars.PRODUCTION_ADMIN_URL }} + run: sh .github/scripts/deploy-railway.sh production "$SHA" + + dry-run: + name: Dry run (RAILWAY_DEPLOY is not true) + needs: gate + if: vars.RAILWAY_DEPLOY != 'true' + runs-on: ${{ fromJSON(vars.RELEASE_RUNNER || vars.CI_RUNNER || '"ubuntu-latest"') }} + timeout-minutes: 10 + # No environment and no token: it prints the commands and deploys nothing. + steps: + - name: Checkout the release + uses: actions/checkout@v7 + with: + ref: ${{ needs.gate.outputs.sha }} + - uses: actions/setup-go@v6 + with: + go-version-file: deploy/go.mod + cache-dependency-path: deploy/go.sum + - name: Print the production deploy + env: + DRY_RUN: '1' + SHA: ${{ needs.gate.outputs.sha }} + WEB_URL: ${{ vars.PRODUCTION_WEB_URL }} + ADMIN_URL: ${{ vars.PRODUCTION_ADMIN_URL }} + run: | + echo "::notice::Dry run: set the RAILWAY_DEPLOY repository variable to true to deploy (docs/RELEASES.md)." + sh .github/scripts/deploy-railway.sh production "$SHA" diff --git a/.github/workflows/deploy-staging.yml b/.github/workflows/deploy-staging.yml new file mode 100644 index 0000000..c5f743e --- /dev/null +++ b/.github/workflows/deploy-staging.yml @@ -0,0 +1,134 @@ +name: Deploy staging +# The title names the deployed commit. "Staging catch-up" reads the latest +# successful run's title to know what staging runs. +run-name: Deploy staging ${{ github.sha }} + +# Staging always runs the latest main (docs/RELEASES.md). Every commit pushed +# to main that make ci-local attested (cilocal attest: local-ci passed on the +# commit, or on the exact head of the pull request whose merge made it; GitHub +# runs no verification on main) goes to Railway staging: web (whose pre-deploy +# command migrates the database), admin when platform.yaml has it, the backup +# cron when backups are on, then the /healthz readiness checks +# (.github/scripts/deploy-railway.sh). Production is never touched: the token +# is a Railway project token scoped to the staging environment. The hourly +# "Staging catch-up" dispatches this on main whenever staging is behind. +# "Deploy production" and "Draft release" read these runs (the title, the +# run's success and the deploy job's result) as proof that staging ran a +# commit, so keep the title and the job name as they are. +# +# Until the RAILWAY_DEPLOY repository variable is `true`, the deploy job is +# skipped and "Dry run" prints the commands it would run instead, so a fresh +# copy of the template goes green without Railway. A dry run proves nothing to +# Deploy production. +# +# Runner: the CI_RUNNER repository variable, as in ci.yml. +on: + push: + branches: [main] + workflow_dispatch: + +# A newer main supersedes an older deploy, but never midway through changing +# staging. Each job holds its own group: the plan only reads, so a newer run +# cancels it (and the older run deploys nothing); the rollout changes staging, +# so rollouts run one at a time and are never cancelled once started. A newer +# one waiting replaces an older one waiting, which has changed nothing yet. + +permissions: + contents: read + +env: + DEPLOY_SHA: ${{ github.sha }} + +jobs: + plan: + name: Plan the staging services + runs-on: ${{ fromJSON(vars.CI_RUNNER || '"ubuntu-latest"') }} + timeout-minutes: 5 + concurrency: + group: deploy-staging-plan + cancel-in-progress: true + permissions: + contents: read + # The local attestation: the commit's and its pull request head's + # local-ci status, and the pull request that merged it. + statuses: read + pull-requests: read + steps: + # The staging environment refuses other refs too; this says why. + - name: Refuse a dispatch from anywhere but main + if: github.event_name == 'workflow_dispatch' && github.ref != 'refs/heads/main' + run: | + echo "::error::Staging runs main only. Dispatch Deploy staging from main, not $GITHUB_REF." + exit 1 + - uses: actions/checkout@v7 + with: + ref: ${{ env.DEPLOY_SHA }} + sparse-checkout: scripts/cilocal # and the root files (cone mode) + # The tool imports only the standard library: no module downloads. + - uses: actions/setup-go@v6 + with: + go-version-file: go.mod + cache: false + # Nothing deploys a commit make ci-local did not attest; the error says + # why and how to attest it. + - name: Require the local attestation of this commit + env: + GH_TOKEN: ${{ github.token }} + run: go run ./scripts/cilocal attest "$DEPLOY_SHA" + + deploy: + name: Deploy main to Railway staging + needs: plan + if: ${{ !cancelled() && needs.plan.result == 'success' && vars.RAILWAY_DEPLOY == 'true' }} + runs-on: ${{ fromJSON(vars.CI_RUNNER || '"ubuntu-latest"') }} + timeout-minutes: 90 + # Migrations and the rollout: never cancelled midway. + concurrency: + group: deploy-staging-rollout + cancel-in-progress: false + # Holds its own RAILWAY_TOKEN (scoped to staging) and admits only main, so no branch workflow + # can read the token or put branch code on staging. + environment: + name: staging + url: ${{ vars.STAGING_WEB_URL }} + steps: + - uses: actions/checkout@v7 + with: + ref: ${{ env.DEPLOY_SHA }} + # deploy-services reads deploy/platform.yaml. + - uses: actions/setup-go@v6 + with: + go-version-file: deploy/go.mod + cache-dependency-path: deploy/go.sum + - name: Install the Railway CLI + run: npm install --global @railway/cli@5.71.0 + - name: Deploy + env: + RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }} + WEB_URL: ${{ vars.STAGING_WEB_URL }} + ADMIN_URL: ${{ vars.STAGING_ADMIN_URL }} + run: sh .github/scripts/deploy-railway.sh staging "$DEPLOY_SHA" + + dry-run: + name: Dry run (RAILWAY_DEPLOY is not true) + needs: plan + if: ${{ !cancelled() && needs.plan.result == 'success' && vars.RAILWAY_DEPLOY != 'true' }} + runs-on: ${{ fromJSON(vars.CI_RUNNER || '"ubuntu-latest"') }} + timeout-minutes: 10 + # No environment and no token: it prints the commands and deploys nothing. + steps: + - uses: actions/checkout@v7 + with: + ref: ${{ env.DEPLOY_SHA }} + - uses: actions/setup-go@v6 + with: + go-version-file: deploy/go.mod + cache-dependency-path: deploy/go.sum + - name: Print the staging deploy + env: + DRY_RUN: '1' + WEB_URL: ${{ vars.STAGING_WEB_URL }} + ADMIN_URL: ${{ vars.STAGING_ADMIN_URL }} + run: | + echo "::notice::Dry run: set the RAILWAY_DEPLOY repository variable to true to deploy (docs/RELEASES.md)." + sh .github/scripts/deploy-railway.sh staging "$DEPLOY_SHA" diff --git a/.github/workflows/draft-release.yml b/.github/workflows/draft-release.yml new file mode 100644 index 0000000..6698cd3 --- /dev/null +++ b/.github/workflows/draft-release.yml @@ -0,0 +1,112 @@ +name: Draft release +run-name: Draft release ${{ inputs.version || format('(next {0})', inputs.bump) }} + +# Cuts a draft GitHub release at the commit staging runs now, with notes +# generated from the pull requests merged since the last release. Nothing is +# tagged or deployed here: publishing the draft creates the tag and starts +# "Deploy production" (docs/RELEASES.md). A release made with the workflow +# token could not start another workflow, which is why this stops at a draft. +# +# Runner: the RELEASE_RUNNER repository variable, else CI_RUNNER (ci.yml). +on: + workflow_dispatch: + inputs: + bump: + description: Which part of the last vX.Y.Z to increase (the first release is v1.0.0) + required: true + type: choice + options: [patch, minor, major] + default: patch + version: + description: Exact version instead (vX.Y.Z). Leave empty to use the bump. + required: false + type: string + +concurrency: + group: draft-release + cancel-in-progress: false + +permissions: + # gh release create + contents: write + # staging_ran / staging_commit read Deploy staging's runs and jobs. + actions: read + +jobs: + draft: + name: Draft a release of what staging runs + runs-on: ${{ fromJSON(vars.RELEASE_RUNNER || vars.CI_RUNNER || '"ubuntu-latest"') }} + timeout-minutes: 5 + env: + GH_TOKEN: ${{ github.token }} + VERSION: ${{ inputs.version }} + BUMP: ${{ inputs.bump }} + ACTOR: ${{ github.triggering_actor }} + RELEASERS: ${{ vars.PRODUCTION_RELEASERS }} + STAGING_WEB_URL: ${{ vars.STAGING_WEB_URL }} + steps: + - name: Checkout main with history + uses: actions/checkout@v7 + with: + ref: main + fetch-depth: 0 + + - name: Draft the release + run: | + . .github/scripts/release-lib.sh + if ! is_releaser "$ACTOR"; then + echo "::error::$ACTOR is not in the PRODUCTION_RELEASERS repository variable." + exit 1 + fi + code=$(staging_ready) + if [ "$code" != 200 ]; then + echo "::error::$(staging_not_ready "$code") Fix staging before releasing." + exit 1 + fi + if ! sha=$(staging_commit) || ! git merge-base --is-ancestor "$sha" origin/main; then + echo "::error::Cannot prove staging runs a commit on main (found '${sha:-}'). A dry run (RAILWAY_DEPLOY not true) deploys nothing, so it proves nothing." + exit 1 + fi + run=$(staging_ran "$sha") || { + echo "::error::No successful Deploy staging run found for $sha." + exit 1 + } + + last=$(git tag --list 'v[0-9]*.[0-9]*.[0-9]*' --sort=-v:refname | head -1) + if [ -z "$VERSION" ]; then + if [ -z "$last" ]; then + VERSION=v1.0.0 + else + rest=${last#v} + major=${rest%%.*} + rest=${rest#*.} + minor=${rest%%.*} + patch=${rest#*.} + case $BUMP in + major) VERSION="v$((major + 1)).0.0" ;; + minor) VERSION="v$major.$((minor + 1)).0" ;; + *) VERSION="v$major.$minor.$((patch + 1))" ;; + esac + fi + fi + if ! echo "$VERSION" | grep -qE '^v[0-9]+\.[0-9]+\.[0-9]+$'; then + echo "::error::'$VERSION' is not vX.Y.Z." + exit 1 + fi + if git rev-parse -q --verify "refs/tags/$VERSION" >/dev/null || gh release view "$VERSION" >/dev/null 2>&1; then + echo "::error::$VERSION already exists as a tag or release." + exit 1 + fi + if [ -n "$last" ] && [ "$(git rev-parse "$last^{commit}")" = "$sha" ]; then + echo "::error::$last already releases $sha, the commit staging runs. Nothing new to release." + exit 1 + fi + + url=$(gh release create "$VERSION" --draft --target "$sha" --title "$VERSION" --generate-notes \ + --notes "Staging ran \`$sha\` and passed readiness ([Deploy staging run]($GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$run)).") + echo "Drafted $VERSION at $sha: $url" + { + echo "Drafted **$VERSION** at \`$sha\` (the commit staging runs)." + echo + echo "Review the notes, then **Publish release** at $url. Publishing tags $sha and deploys it to production through Deploy production's gate." + } >>"$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/post-merge-report.yml b/.github/workflows/post-merge-report.yml new file mode 100644 index 0000000..c53b462 --- /dev/null +++ b/.github/workflows/post-merge-report.yml @@ -0,0 +1,50 @@ +# Reports a push-triggered workflow that failed on main to the PR whose merge +# started it, so a failure after merge (a staging deploy) never sits +# unnoticed. A green or cancelled run skips the job: no runner, no comment. +# Every workflow that runs on a push to main belongs in the list below; +# scripts/cilocal/verify's TestThePostMergeReportListsEveryPushToMainWorkflow +# fails when one is missing. +# +# Runner: the CI_RUNNER repository variable, as in ci.yml. +name: Post-merge report + +on: + workflow_run: + workflows: + - Deploy staging + types: [completed] + +permissions: + contents: read + pull-requests: write + +jobs: + report: + name: Report the failure on the merged PR + if: >- + github.event.workflow_run.event == 'push' && + github.event.workflow_run.head_branch == 'main' && + contains(fromJSON('["failure","timed_out","startup_failure"]'), github.event.workflow_run.conclusion) + runs-on: ${{ fromJSON(vars.CI_RUNNER || '"ubuntu-latest"') }} + timeout-minutes: 5 + env: + GH_TOKEN: ${{ github.token }} + WORKFLOW: ${{ github.event.workflow_run.name }} + CONCLUSION: ${{ github.event.workflow_run.conclusion }} + SHA: ${{ github.event.workflow_run.head_sha }} + RUN_URL: ${{ github.event.workflow_run.html_url }} + steps: + - name: Comment on the merged pull request + run: | + pr=$(gh api "repos/$GITHUB_REPOSITORY/commits/$SHA/pulls" \ + -q '[.[] | select(.merged_at != null)][0].number // empty') + short=$(printf '%s' "$SHA" | cut -c1-9) + line="❌ **$WORKFLOW** ended \`$CONCLUSION\` on \`$short\` after merge: $RUN_URL" + echo "$line${pr:+ (#$pr)}" >>"$GITHUB_STEP_SUMMARY" + echo "::error::$WORKFLOW ended $CONCLUSION on $short${pr:+ (#$pr)}: $RUN_URL" + if [ -z "$pr" ]; then + echo "No merged pull request for $short; nothing to comment on." + exit 0 + fi + gh pr comment "$pr" -R "$GITHUB_REPOSITORY" \ + --body "$line"$'\n\nFix it forward before merging more (AGENTS.md, "Environments: staging is always current").' diff --git a/.github/workflows/staging-catch-up.yml b/.github/workflows/staging-catch-up.yml new file mode 100644 index 0000000..c491929 --- /dev/null +++ b/.github/workflows/staging-catch-up.yml @@ -0,0 +1,73 @@ +name: Staging catch-up + +# Staging always runs the latest main (docs/RELEASES.md). Every hour this +# checks what staging last received, from the title of the latest successful +# "Deploy staging" run. When that isn't the current main and make ci-local +# attested main (cilocal attest), it dispatches "Deploy staging" on main. That +# covers a missed, cancelled or failed deploy. It never deploys anything +# itself. An unattested main fails the check rather than dispatching (the +# deploy would refuse it too): attest the merged pull request's head, or main +# itself (make ci-local COMMIT= RAN="…" in a clean worktree of it), then +# it catches up within the hour. +# +# Runner: the CI_RUNNER repository variable, as in ci.yml. +on: + schedule: + - cron: '17 * * * *' + workflow_dispatch: + +concurrency: + group: staging-catch-up + cancel-in-progress: true + +permissions: + contents: read + # gh workflow run: a dispatch is the one event the workflow token may start. + actions: write + # cilocal attest + statuses: read + pull-requests: read + +jobs: + check: + name: Keep staging on main + runs-on: ${{ fromJSON(vars.CI_RUNNER || '"ubuntu-latest"') }} + timeout-minutes: 5 + env: + GH_TOKEN: ${{ github.token }} + steps: + - uses: actions/checkout@v7 + with: + ref: main + sparse-checkout: scripts/cilocal # and the root files (cone mode) + - uses: actions/setup-go@v6 + with: + go-version-file: go.mod + cache: false + - name: Dispatch a deploy when staging is behind main + run: | + main=$(gh api "repos/$GITHUB_REPOSITORY/commits/main" -q .sha) + last=$(gh run list -R "$GITHUB_REPOSITORY" --workflow deploy-staging.yml \ + --status success --limit 1 --json displayTitle -q '.[0].displayTitle // ""') + pending=$(gh run list -R "$GITHUB_REPOSITORY" --workflow deploy-staging.yml \ + --limit 5 --json status,displayTitle \ + -q "[.[] | select(.status != \"completed\" and (.displayTitle | contains(\"$main\")))] | length") + echo "main $main" + echo "last successful deploy: '$last'" + echo "deploys of main in progress: $pending" + case "$last" in + *"$main"*) + echo "Staging runs main." + exit 0 + ;; + esac + if [ "$pending" != 0 ]; then + echo "A deploy of main is already running." + exit 0 + fi + if ! go run ./scripts/cilocal attest "$main"; then + echo "::error::Staging is behind main, and main is not attested: the top priority, as a red main was." + exit 1 + fi + echo "::warning::Staging is behind main. Dispatching Deploy staging on main." + gh workflow run deploy-staging.yml -R "$GITHUB_REPOSITORY" --ref main diff --git a/.gitleaks.toml b/.gitleaks.toml new file mode 100644 index 0000000..47c0381 --- /dev/null +++ b/.gitleaks.toml @@ -0,0 +1,26 @@ +# gitleaks configuration for the secret scan (scripts/cilocal secret-scan, +# docs/CI.md "Secret scan"): gitleaks' own rules, with one exception for the +# dev-loop bench script, which boots the binaries with throwaway values. Only +# those exact synthetic strings are allowed, and only in that script: a real +# secret in a test, in testdata, in an example env file or in the bench script +# is a leak. A new deliberate fixture gets its own obviously fake value and an +# exact entry here; a real secret is removed and rotated, because history +# keeps it. +[extend] +useDefault = true + +# Rule-scoped and value-based: the generic-api-key rule, in the bench script, +# matching the bench's synthetic values. condition = "AND" is what narrows it: +# gitleaks ORs an allowlist's paths and regexes by default, which would allow +# every value in the script. +[[rules]] +id = "generic-api-key" + +[[rules.allowlists]] +description = "scripts/bench-devloop.sh boots the binaries with these synthetic values" +condition = "AND" +paths = ['''^scripts/bench-devloop\.sh$'''] +regexes = [ + '''^bench-secret-bench-secret-0123456789abcdef$''', + '''^bench-admin-secret-0123456789abcdef0123$''', +] diff --git a/.golangci.yml b/.golangci.yml index cf61a9b..5c39f5d 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -435,6 +435,16 @@ linters: desc: those are port mocks; handler tests mock the SERVICE, never what is underneath it - pkg: github.com/jayleaton/go-fast-stack/packages/auth desc: handler tests mock the session; real auth is tested in packages/auth + # Local verification's rules stay pure: git, gh and HTTP run in + # scripts/cilocal, which passes their results in. + cilocal-verify: + files: + - "**/scripts/cilocal/verify/**" + deny: + - pkg: os/exec + desc: scripts/cilocal/verify is pure; run commands in scripts/cilocal and pass the results in + - pkg: net/http + desc: scripts/cilocal/verify is pure; call GitHub in scripts/cilocal and pass the results in no-test-helpers: files: - $all diff --git a/AGENTS.md b/AGENTS.md index 28e1f0e..31d0beb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,7 +7,7 @@ Shared vocabulary lives in [CONTEXT.md](CONTEXT.md). Use those words. ## What makes this template good 1. **Template first.** People clone this and diverge right away. Keep it generic and minimal. The example flows (the users API, the dashboard) are documentation as code: keep them canonical and small. If a change only makes sense for one project, it does not belong here. -2. **Performance without compromise.** Render on the server, keep client JavaScript to what Datastar needs, and treat binary size, memory and CI time as features. Prefer the standard library; every dependency needs a reason. +2. **Performance without compromise.** Render on the server, keep client JavaScript to what Datastar needs, and treat binary size, memory and verification time as features. Prefer the standard library; every dependency needs a reason. 3. **Network-ready dev servers.** The server binds `0.0.0.0:3000`, so it works the same on localhost, the LAN, and Tailscale (`http://:3000`). Never bake `localhost` into code, emails or auth callbacks. 4. **Monorepo-aware.** `packages/database`, `packages/auth`, `apps/web` and `apps/mobile-app` move together. Every change gets a decision per workspace, even if it is "no change needed". @@ -39,7 +39,7 @@ mobile app, servers ── JSON over HTTP ──▶ api.Endpoint.Serve ─── users.updateSelf = Endpoint[validate.UpdateUserSettings, UserUpdated]{"PATCH", routes.Me, access.SignedIn, users.updateMine} ``` - The browser never calls `/api/...`, never receives the API's JSON, and no rendered HTML or JS names an API path: forbidigo rejects `config.APIRoutes` in `ui/**`, and `scripts/check-ui-api-paths.sh` (in `make lint`, `make check-fast` and CI; probed by `make lint-layering`) fails on any `/api/` in UI templates, code or static assets. + The browser never calls `/api/...`, never receives the API's JSON, and no rendered HTML or JS names an API path: forbidigo rejects `config.APIRoutes` in `ui/**`, and `scripts/check-ui-api-paths.sh` (in `make lint` and `make check-fast`; probed by `make lint-layering`) fails on any `/api/` in UI templates, code or static assets. - **Sign-in flows are API endpoints too.** Sign-up, sign-in, sign-out, verification, password reset and 2FA are `api.Endpoint`s (`api.Sessions`) over an `api.Authenticator` port that `server.Authenticator` adapts from `packages/auth`. The UI server action calls the client and sets the cookies the result carries with `setCookies` (`ui/cookies.go`); auth signs them, the UI only copies them onto its response. - The UI never imports `services`, `validate`, `access`, `api`, `packages/auth`, repositories, `packages/database` or sqlc output, and `server` never imports its app's `ui` (no adapters for UI-owned ports); transports never import `pgx`, sqlc output or repositories. Services never import `net/http`, templ or Datastar. `packages/database` never imports `apps/`. `depguard` in `.golangci.yml` enforces this, and `make lint-layering` proves the rules still bite. - Ports are defined by the consumer. A service declares the interfaces it needs (`services.UserRepository`, `services.AccountCredentials`); repositories in `packages/database` and adapters in `apps/web/internal/server` implement them. The API in turn declares the service it calls (`api.UserService`); the UI declares nothing below the API: it takes `apiclient.Client`. @@ -85,11 +85,12 @@ The most common defect is a change that works on the path you tested and is miss ## Verifying -- **`make check-fast` before every push (mandatory).** It runs gofmt, `go vet`, golangci-lint, the test-location check and plain `go test` (no `-race`) on only the packages your changes can affect, plus the generators whose inputs changed, the `deploy/` module (vet, lint, names and `make deploy-vars`) when it, `.env.example` or the Makefile changed, and the migration order when a migration changed (see "Migrations"). Changes mean everything that differs from the merge base with origin/dev (origin/main when there is no dev): commits, staged, unstaged and untracked files, mapped to packages by `scripts/affected.sh` exactly as CI does, so a shared input (go.mod, tooling, lint config, migrations) checks everything. It takes seconds (a docs-only change: under a second; a full run: under 10 s warm), so it fits machines where a full `make ci` is not allowed. Database tests skip unless `TEST_DATABASE_URL` is set; CI still runs `-race`, Postgres and the probes. `make hooks` installs a pre-push hook that runs it from every worktree of the clone (`make hooks-remove` undoes it; `git push --no-verify` skips it once, for emergencies only). Anything that runs git or go from a hook drops git's repository env first (`git rev-parse --local-env-vars`), as the hook and `scripts/check-fast.sh` do. -- **Release smoke** (`make release-smoke`; CI runs it in the `ci` job whenever a build input changes: a Dockerfile, or code a Dockerfile's binary is built from): builds every binary that has a Dockerfile with that Dockerfile's own build-stage `ENV` and `RUN` line (`scripts/dockerfile-builds.sh` reads them, so never copy build flags elsewhere) and fails on a dynamically linked one, runs `server migrate` and `admin migrate` on fresh databases, and boots web and admin with `APP_ENV=production` and only the config production requires until `/healthz` answers 200. It catches what tests cannot: an image that will not start on distroless (a dependency that links dynamically even with `CGO_ENABLED=0`), and a production config that refuses to boot without an optional integration. A new required production setting goes in its env list. -- `make lint test` before every commit when you can run the full suite. `make ci` runs exactly what CI runs: build, generated-code diff (sqlc, mocks, templ), gofmt + golangci-lint, the test-location check, `go test -race` against a throwaway Postgres with per-boundary coverage floors, and the probes that prove the layering and test rules still bite (`make lint-layering lint-tests`). -- CI on a PR checks only what the change can affect: `scripts/affected.sh` maps changed files to Go packages and adds every package whose code or tests import them (`go list -test -deps`), then builds, lints, tests and coverage-checks just those, and reruns only the generators whose inputs or outputs changed. Changes to go.mod/go.sum, tools/, scripts/, workflows, `.golangci.yml`, the Makefile, migrations or sqlc config run everything, as do pushes to dev and main and the merge queue. `make affected` shows what your branch would run. A new shared input (a file that can change many packages' results without being in their package graph) belongs in the script's full-run list. -- CI layout: jobs bill rounded up to whole minutes, so CI is ONE job (`ci`): affected-only generators, gofmt, golangci-lint, the migration order, the release smoke (when a build input changed) and race tests (all packages on pushes to dev and main, the merge queue and manual runs), plus the mobile app and `deploy/` module checks when those change. Docs-only PRs post `ci-ok` from that job with nothing to check; drafts start no jobs. Sharding is an opt-in fallback only: label a PR `ci:fanout` (takes effect on its next push) or run the workflow manually with `fanout`, which splits into `lint + test (db)` and `test (other)` for faster wall time at more billed minutes. Timeouts are ~2x measured averages (see the workflow comments). Shared steps live in `.github/actions/check`. Avoid adding jobs: fold new checks into the existing one. A push to dev or main whose tree is identical to a PR merge commit that already passed CI is not tested again: `ci-ok` is posted from that result (`scripts/ci-reuse.sh`; PR results record the tested tree in their `ci-ok` description). Caches: modules keyed by go.sum; one build/lint cache keyed by commit, saved by dev and main pushes that run and by a scheduled run every 4 hours on the default branch (skipped when it has not moved), restored by prefix elsewhere. +- **`make check-fast` before every push (mandatory).** It runs gofmt, `go vet`, golangci-lint, the test-location check and plain `go test` (no `-race`) on only the packages your changes can affect, plus the generators whose inputs changed, the `deploy/` module (vet, lint, names and `make deploy-vars`) when it, `.env.example` or the Makefile changed, and the migration order when a migration changed (see "Migrations"). Changes mean everything that differs from the merge base with origin/main: commits, staged, unstaged and untracked files, mapped to packages by `scripts/affected.sh` (`make affected` prints them), so a shared input (go.mod, tooling, lint config, migrations) checks everything. It takes seconds (a docs-only change: under a second; a full run: under 10 s warm), so it fits machines where a full `make ci` is not allowed. Database tests skip unless `TEST_DATABASE_URL` is set; `make test` runs them with `-race` against Postgres. `make hooks` installs a pre-push hook that runs it from every worktree of the clone (`make hooks-remove` undoes it; `git push --no-verify` skips it once, for emergencies only). Anything that runs git or go from a hook drops git's repository env first (`git rev-parse --local-env-vars`), as the hook and `scripts/check-fast.sh` do. +- **Release smoke** (`make release-smoke`; run it whenever a build input changes: a Dockerfile, or code a Dockerfile's binary is built from): builds every binary that has a Dockerfile with that Dockerfile's own build-stage `ENV` and `RUN` line (`scripts/dockerfile-builds.sh` reads them, so never copy build flags elsewhere) and fails on a dynamically linked one, runs `server migrate` and `admin migrate` on fresh databases, and boots web and admin with `APP_ENV=production` and only the config production requires until `/healthz` answers 200. It catches what tests cannot: an image that will not start on distroless (a dependency that links dynamically even with `CGO_ENABLED=0`), and a production config that refuses to boot without an optional integration. A new required production setting goes in its env list. +- `make lint test` before every commit when you can run the full suite. `make ci` runs every check on the checkout as it is: build, generated-code diff (sqlc, mocks, templ), gofmt + golangci-lint, the test-location check, `go test -race` against a throwaway Postgres with per-boundary coverage floors, and the probes that prove the layering and test rules still bite (`make lint-layering lint-tests`). +- **A pull request is verified locally, on its exact head, then attested (mandatory; [docs/CI.md](docs/CI.md)).** Run what the change needs ("What to run" there): for Go, `make lint test` scoped with `LINT_DIRS`/`TEST_PKGS` from `scripts/affected.sh`, plus `make check-generated`, `make check-migrations`, `make deploy-check`, `make actionlint` or `make release-smoke` for what you touched, `make ci` for a change to the tooling. Commit, push, then `make ci-local RAN=""` in the clean worktree. It runs nothing and takes seconds: it refuses a dirty, unpushed or already-failed head, posts the `local-ci` commit status and fills the PR body's "Local verification" section with your statement, which is never edited by hand. Every push or rebase needs a new attestation. Stack PRs and merge right away. +- **A failed head is final.** A failure is fixed forward with a new commit, never attested until green: a head attested as failed (`make ci-local FAILED=""`) is refused for good by `make ci-local`, the `Local verification` check and `make pr-merge`. A failure in what the branch changed is fixed in the PR; any other is fixed forward at once in its own PR off `main` with a regression test that fails before the fix, and the branch rebases on it. +- GitHub runs no verification: on a PR only `Secret scan` and `Local verification` (which shows "Expected" until `make ci-local` claims the head), started by `make ci-local`'s body edit, and on `main` only `Deploy staging`, which ships a commit only when `make ci-local` attested it. Avoid adding workflow jobs: a new check is a make target the agent runs, listed in docs/CI.md "What to run". The PR jobs run only `main`'s code (`pull_request_target`) and read the head as data; never check out or run the head there ([docs/CI.md](docs/CI.md#trusted-code-only)). - Run the narrowest thing first: `go test ./apps/web/internal/services/...`. - Behaviour changes ship with tests at the boundaries below, and only there. - A test that needs a sleep or timeout to pass is wrong. See "Fast tests". @@ -98,35 +99,36 @@ The most common defect is a change that works on the path you tested and is miss ## Test boundaries (mandatory) -The business logic is the testing layer. Tests live only in these five directories; everything else (UI, config, server wiring, repositories, `packages/database`, `packages/secrets`, `deploy/`) has no tests and must not get any. The database is assumed working: migrations are checked against real state before they ship, and services see repositories only through mocked ports. All of it is enforced by `make lint` / CI, not by review: `scripts/check-test-boundaries.sh` rejects a `_test.go` anywhere else (there is no allowlist; do not add one), depguard rules in `.golangci.yml` reject the wrong imports in test files, and `scripts/check-coverage.sh` holds a coverage floor per boundary. +The business logic is the testing layer. Tests live only in these six directories; everything else (UI, config, server wiring, repositories, `packages/database`, `packages/secrets`, `deploy/`) has no tests and must not get any. The database is assumed working: migrations are checked against real state before they ship, and services see repositories only through mocked ports. All of it is enforced by `make lint`, not by review: `scripts/check-test-boundaries.sh` rejects a `_test.go` anywhere else (there is no allowlist; do not add one), depguard rules in `.golangci.yml` reject the wrong imports in test files, and `scripts/check-coverage.sh` holds a coverage floor per boundary. | Boundary | Directory (not subdirectories) | What a test does | It may use | Floor | |---|---|---|---|---| | service | `apps/web/internal/services`, `apps/admin/internal/services` | unit-tests every method: happy path, 403 for another user, each rule, every port error mapped | generated mocks of the ports (`services/mocks`) | 90 % | | route handler | `apps/web/internal/api`, `apps/admin/internal/api` | per route: no session → 401, other user → 403, invalid input → 400, happy path → 2xx + envelope; each `api.Client` method (the UI's in-process path) reaches its service; validators are covered here | generated mock of the service (`api/mocks`), `apitest.AsUser(u)` / `apitest.AsAdmin(a)` / `apitest.Anonymous` through `apitest.Gate()`, the real router and `api.Route`; `route_test.go` covers the access outcomes (public, 401, 403 from a custom policy, 500 from the resolver) | 85 % | | backend auth | `packages/auth` | its own suite: sessions, passwords, rate limits, rotation | real Postgres | 80 % | +| local verification | `scripts/cilocal/verify` | the rules `make ci-local` and its GitHub checks enforce: the head and failed-head refusals, the PR section and its checks, the merge decision, the staging verdict, the post-merge report's list | the standard library only; the package stays pure (depguard rejects `os/exec` and `net/http` in it) | 85 % | If behaviour outside these directories matters, move the logic into a service and test it there. Service tests may not import pgx, `packages/database` (services re-export `User`, `ErrRecordNotFound`, ...), `packages/auth` or `api`; handler tests may not import pgx, `packages/database`, `packages/auth` or `services/mocks` (mock the service, never what is underneath it); production code may not import mocks or `apitest`. -Mocks are generated, never written: `//go:generate go tool mockgen -source=.go -destination=mocks/.go -package=mocks` next to the interfaces, `make generate`, commit the output (CI fails on a stale mock). `mockgen` is a `tool` in the root `go.mod` so it always matches the `go.uber.org/mock` version the mocks import. +Mocks are generated, never written: `//go:generate go tool mockgen -source=.go -destination=mocks/.go -package=mocks` next to the interfaces, `make generate`, commit the output (`make check-generated` fails on a stale mock). `mockgen` is a `tool` in the root `go.mod` so it always matches the `go.uber.org/mock` version the mocks import. ## Fast tests -The suite runs on every change it can affect, under `-race` on 2-vCPU CI runners, so its speed is a feature that has to survive growth. Lint and `make test` enforce these; `make lint-tests` proves the budgets trip. +The suite runs on every change it can affect, under `-race` on machines as small as a 2-vCPU box, so its speed is a feature that has to survive growth. Lint and `make test` enforce these; `make lint-tests` proves the budgets trip. -- **Parallel.** Every test starts with `t.Parallel()` (`paralleltest`). Slow table tests make their cases parallel subtests too; `tparallel` keeps a test and its subtests consistent. Build expensive fixtures once per group and share them with the parallel subtests: CI has 2 vCPUs, so a split that doubles CPU time gains nothing, and a long subtest trips the budget. Judge CPU cost with `GOMAXPROCS=2`; on a big machine idle parallel workers inflate it. Never share a gomock controller, `httptest.Server` or recorder across parallel subtests; build them per case. A test that must touch process-wide state stays serial with `//nolint:paralleltest // `; prefer removing the global instead. +- **Parallel.** Every test starts with `t.Parallel()` (`paralleltest`). Slow table tests make their cases parallel subtests too; `tparallel` keeps a test and its subtests consistent. Build expensive fixtures once per group and share them with the parallel subtests: a sandbox or runner may have 2 vCPUs, so a split that doubles CPU time gains nothing, and a long subtest trips the budget. Judge CPU cost with `GOMAXPROCS=2`; on a big machine idle parallel workers inflate it. Never share a gomock controller, `httptest.Server` or recorder across parallel subtests; build them per case. A test that must touch process-wide state stays serial with `//nolint:paralleltest // `; prefer removing the global instead. - **No sleeps.** `time.Sleep` in a test fails lint (forbidigo). Wait on a channel, a condition or `synctest.Wait`; for intervals, expiry and backoff use `testing/synctest` or an injected clock (`auth`'s `testClock`). - **One database per test, cloned.** `packages/auth` (the only suite on Postgres) clones a database migrated once per server in its `TestMain` (`newDatabase`), so tests never migrate; tests on its shared database use unique data (`uniqueEmail`). - **Cheap crypto.** `auth.New` uses cheap argon2 inside test binaries; only `TestPasswordHashDefaults` pays production cost. - **Table tests.** One test with a case table, not a copy per case. -- **Budgets.** `make test` and CI pipe `go test -json` through `scripts/testbudget`: plain `go test` output, then the slowest tests, then a failure if a package's test binary runs over 60 s or a single test (a leaf test or subtest) over 10 s, measured on CI under `-race` (about twice today's slowest, so a doubling fails). CI measures tests, not compilation: it builds every test binary first, then runs the timed tests at normal CPU priority while its other tasks (lint, generators, smoke, `deploy/`, mobile) run at nice 19 (`scripts/ci-check.sh`), so a PR that adds a heavy dependency cannot push an unchanged test over its budget. Over budget means split or speed up the test. Raise a budget (constants in `scripts/testbudget`) only with the reason in the commit. -- **Measure, don't guess.** `make bench-devloop` prints the timing tables (build, generate, lint per linter, probes, tests per package with and without `-race`); `BENCH_COLD=1` adds cold caches, `BENCH_PKGS` narrows the packages, and `BENCH_BASE=` adds an A/B of the CI test run against that ref on the same machine (medians of alternating rounds). Separate CI runs land on different machines and vary by tens of percent, so compare a perf change with the A/B: push it to a `bench/` branch (the "Bench dev loop" workflow compares it with main) and delete the branch afterwards. +- **Budgets.** `make test` pipes `go test -json` through `scripts/testbudget`: plain `go test` output, then the slowest tests, then a failure if a package's test binary runs over 60 s or a single test (a leaf test or subtest) over 10 s, under `-race` (about twice today's slowest on a 2-vCPU runner, so a doubling fails). The budget times test binaries, not their compilation; run lint and the timed tests one after the other, so they never share the CPUs. Over budget means split or speed up the test. Raise a budget (constants in `scripts/testbudget`) only with the reason in the commit. +- **Measure, don't guess.** `make bench-devloop` prints the timing tables (build, generate, lint per linter, probes, tests per package with and without `-race`); `BENCH_COLD=1` adds cold caches, `BENCH_PKGS` narrows the packages, and `BENCH_BASE=` adds an A/B of the test run against that ref on the same machine (medians of alternating rounds). Separate runs land on different machines and vary by tens of percent, so compare a perf change with the A/B: push it to a `bench/` branch (the "Bench dev loop" workflow compares it with main) and delete the branch afterwards. ## Migrations -- **A new migration takes the next number above the highest on both origin/dev and origin/main (whichever exist), per goose set** (`packages/database/migrations` `goose_app`, `packages/auth/migrations` `goose_auth`, `packages/database/adminrepo/migrations` `goose_admin`). If either moves past it before your branch merges, renumber it. Goose refuses out-of-order migrations: once a deployed database has applied 00007, a later-merged 00005 fails `migrate` on every deploy. A gap is fine; a number below an applied one is not. +- **A new migration takes the next number above the highest on origin/main, per goose set** (`packages/database/migrations` `goose_app`, `packages/auth/migrations` `goose_auth`, `packages/database/adminrepo/migrations` `goose_admin`). If it moves past it before your branch merges, renumber it. Goose refuses out-of-order migrations: once a deployed database has applied 00007, a later-merged 00005 fails `migrate` on every deploy. A gap is fine; a number below an applied one is not. - Append-only once merged: fix a mistake with a new migration, never by editing or renumbering a merged one (deployed databases record its version). -- `scripts/check-migration-order.sh` enforces it and names the number to renumber to: `make check-fast` runs it when a migration changed (against the tips of origin/dev and origin/main, so a branch either has moved past fails too), CI runs it in the `ci` job against the target branch plus the tip of the other deployed branch, and `make check-migrations` runs it alone. Two open branches may take the same next number; whichever merges second fails and renumbers. +- `scripts/check-migration-order.sh` enforces it and names the number to renumber to: `make check-fast` runs it when a migration changed (against the tip of origin/main, so a branch main has moved past fails too), and `make check-migrations` runs it alone (run it before attesting a migration). Two open branches may take the same next number; whichever merges second fails and renumbers. ## Admin app @@ -170,10 +172,11 @@ Only when the developer asks: it creates billed resources in their Railway accou 1. **Preflight.** `bin/railway-setup` is built by any `make railway-*` target. Run `make railway-doctor ARGS=--json` and fix every check with `"status": "fail"`; each check carries a `fix`. Typical fixes: `railway login` (the developer's browser), `--workspace` when they belong to several, and `railway ssh keys add` when there is more than one environment. 2. **Setup.** `make railway-setup ARGS="--yes --json --environments staging,production --email-from noreply@"`. A real sender is required: plan, setup and apply refuse an unset or example `email.from` (production refuses to boot with one), and `nextSteps` says so. Ask the developer for the sender on their verified email domain; never invent one. Add `--admin true`, `--region`, `--project` or `--workspace` as asked. Every flag has a `PLATFORM_*` env equivalent (`PLATFORM_YES=1`, `PLATFORM_JSON=1`, `PLATFORM_ENVIRONMENTS`, ...). The flags are written into `deploy/platform.yaml`: commit that file. Without `--yes`, and with no terminal to ask on, creating a project stops with exit 3. -3. **Read the summary.** stdout is one JSON object: `ok`, `project.{id,url}`, `source` (`github` or `upload`), each environment's `branch` (with `github`: production `main`, staging and qa `dev` unless `branches` says otherwise) and services with `url`, `deployment` and `health`, and `nextSteps`. Logs go to stderr. Relay `nextSteps` to the developer verbatim. They are things only a person can do, such as installing the GitHub app, Doppler tokens or a real `RESEND_API_KEY`. Never invent or fill in secrets yourself. +3. **Read the summary.** stdout is one JSON object: `ok`, `project.{id,url}`, `source` (`github` or `upload`), each environment's `branch` (only with `github`, which this template leaves off: [Environments](#environments-staging-is-always-current)) and services with `url`, `deployment` and `health`, and `nextSteps`. Logs go to stderr. Relay `nextSteps` to the developer verbatim. They are things only a person can do, such as installing the GitHub app, Doppler tokens or a real `RESEND_API_KEY`. Never invent or fill in secrets yourself. 4. **Doppler** (`secrets.provider: doppler`): when the developer has run `doppler login`, setup also moves the app secrets into Doppler ([docs/secrets.md](docs/secrets.md#secrets-and-doppler)); otherwise `nextSteps` says so, and `make doppler-setup ARGS="--json"` does it later. Its output names configs, secret names and fingerprints only. Keep it that way: never print, log or paste a token or secret value. 5. **Hand-set values.** Never set a Railway variable by hand without listing it in `unmanaged_variables` or as `manual` in `per_environment` ([docs/deploy.md](docs/deploy.md#hand-set-variables)): the next apply deletes it. `make deploy-vars` must pass. -6. **Re-run anytime.** `make railway-setup` is idempotent: no changes, no redeploys when healthy. Removing an environment or `admin` never deletes anything without `CONFIRM_DESTRUCTIVE=1`; ask before passing it. +6. **Workflows deploy, not setup.** Once the project exists, merges ship through `Deploy staging` and releases through `Deploy production` with environment-scoped project tokens ([docs/RELEASES.md](docs/RELEASES.md#one-time-setup)); `make railway-setup` changes infrastructure and variables only. +7. **Re-run anytime.** `make railway-setup` is idempotent: no changes, no redeploys when healthy. Removing an environment or `admin` never deletes anything without `CONFIRM_DESTRUCTIVE=1`; ask before passing it. Exit codes: @@ -195,13 +198,21 @@ Exit codes: - Never weaken the restore guards or log a value: logs carry ids, sizes and fingerprints. The encryption key is a keyring (`BACKUP_ENCRYPTION_KEY`, `_PREVIOUS`), validated with `Len` and revealed only in `apps/backup/cmd`. - Retention is the job's (Railway buckets have no lifecycle rules). Before changing `Retention.Expired`, prove the change with a local prune run: plant old backup ids in a local store and check what `backup prune` keeps. +## Environments: staging is always current + +- **Staging always runs the latest `main`.** `main` runs no CI: every merge of a head `make ci-local` attested deploys to staging automatically (`Deploy staging`, `.github/workflows/deploy-staging.yml`), and the hourly "Staging catch-up" deploys `main` whenever staging is behind ([docs/RELEASES.md](docs/RELEASES.md)). +- **Production may lag staging, never lead it.** Production only ever runs a commit that staging has already run and passed its readiness check on. No hotfixes straight to production, no production deploys of branches, and never a production commit that staging has not run. +- **A failed staging deploy, or an unattested `main` that staging refuses, is the top priority.** After a merge, check its deploy once with `make pr-merge ARGS="--check-deploy "` (one read, not a polling loop); "Post-merge report" also comments on the merged PR when Deploy staging fails on `main`. If it failed, fix it, or attest and dispatch `main` ([docs/CI.md](docs/CI.md#required-checks-and-staging)), before merging anything else. Never leave staging behind `main`. +- **Production ships only as a GitHub release** ([docs/RELEASES.md](docs/RELEASES.md)): publishing a `vX.Y.Z` release starts "Deploy production", whose gate proves staging ran that commit, still runs it or a later `main` commit and is healthy, and that the publisher is in `PRODUCTION_RELEASERS`. An agent publishes or dispatches one only when the developer explicitly says so. +- Railway never deploys on its own: `deploy/platform.yaml` keeps `repo: ""` (upload mode), so Railway's GitHub auto-deploy stays off and only the workflows (or `make railway-up` by hand) ship code. + ## Branches and releases -`main` deploys production. A team that wants a staging gate adds a `dev` branch: `dev` deploys staging, and `main` only moves by a promotion PR from `dev` (the Railway side is `branch`/`branches` in `deploy/platform.yaml`). The deployed branches are listed once, in `scripts/branches.sh`; everything that means "the base" uses the ones that exist, so a repo with only `main` needs no change: +`main` is the only long-lived branch. Every PR targets it; staging runs it and production runs releases tagged on it. The deployed branch is named once, in `scripts/branches.sh` (and `landingBranch` in `scripts/cilocal`): -- CI runs on PRs to any branch and on pushes to `dev` and `main` (full runs, the build cache saved by either). -- `make check-fast`, `make affected` and the bench A/B compare with origin/dev, or origin/main when there is no dev. -- The migration order check needs a new migration above the highest on both (a promotion PR to `main` is checked against `main`, a PR to `dev` against `dev` and `main`'s tip). +- `make ci-local` attests a branch head or (with `COMMIT=`) a commit on origin/main; a new project's first attestation runs `make ci` ([docs/CI.md](docs/CI.md#local-verification)). `Deploy staging` runs on pushes to `main`. +- `make check-fast`, `make affected` and the bench A/B compare with origin/main. +- The migration order check needs a new migration above the highest on origin/main. ## Pull requests @@ -209,7 +220,9 @@ Exit codes: - Conventional commit titles in plain language: `fix(api): PATCH /users no longer drops the image`. - Body: the problem in a sentence or two, then the fix. UI changes need before/after images; motion needs a short video. - One concern per PR. If the description says "also", split it. -- `ci-ok` is the single required check. It is a commit status posted by the CI jobs on the commit they tested (pending, then success/failure), not a job, so skipped or cancelled jobs and drafts can never satisfy it. Require it in branch protection as a status check (source: GitHub Actions). When babysitting CI, verify each finding against the source, fix real ones, and dismiss false positives with a reason. +- How to verify, attest, open, stack and merge, step by step: the [github-prs](.claude/skills/github-prs/SKILL.md) skill. +- Open it from [.github/pull_request_template.md](.github/pull_request_template.md), ready (not draft), then run what it needs and `make ci-local RAN=""` on its pushed head; it fills "Local verification". A new push or rebase needs a new attestation. +- Merge with `make pr-merge PR="..."` only: it squash-merges each PR whose `local-ci` and section hold for its exact head and whose own checks are green (or, where Actions cannot run, on `local-ci` alone), reports the rest without waiting, and retargets PRs stacked on it. After each merge, once its deploy has run, `make pr-merge ARGS="--check-deploy "` (it prints the command): a follow-up, never a gate. The ruleset requires `Secret scan` and `Local verification` ([docs/CI.md](docs/CI.md#required-checks-and-staging)). When babysitting, verify each finding against the source, fix real ones, and dismiss false positives with a reason. ## Where code lives diff --git a/CLAUDE.md b/CLAUDE.md index afd7b40..976e98d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,3 +1,5 @@ # CLAUDE.md Read and follow [AGENTS.md](AGENTS.md). Shared vocabulary is in [CONTEXT.md](CONTEXT.md). + +Pull requests (verify, attest, stack, merge): the [github-prs](.claude/skills/github-prs/SKILL.md) skill. diff --git a/CONTEXT.md b/CONTEXT.md index e7c99ca..43ecf9f 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -34,7 +34,13 @@ Use these words when talking about Go Fast Stack (this repo), in code review, co | service error | a named `*services.Error` value (`services.ErrEmailTaken`), matched with `errors.Is`; kind sentinels (`ErrNotFound`...) match a whole kind | `apps/web/internal/services/` | | mock | mockgen output for a port, generated by `go generate` into `/mocks`, committed, never hand-written | `services/mocks`, `api/mocks` | | mock session | `apitest.AsUser(u)` / `apitest.Anonymous`: who a handler test request is signed in as (admitted by `apitest.Gate()`) | `apps/web/internal/api/apitest/` | -| test boundary | one of the only places tests may live (service, route handler, backend auth), each with fixed allowed dependencies and a coverage floor | AGENTS.md "Test boundaries" | +| test boundary | one of the only places tests may live (service, route handler, backend auth, local verification), each with fixed allowed dependencies and a coverage floor | AGENTS.md "Test boundaries" | +| local verification | the agent runs what its change needs on the exact pushed head, then attests it with `make ci-local RAN=""` (which runs nothing), recorded as the `local-ci` commit status and the PR body's "Local verification" section | `scripts/cilocal/`, docs/CI.md | +| `local-ci` | the commit status `make ci-local` posts on the head it attests; its description carries the result and the section's digest, so a hand-edited section no longer matches | `scripts/cilocal/verify/section.go` | +| claimed | a head `make ci-local` has touched: a `local-ci` on it, or a section naming it. Until then ci.yml's checking job runs as `Awaiting make ci-local` and the required `Local verification` stays "Expected" | `verify.ClaimsHead` | +| attested | a head whose section names it and says what ran, with a `local-ci` success matching the section and no failure ever; a main commit is attested by `local-ci` on it, or on the merged head of the pull request that made it. Only attested commits deploy to staging (Deploy staging) | `verify.LocalVerificationProblems`, `cilocal attest` | +| release | a published `vX.Y.Z` GitHub release on a `main` commit staging already ran; the only way code reaches production | `Deploy production`, docs/RELEASES.md | +| failed head | a head with a failed `local-ci` (`make ci-local FAILED=`): final, never attested again; the fix is a new commit | `verify.FailedHeadRefusal` | | boundary test | a test at a test boundary, e.g. the four cases of a route test (401, 403, 400, 2xx) | `*_test.go` in a boundary dir | | routes config | `AppRoutes` (pages), `APIRoutes` (endpoints) | `apps/web/internal/config/routes.go` | | feature toggles | `config.Features`: email verification, template welcome, Expo dev origins | `apps/web/internal/config/features.go` | diff --git a/Makefile b/Makefile index 38d5084..0ec0a06 100644 --- a/Makefile +++ b/Makefile @@ -63,16 +63,21 @@ check-generated: generate ## Fail if generated code is out of date @git diff --exit-code -- '*.go' || (echo 'generated code is stale: run make generate and commit' && exit 1) @test -z "$$(git status --porcelain --untracked-files=all -- '*.go')" || (git status --short -- '*.go'; echo 'untracked generated files: run make generate and commit'; exit 1) +# What lint and test cover: everything, or the affected packages +# (scripts/affected.sh's dirs and import paths). +LINT_DIRS ?= ./... +TEST_PKGS ?= ./... + .PHONY: lint lint: ## gofmt + golangci-lint (incl. layering rules) + single-character names + test file locations @test -z "$$(gofmt -l .)" || (gofmt -l . && echo 'gofmt: run gofmt -w .' && exit 1) - $(TOOL) golangci-lint run - $(GO) run ./scripts/namelint ./... + $(TOOL) golangci-lint run $(LINT_DIRS) + $(GO) run ./scripts/namelint $(LINT_DIRS) ./scripts/check-test-boundaries.sh ./scripts/check-ui-api-paths.sh .PHONY: check-migrations -check-migrations: ## Fail if a new migration is numbered at or below the highest on origin/dev or origin/main (check-fast and CI run it too) +check-migrations: ## Fail if a new migration is numbered at or below the highest on origin/main (check-fast and CI run it too) ./scripts/check-migration-order.sh .PHONY: lint-layering @@ -85,7 +90,9 @@ lint-tests: ## Prove misplaced test files, coverage drops and slow tests fail (P # go test -json through scripts/testbudget: plain go test output, the slowest # tests, and a failure when a package or test is over its time budget. -GOTEST := set -o pipefail; $(GO) test -race -json -coverprofile=.cache/coverage.out ./... | $(GO) run ./scripts/testbudget +GOTEST := set -o pipefail; $(GO) test -race -json -coverprofile=.cache/coverage.out $(TEST_PKGS) | $(GO) run ./scripts/testbudget +# Coverage floors: all of them, or only those of the packages tested. +COVERAGE := $(if $(filter ./...,$(TEST_PKGS)),,TESTED='$(TEST_PKGS)') ./scripts/check-coverage.sh .cache/coverage.out .PHONY: test test: ## go test -race against this checkout's throwaway Postgres (or TEST_DATABASE_URL), then time budgets and coverage floors @@ -98,10 +105,10 @@ else echo "test db: 127.0.0.1:$$port (project $(COMPOSE_PROJECT_NAME))"; \ TEST_DATABASE_URL="postgres://postgres:postgres@127.0.0.1:$$port/postgres?sslmode=disable" bash -c '$(GOTEST)' endif - ./scripts/check-coverage.sh .cache/coverage.out + $(COVERAGE) .PHONY: ci -ci: ## Run the same checks as CI +ci: ## Every check, on this checkout as it is (make ci-local then attests a PR head) $(GO) build ./... $(MAKE) check-generated $(MAKE) lint test @@ -110,8 +117,29 @@ ci: ## Run the same checks as CI $(MAKE) lint-tests PROFILE=.cache/coverage.out # the probes reuse make test's profile $(MAKE) deploy-check +# make ci-local's statements reach it unexpanded and as single arguments: +# taken raw ($(value)), exported, and quoted by the shell, so a $, a quote +# or a newline in RAN stays what you typed. +override RAN := $(value RAN) +override FAILED := $(value FAILED) +override COMMIT := $(value COMMIT) +export RAN FAILED COMMIT + +.PHONY: ci-local +ci-local: ## Attest the pushed PR head after running what it needs: make ci-local RAN="make lint test TEST_PKGS=./x" (FAILED="", COMMIT=
); runs nothing, posts local-ci, fills the PR body + $(GO) run ./scripts/cilocal run $${RAN:+--ran "$$RAN"} $${FAILED:+--failed "$$FAILED"} $${COMMIT:+--commit "$$COMMIT"} + +.PHONY: pr-merge +pr-merge: ## Squash-merge PRs attested on their exact head: make pr-merge PR="12 13" (ARGS="--dry-run|--any-base|--require-actions|--then-check-staging"); ARGS="--check-deploy |--check-staging" after a merge + @test -n "$(PR)$(ARGS)" || (echo 'usage: make pr-merge PR="..." [ARGS=...], or ARGS="--check-deploy |--check-staging"' && exit 2) + $(GO) run ./scripts/cilocal merge $(ARGS) $(PR) + +.PHONY: secret-scan +secret-scan: ## gitleaks (pinned) on the files changed since the merge base, or every tracked file with ARGS=--all + $(GO) run ./scripts/cilocal secret-scan $(if $(ARGS),$(ARGS),--base $$(git merge-base HEAD $$(. scripts/branches.sh && landing_ref))) + .PHONY: check-fast -check-fast: ## Before every push: gofmt, vet, golangci-lint, plain tests for only what your changes vs origin/dev (else origin/main) affect (seconds) +check-fast: ## Before every push: gofmt, vet, golangci-lint, plain tests for only what your changes vs origin/main affect (seconds) ./scripts/check-fast.sh .PHONY: release-smoke @@ -130,7 +158,7 @@ hooks-remove: ## Remove the pre-push hook make hooks installed ./scripts/install-hooks.sh --uninstall .PHONY: affected -affected: ## Show what CI checks for this branch (packages affected vs origin/dev, else origin/main) +affected: ## Show which packages this branch affects vs origin/main (what to lint and test before make ci-local) ./scripts/affected.sh .PHONY: bench-devloop @@ -307,7 +335,7 @@ DEPLOY_READS := -reads 'web=*,!ADMIN_*,!BACKUP_*' \ DEPLOY_KEEP ?= .PHONY: deploy-check -deploy-check: ## vet and lint the deploy/ module (it has no tests), then deploy-vars +deploy-check: ## vet and lint the deploy/ module (it has no tests: AGENTS.md "Test boundaries"), then deploy-vars $(GO) build -o bin/namelint ./scripts/namelint cd deploy && $(GO) vet ./... && (cd .railway && $(GO) vet railway.go) && $(GO) tool -modfile=../tools/go.mod golangci-lint run -c ../.golangci.yml ./... && ../bin/namelint ./... $(MAKE) --no-print-directory deploy-vars diff --git a/README.md b/README.md index 3526237..ca7c205 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,7 @@ Pages are rendered on the server in Go, and [Datastar](https://data-star.dev) ad 7.6 MB of the binary is the pure-Go image codecs used by the optional image uploads ([docs/storage.md](docs/storage.md#pure-go-static-binary)). The binary and image sizes come from a production build (`make build`: `-tags nodynamic -trimpath -ldflags='-s -w'`, Go 1.26.8, linux/amd64); the memory and page numbers come from the same kind of build before the codecs were added. The Railway figure is from the [deploy cost notes](docs/deploy.md#cost). -**Fast CI and fast tests.** CI checks only the packages a change can affect, in one job. +**Fast verification and fast tests.** The agent checks only the packages a change can affect, on its own machine, and `make ci-local` attests what it ran on the PR in seconds ([docs/CI.md](docs/CI.md)). The CI times below were measured on the GitHub job that ran the same checks before they moved there. | | Time | |---|---| @@ -50,7 +50,7 @@ Pages are rendered on the server in Go, and [Datastar](https://data-star.dev) ad | Full test suite with the race detector, 2-vCPU runner | 29 s median ([#74](https://github.com/jayleaton/go-fast-stack/pull/74)) | | `make check-fast` (local, before every push) | under 1 s for docs, under 10 s for a full warm run | -Every test runs in parallel, and none may sleep. Lint enforces both, and CI fails any package that takes over 60 s or any single test over 10 s. +Every test runs in parallel, and none may sleep. Lint enforces both, and `make test` fails any package that takes over 60 s or any single test over 10 s. ## Cost @@ -64,7 +64,7 @@ Every test runs in parallel, and none may sleep. Lint enforces both, and CI fail These figures come from idle usage measured on Railway on 2026-10-22. Traffic, egress and backups come on top. Nightly backups of a template-sized database cost fractions of a cent. Details: [docs/deploy.md](docs/deploy.md#cost) and [docs/backups.md](docs/backups.md). -**CI minutes.** GitHub bills every CI job rounded up to a whole minute, so the template runs one job per PR, and drafts run nothing: +**CI minutes.** GitHub bills every job rounded up to a whole minute. The checks now run on the agent's machine before `make ci-local` attests them, so a PR costs two small jobs (`Secret scan` and `Local verification`, about a minute each) and a merge to `main` the staging deploy (`Deploy staging`: [docs/RELEASES.md](docs/RELEASES.md)). Before that, one job per PR: | | Before | Now | |---|---:|---:| @@ -78,9 +78,9 @@ Sources: [#54](https://github.com/jayleaton/go-fast-stack/pull/54) and [#67](htt Agents do their best work when the rules are checked by machines, not by review. This template makes the right path the easy one and the wrong path fail fast: - **Strict layers, enforced by lint.** Database → services → transports (JSON API and HTML pages). A handler that reaches for the database, or a service that imports HTTP, fails `make lint` (depguard). The agent can't route around the architecture, and `make lint-layering` proves the rules still catch planted violations. -- **Tests only where the business logic is.** Tests are allowed only in services, route handlers and the auth package. A script rejects a test file anywhere else, and each of those places has a coverage floor. Agents don't spend time writing tests for UI or wiring. -- **Generated mocks.** Mocks are generated from the interfaces (`make generate`), never written by hand. CI fails when one is stale. -- **A pre-push check that takes seconds.** `make check-fast` runs format, vet, lint and tests on only what the change affects. Agents run it on every push, even on machines where a full CI run is too heavy. +- **Tests only where the business logic is.** Tests are allowed only in services, route handlers, the auth package and the rules of local verification. A script rejects a test file anywhere else, and each of those places has a coverage floor. Agents don't spend time writing tests for UI or wiring. +- **Generated mocks.** Mocks are generated from the interfaces (`make generate`), never written by hand. `make check-generated` fails when one is stale. +- **A pre-push check that takes seconds.** `make check-fast` runs format, vet, lint and tests on only what the change affects. Agents run it on every push, even on machines where a full run is too heavy, and `make ci-local` attests each pushed PR head before it merges. - **Written instructions.** [AGENTS.md](AGENTS.md) holds the architecture, test rules and PR rules, and [CONTEXT.md](CONTEXT.md) holds the shared vocabulary. Agents and humans read the same rules and use the same words. - **Parallel agents never collide.** Each clone or git worktree gets its own database and ports, derived from its path. Any number of agents can run `make db-up` and `make test` at once. - **Errors that name the fix.** Lint messages say what to do instead ("transports call services; only packages/database talks to Postgres"). The server refuses to boot with bad config and lists every problem. `make railway-doctor` prints the fix for each missing deploy prerequisite. @@ -102,6 +102,8 @@ make dev # http://:3000 Then open the site and sign up. In development, verification and reset links are printed in the server log instead of emailed. To try the API with `curl`, see [docs/development.md](docs/development.md#try-the-api). +A new project made from this template attests its first `main` commit once: `make ci`, then `make ci-local RAN="make ci"` on `main` ([docs/CI.md](docs/CI.md#local-verification)). After that every pull request is attested on its head. + To deploy, run `make railway-setup`. It creates the whole Railway project from [`deploy/platform.yaml`](deploy/platform.yaml) and checks that it is healthy ([docs/deploy.md](docs/deploy.md)). ## Learn more diff --git a/apps/mobile-app/README.md b/apps/mobile-app/README.md index 35a568e..330d343 100644 --- a/apps/mobile-app/README.md +++ b/apps/mobile-app/README.md @@ -65,8 +65,8 @@ clicked. (`apps/web/internal/config/routes.go`) from this checkout and fails if any path in `ApiRoutes` is missing there. `ApiRoutes` therefore uses the same patterns as Go (`/api/core/v1/users/{id}`); build concrete paths with the -helpers in `routes.ts`. It runs in plain `pnpm test`, so CI catches a renamed -endpoint without Docker. CI runs the mobile job when that Go file changes too. +helpers in `routes.ts`. It runs in plain `pnpm test`, so the mobile checks catch a renamed +endpoint without Docker. Run them before attesting a change to that Go file too (docs/CI.md, "What to run"). ### Contract test diff --git a/deploy/cmd/deploy-services/main.go b/deploy/cmd/deploy-services/main.go new file mode 100644 index 0000000..202117d --- /dev/null +++ b/deploy/cmd/deploy-services/main.go @@ -0,0 +1,36 @@ +// Command deploy-services prints the Railway app services platform.yaml +// gives an environment, in the order a deploy rolls them out +// (.github/scripts/deploy-railway.sh): web first, whose pre-deploy command +// migrates the app database, then admin (its own migration) when admin is +// on, then the backup cron when backups are on. It reads no network and no +// credentials; Deploy staging and Deploy production run it, so the +// workflows follow platform.yaml instead of naming services themselves. +// +// deploy-services -config platform.yaml # "web admin backup" +// deploy-services -config platform.yaml -public # "web admin": the services with a /healthz +package main + +import ( + "flag" + "fmt" + "os" + "strings" + + "github.com/jayleaton/go-fast-stack/deploy/platform" +) + +func main() { + config := flag.String("config", "platform.yaml", "path to platform.yaml") + public := flag.Bool("public", false, "only the services with a public /healthz") + flag.Parse() + cfg, err := platform.Load(*config) + if err != nil { + fmt.Fprintln(os.Stderr, "deploy-services:", err) + os.Exit(1) + } + services := cfg.PublicServices() + if !*public && cfg.Backups.Enabled { + services = append(services, platform.ServiceBackup) + } + fmt.Println(strings.Join(services, " ")) +} diff --git a/deploy/platform.yaml b/deploy/platform.yaml index 8d0b03f..a9ba07f 100644 --- a/deploy/platform.yaml +++ b/deploy/platform.yaml @@ -14,29 +14,26 @@ project: auto # belong to more than one). workspace: "" -# GitHub repo Railway builds on push. auto: this checkout's origin remote. -# Until the Railway GitHub app can read the repo, setup deploys your -# checkout by upload instead and prints the install link; the next make -# railway-apply after installing switches to deploy-on-push. -# "" never connects GitHub: deploy with make railway-up. -repo: auto -# The branch production deploys (and every environment, when production is -# the only one). +# How code reaches Railway. "" (the default): never through Railway's GitHub +# integration. GitHub Actions deploy every environment by upload instead +# (docs/RELEASES.md): Deploy staging ships each attested merge to main, +# Deploy production ships a published release. By hand: make railway-up. +# auto (this checkout's origin) or owner/repo turns Railway's own +# deploy-on-push back on, from the branches below, beside the workflows; +# leave it off so only attested commits deploy. +repo: "" +# With repo set: the branch production deploys, and per environment its own +# (branches., default dev). Unused in upload mode. branch: main -# With more environments than production, staging and qa deploy dev: merge -# to dev to ship to staging, then dev -> main to release. Override per -# environment (production may not share a branch with another environment): -# branches: -# staging: develop -# qa: release # The admin dashboard (apps/admin) as its own service with its own small # Postgres. Adds two services per environment. admin: false # production | staging,production | staging,qa,production -# Production-only is the cheapest start; teams usually want staging,production. -environments: production +# The release pipeline (docs/RELEASES.md) needs staging: staging always runs +# the latest main, and production only takes a release staging already ran. +environments: staging,production # Railway PR environments: a temporary copy of staging per pull request # (needs staging in environments; a preview never copies production). Off by diff --git a/docs/CI.md b/docs/CI.md new file mode 100644 index 0000000..455a645 --- /dev/null +++ b/docs/CI.md @@ -0,0 +1,89 @@ +# CI + +The checks a change needs run on the agent's machine before it is attested ([Local verification](#local-verification)); `make ci-local` itself runs nothing. GitHub runs no verification on `main`, and on a pull request only two small jobs that check the attestation. Actions minutes are billed per job, rounded up to the whole minute, and a PR's checks cost about two of them. + +## What runs + +`CI` (`.github/workflows/ci.yml`) runs on pull request edits alone (`pull_request_target`, [below](#trusted-code-only)). `make ci-local` posts the `local-ci` commit status and then fills the PR body, and that edit starts the run that checks both; a push or a new PR starts nothing. Until `make ci-local` claims a head (a `local-ci` status on it, or a section naming it: `verify.ClaimsHead`), an edit's run names its checking job `Awaiting make ci-local`, so the required check shows "Expected", never red, and merging stays blocked (a job skipped by `if` would count as passed, which is why it is a rename). Red is a real problem: a failed `local-ci`, or a hand-edited section for this head. Its jobs: + +- `Secret scan` (required): the PR's changed files with gitleaks ([below](#secret-scan)). It does not depend on the agent's machine. +- `Local verification claim`: whether `make ci-local` has claimed the head (`cilocal check --claim`). +- `Local verification` (required; `Awaiting make ci-local` until claimed): the `local-ci` status and the PR body's section on the head (`cilocal check`). + +### Trusted code only + +The required checks run only the code on `main`, never the pull request's. On `pull_request_target` GitHub takes the workflow from the base branch and checks out the base, so a branch or fork that rewrites `scripts/cilocal` (a Go `init()` that exits 0), `.gitleaks.toml` or `ci.yml` itself changes nothing the jobs do until it merges. They read the head only as data: its body and statuses through the API, and its files from git's objects (`cilocal secret-scan --head `), never a checkout of it. A fork's head fails `Local verification`, `make pr-merge` and the staging gate: `make ci-local` attests only a branch pushed to this repository, and a fork's author writes its body. `verify.TrustProblems` holds `ci.yml` to this, and a verify test fails on a `pull_request` trigger, a checkout of a ref or repository, an expression inside a `run` script, or a scan or check that does not read the head as data. A change to `ci.yml`, the scanner or its config therefore takes effect on the pull requests after it merges, not on its own. + +A newer edit cancels the PR's older run (`ci-pr-`). The jobs run on the `CI_RUNNER` repository variable, a JSON `runs-on` value (`'["self-hosted","linux"]'`), or GitHub-hosted `ubuntu-latest` without it. They build `scripts/cilocal`, which imports only the standard library, so they download no modules. + +A push to `main` runs only `Deploy staging` (`deploy-staging.yml`), which deploys a commit `make ci-local` attested to Railway staging ([Required checks and staging](#required-checks-and-staging)); production runs releases ([RELEASES.md](RELEASES.md)). `Post-merge report` comments on the merged PR when a push workflow fails on `main`. `Bench dev loop` still runs on `bench/**` pushes and by hand. + +### What to run + +Run what your change needs on the head you attest, and nothing more. `scripts/affected.sh` (`make affected`) prints the Go dirs and packages a change can affect; `make check-fast` is the seconds-long pass over them. + +| Changed | Run | +|---|---| +| Any file | `make secret-scan` (gitleaks on what the branch changed); `gofmt -l` on changed Go files | +| Go packages | `make lint LINT_DIRS=` (gofmt, golangci-lint with the layering rules, namelint, the test-boundary and UI API-path checks) and `make test TEST_PKGS=` (`-race`, a throwaway Postgres, time budgets, the tested packages' coverage floors); `make check-fast` for a quick pass | +| sqlc queries, templ, mocks or other `go:generate` inputs | `make check-generated` | +| A migration | `make check-migrations` | +| `deploy/` or `deploy/platform.yaml` | `make deploy-check` (vet, lint and names in the `deploy/` module, then `make deploy-vars`) | +| `.github/workflows/` or `.github/actions/` | `make actionlint` | +| A Dockerfile, or code a Dockerfile builds | `make release-smoke` | +| `apps/mobile-app`, or the route table its test reads (`apps/web/internal/config/routes.go`) | its pnpm checks in `apps/mobile-app`: `pnpm install --frozen-lockfile`, `tsc`, `expo lint`, `expo install --check`, `pnpm test` | +| Tooling, the Makefile, lint config, or everything | `make ci` (every check on the checkout, the probes `make lint-layering lint-tests` included) | + +The probes (`make lint-layering`, `make lint-tests`) prove the layering and test rules still bite; run them when you change those rules or the scripts behind them. Say exactly what ran in the attestation: a reviewer reads the scope from it. + +## Local verification + +`make ci-local` (`scripts/cilocal run`) runs no checks: it tells the PR you ran what CI would have checked, and says what that was. Run what the change needs ([What to run](#what-to-run)), commit, push, then attest: + + make ci-local RAN="make lint test LINT_DIRS=./apps/web/... TEST_PKGS=./apps/web/internal/services" + +- **What it checks.** A clean worktree whose HEAD is the pushed branch head (`Head.Refusal`), and no failed `local-ci` ever posted on that head. It reads the checkout again just before it posts, and posts only for exactly that clean, pushed head and tree (`verify.PublishRefusal`). It takes seconds: no worktree checkout, no slot, no queue. +- **What it posts.** The `local-ci` status on that exact SHA, whose description carries a digest of the section (` · body · by on `), then the body's "Local verification" section (`scripts/cilocal/verify`, template [.github/pull_request_template.md](../.github/pull_request_template.md)): head, tree, who attested where, and the statement (`RAN` may span lines; they are joined with `; `). The edit starts the PR run that checks both; when the body was already current it re-runs that run (its failed jobs, or all of it when it ran only as awaiting). Before the PR exists it posts the status only: open the PR from the template and run it again. +- **A failure.** `make ci-local FAILED=""` posts `failure`, final for that head. A failure in what the branch changed is fixed in the PR; any other is fixed forward in its own PR off `main` with a regression test that fails before the fix, and the branch rebases on it. Never attest a head until green. +- **Enforced.** The `Local verification` job (`cilocal check`), `make pr-merge` and the staging gate (`cilocal attest`) refuse a PR unless `local-ci` is `success` on its exact head, the section names that head and says what ran, it matches the status digest, and the head never failed (`verify.LocalVerificationProblems`). A new push or rebase needs a new attestation. +- **Main commits.** `make ci-local COMMIT= RAN="…"` in a clean detached checkout attests a `main` commit after the fact (an admin merge, a direct push). It reads the commit's first-parent ancestors for the last one the staging gate accepts (at most 50) and says the range the statement covers. +- **A new project's first attestation.** A repository made from this template has no attested commit in its own statuses, so nothing narrows what changed: run `make ci` (every check, the whole tree) on `main`'s head, then `make ci-local RAN="make ci"` on `main` (or `COMMIT=` in a detached checkout of it). With no attested main commit before the one it attests, `make ci-local` refuses a statement that does not name `make ci` (`verify.FirstAttestationProblem`); a root commit is fine. From then on each merge is attested on its PR head. +- **Trust.** It is a signoff: someone with write access says what they ran. Nothing on GitHub runs the checks again, so a wrong one shows up on staging, which is why the merging agent checks the deploy. + +## Secret scan + +`cilocal secret-scan` copies the files a change touches (the branch's diff since the merge-base in `make secret-scan`; the pull request's file list in the `Secret scan` job, or every file of the head when GitHub cannot list them, read from the head's objects: `--head`) and scans them with gitleaks 8.30.1, downloaded once into the user cache directory and checked against its release SHA-256. A finding fails the step or job, printing the rule, file and line with the value redacted. + +[.gitleaks.toml](../.gitleaks.toml) keeps gitleaks' own rules and allows only the dev-loop bench script's known synthetic values: a rule-scoped allowlist for `generic-api-key` that pairs the script's exact path with those exact values (`condition = "AND"`), never a path on its own. Tests, `testdata` and example env files get no exemption, so a real credential in one fails the scan. Over the whole tree it reports nothing (`make secret-scan ARGS=--all`). A new deliberate fixture gets its own obviously fake value and the same kind of exact entry; `scripts/cilocal/verify`'s `TestScannerPolicyAllowlistsValuesOnly` fails a blanket path allowlist. A real secret is removed and rotated, because history keeps it. **The scan's policy is never the change's.** `.gitleaks.toml` and `.gitleaksignore` (`verify.ScannerPolicy`) come only from the trusted commit: `main` in the `Secret scan` job ([Trusted code only](#trusted-code-only)), the merge-base in `make secret-scan`, with gitleaks' default rules and an empty ignore list where it has none. They are written outside the scanned directory, because gitleaks loads `.gitleaksignore` from the root of what it scans whatever `--gitleaks-ignore-path` says. The change's own copies of those files are scanned as content under another name, inline `gitleaks:allow` comments are ignored (`--ignore-gitleaks-allow`), and so are `GITLEAKS_*` variables. A change that adds, edits or removes a policy file fails the scan even when it finds nothing (`verify.PolicyProblem`): the new policy was never checked, so a maintainer reviews it and merges it on its own, past the failed check. + +## Required checks and staging + +The main ruleset should require `Secret scan` and `Local verification` from GitHub Actions on every pull request, and nothing else from CI. A pull request can add a workflow of its own whose job takes a required check's name; requiring the `local-ci` status as well closes that, because only someone with write access can post it, and `make pr-merge`, which runs on the merger's machine, makes the same checks itself. It need not require a head to be up to date with its base, and neither does `make pr-merge`: only a conflict stops it. A repository whose Actions cannot run (billing, or turned off) requires the `local-ci` status itself instead ([below](#when-actions-cannot-run)). + +**Staging deploys only what `make ci-local` attested.** Railway's GitHub auto-deploy is off (`repo: ""` in `deploy/platform.yaml`): `Deploy staging` runs on every push to `main`, and its plan job (`cilocal attest `) lets the rollout start only when either: + +- the pushed commit itself has a successful `local-ci` status (`make ci-local COMMIT= RAN="…"` in a clean worktree of it posts it), or +- the pull request whose squash made it (by GitHub's commit-to-PR index, or its `(#NNN)` squash title) passed local verification on the merged head. + +A direct push, or the squash of an unattested head, deploys nothing, and the hourly `Staging catch-up` keeps refusing it until it is attested. Production takes only a release staging already ran ([RELEASES.md](RELEASES.md#what-is-gated)). **The merged tree of two verified PRs is not itself verified**: a head behind its base squashes into a tree nobody ran; a clash surfaces on staging and is fixed forward. An unattested `main` is the top priority: `git worktree add --detach `, run what the commits since the last attested one need (`make ci` when there is none, as in a [new project](#local-verification)), then `make ci-local COMMIT= RAN="…"` there; then dispatch `Deploy staging` (or wait for the catch-up). + +## Merging on local verification + +Pull requests open ready and stack. Each merges the moment it is attested, conflict-free and green, with `make pr-merge PR="..."` (`cilocal merge`; the skill [github-prs](../.claude/skills/github-prs/SKILL.md)), which squashes exactly the attested head with `--match-head-commit` and never waits. Per PR it decides (`verify.Decide`): merged is done; closed, draft (`gh pr ready `), stacked on another branch (unless `ARGS=--any-base`), conflicting, not attested, a failed check, a hollow green (`Local verification` passed but `Secret scan` skipped) or a pending one (including `Local verification` not reported yet) is skipped with its next step, and the others still merge. Only the newest run of each workflow, and the newest attempt of each job, counts. Before merging it retargets the open PRs stacked on the branch to `main` and prints the `git rebase --onto` for each. `ARGS=--dry-run` says what it would do. + +- **Never wait** on another PR, its verification or a staging deploy. Behind `main` is fine; a conflict means rebase, push and attest again. +- **After the merge, check its deploy.** `make pr-merge` prints the squash commit and `make pr-merge ARGS="--check-deploy "`. Once `Deploy staging` has run for it, run that once: one read of that commit's `Deploy staging` runs and of `/healthz` (200 with `data.status` `ok`; `STAGING_WEB_URL` from the env, else the repository variable). A failed or refused deploy is the top priority, fixed forward; one cancelled by a newer `main` commit is checked on main's head (`ARGS=--check-staging`); a dry run (`RAILWAY_DEPLOY` is not `true`) says nothing was deployed. It is a follow-up, not a gate: keep merging. `ARGS=--then-check-staging` follows main's deploy (bounded) after a burst. `Post-merge report` comments on the PR when its deploy fails. +- **Ruleset first.** `make pr-merge` calls `gh pr merge`, so the ruleset (required reviews, required checks) still applies on top of it. Never `--delete-branch`: it closes the PRs stacked on the branch. + +### When Actions cannot run + +On a repository whose GitHub Actions do not start (a spent or blocked billing budget, or Actions turned off), no workflow runs: not `CI`, and not `Deploy staging` or `Deploy production`. Then: + +- `make pr-merge` merges on `local-ci` alone when the head has no CI job that ran, either no CI run at all or only jobs GitHub never started, and says so (`ARGS=--require-actions` refuses instead). A blocked account still gets runs: GitHub fails each job within seconds, with no runner and the annotation "The job was not started because recent account payments have failed or your spending limit needs to be increased". Those jobs are read as not run, never as a verdict. A job that did run and failed still refuses. The checks it makes on the attestation are the ones `Local verification` would have made, from the same code. +- Require the `local-ci` commit status in the ruleset (it is posted by `make ci-local`, not by Actions), not `Secret scan` or `Local verification`, which would wait forever. +- Nothing deploys on its own. Deploy by hand with `make railway-up PLATFORM_ENV=staging` from a clean checkout of `main` after `go run ./scripts/cilocal attest ` passes, and to production only a release staging ran ([deploy.md](deploy.md)). Do not turn Railway's GitHub auto-deploy back on to work around it: that deploys unattested commits. +- `CI_RUNNER` pointed at a self-hosted runner brings every workflow back without hosted minutes (`RELEASE_RUNNER` overrides it for the release jobs). + +## Watching CI + +One read per question: `gh pr checks ` or `gh run view --json jobs` after its expected completion, never a polling loop. Fetch logs only for a concrete failure. `Post-merge report` comments on the merged PR when a push workflow fails on `main`. diff --git a/docs/RELEASES.md b/docs/RELEASES.md new file mode 100644 index 0000000..4b5a0fb --- /dev/null +++ b/docs/RELEASES.md @@ -0,0 +1,194 @@ +# Releases and production deploys + +Staging always runs the latest `main`. Production runs a **release**: a +`vX.Y.Z` tag on a commit staging has already run. The standing rules are in +`AGENTS.md`, "Environments: staging is always current"; this page is how the +pipeline enforces them. + +The layout it expects is one Railway project with two environments, `staging` +and `production` (`deploy/platform.yaml`: `environments: staging,production`), +created once with `make railway-setup` ([deploy.md](deploy.md)). Railway's own +GitHub auto-deploy stays off (`repo: ""`): only these workflows ship code. + +## The flow + +1. **Merge to `main`.** `make ci-local` attested the PR's head + ([CI.md](CI.md)), then `Deploy staging` checks that attestation and ships + the commit to Railway staging (see [Railway rollout](#railway-rollout)). +2. **Draft a release.** Actions → **Draft release** → Run workflow (`bump` + patch, minor or major, or an exact `version`). It drafts a release at the + commit staging runs now, with notes generated from the pull requests merged + since the last release. Nothing is tagged yet. Edit the notes or the version + if you like. +3. **Publish the release.** On the draft, click **Publish release**. That + creates the tag and starts `Deploy production`. Publishing is the explicit + yes every production deploy needs, so only people in `PRODUCTION_RELEASERS` + can do it in a way that deploys. The run's gate checks the release (see + [What is gated](#what-is-gated)) and refuses anything it cannot prove. + An agent may do this step only when the developer explicitly tells it to + publish or deploy (`gh release edit vX.Y.Z --draft=false` with an + allowlisted account), after confirming staging ran that commit and is + healthy, and adds "Authorized by , published by " to the + release notes. + +Required reviewers on an environment exist only for public repositories or on +paid plans. Where they are available, add them to the `production` environment +and the deploy job also waits for an approval; nothing else changes. + +Without the helper, step 2 is one command (the commit must be the one staging +runs, shown in the title of the latest successful `Deploy staging` run): + +```sh +gh release create v1.4.0 --draft --target --generate-notes +``` + +### Railway rollout + +Staging and production run the same script, `.github/scripts/deploy-railway.sh`, +which is the authority for the order. It uploads the checkout with +`railway up --service ` (nothing is built on GitHub) and waits for each +deployment before the next: + +- `web`, whose pre-deploy command migrates the app and auth databases + (`/server migrate`); a failed migration or `/healthz` stops the deploy with + the old deployment still serving. +- `admin` when `deploy/platform.yaml` enables it (`/admin migrate` first). +- `backup` when backups are on (a cron service: it counts as deployed once + Railway has it `SLEEPING` until its schedule). + +The services come from `deploy/cmd/deploy-services`, which reads +`deploy/platform.yaml`. Then the readiness check: `/healthz` (and +`/healthz` when set) must answer 200 with +`{"data":{"status":"ok"}}`. `DRY_RUN=1` prints every +command instead of running it. + +The token is a Railway project token scoped to one environment, so no +`--project` or `--environment` flag is passed: the token picks both. + +### Dry run + +Until the repository variable `RAILWAY_DEPLOY` is `true`, `Deploy staging` +skips its deploy job and runs `Dry run (RAILWAY_DEPLOY is not true)`, which +prints the rollout. A fresh copy of the template therefore goes green without +a Railway project. A dry run is never proof for production: the gate reads the +`Deploy main to Railway staging` job, so `Deploy production` refuses every +release until staging has really deployed. + +## What is gated + +`Deploy staging` refuses a commit `make ci-local` did not attest +(`cilocal attest`: `local-ci` passed on the commit, or on the exact head of the +pull request whose merge made it), and a dispatch from anything but `main`. + +`Deploy production` (`.github/scripts/production-gate.sh`) refuses, before any +production step, unless all of these hold. Every check runs and reports, so the +run summary shows everything that is wrong at once. Anything that cannot be +proven counts as a failure. + +| Check | Why | +|---|---| +| The ref is a `vX.Y.Z` tag, resolved through `refs/tags/` | No branch or ad-hoc commit deploys | +| It is a published release, not a draft or pre-release | Pre-releases are skipped entirely | +| A dispatch was started from `main` | A dispatch runs the workflow file from its ref; only main's copy carries this gate | +| The person publishing or dispatching is in `PRODUCTION_RELEASERS` | One allowlist, kept in a repository variable | +| The `production` environment exists and admits only `main` and `v*`; `staging` admits only `main` | GitHub creates a missing environment unrestricted on first use; this stops that | +| An active tag ruleset restricts creating, moving and deleting `v*` | A tag runs the workflow file at its own commit, so only the people who may release may make one | +| The tag's commit is on `main` | No hotfixes straight to production | +| A successful `Deploy staging` run deployed that exact commit | Production only runs a commit staging ran and passed readiness on (run title, the run's success, and its `Deploy main to Railway staging` job) | +| Staging still runs that commit or a later `main` commit | Production may lag staging, never lead it | +| `STAGING_WEB_URL/healthz` answers 200 right now | Staging is healthy at release time | +| The tag still points at the same commit when the deploy job starts | A queued deploy starts later than its gate; a moved tag means what was checked would not ship | + +Runs never overlap: the workflow uses the `deploy-production` concurrency group +without cancelling. A run started while another is running queues behind it; +GitHub keeps only the newest queued run. + +The gate runs read-only from a laptop too (`gh` logged in): +`TAG=v1.0.0 SHA=$(git rev-parse origin/main) sh .github/scripts/production-gate.sh`. + +### Deploy credentials + +The gate lives inside `deploy-production.yml`, and a workflow always runs the +file at its own ref. So what keeps the production token away from a pushed +workflow is not the gate but these repository settings, which no workflow file +can change: + +| Setting | What it admits | Why | +|---|---|---| +| Environment `production` | Branch `main`, tag pattern `v*`. Holds its `RAILWAY_TOKEN` (scoped to production) | A dispatch runs from `main` (reviewed through its ruleset); a release runs from its `v*` tag | +| Environment `staging` | Branch `main` only. Holds its `RAILWAY_TOKEN` (scoped to staging) | `Deploy staging` runs from `main` (its push trigger and catch-up's dispatch both do), so no branch workflow can read the token or put branch code on staging | +| Tag ruleset "Release tags" (`refs/tags/v*`) | Creation, update and deletion restricted; bypass: the repository Admin role | The environments admit `v*` tags, so only releasers may make one | + +The gate fails closed when an environment or the ruleset is missing or +different. What these settings do not stop: a releaser's own credentials. A +leaked admin token can push a `v*` tag as easily as it can publish a release. + +## Staging catch-up + +`Staging catch-up` runs hourly (and by hand). When the latest successful +`Deploy staging` run's commit is not `main`'s head (a refused, failed or +cancelled deploy), it dispatches `Deploy staging` on `main`. A `main` that is +not attested is refused again, every hour, until someone attests it +([CI.md](CI.md#required-checks-and-staging)). + +## Rollback or redeploy + +Actions → **Deploy production** → Run workflow, branch `main`, and enter an +existing release tag (for example the previous one). The same gate applies, and +dispatching is the explicit yes, so only people in `PRODUCTION_RELEASERS` can. +An older release passes the staging checks as long as staging once ran its +commit and still runs that commit or a later one. + +Migrations only move forward. Rolling back the code does not undo a migration, +so a release whose migration the older code cannot run against needs a forward +fix instead (merge it, let staging deploy it, release again). + +## One-time setup + +The pipeline is inert until this is done: staging dry-runs, and every +production run stops at the gate with an error naming what is missing. + +1. **Create the project:** `make railway-setup ARGS="--environments staging,production ..."` + (AGENTS.md, "Deploy to Railway"), with `repo: ""` in `deploy/platform.yaml`. +2. **Environments and the tag ruleset:** `.github/scripts/release-settings.sh` + (an administrator's `gh` login) creates `staging` (branch `main`) and + `production` (branch `main`, tag `v*`) and the "Release tags" ruleset, + idempotently. It never touches secrets. Leave "Allow administrators to + bypass configured protection rules" off. +3. **Secrets**, each a Railway project token created for that environment of + the project (the token form's environment picker must say so): + `gh secret set RAILWAY_TOKEN --env staging` and + `gh secret set RAILWAY_TOKEN --env production` (each reads the value from + stdin). The name is the same; the environment picks the token. Never as a + repository secret, which every branch can read. +4. **Repository variables** (Settings → Secrets and variables → Actions → + Variables): + - `PRODUCTION_RELEASERS`: comma-separated GitHub logins who may publish a + release that deploys, dispatch a redeploy or rollback, and run Draft + release. + - `STAGING_WEB_URL`, `PRODUCTION_WEB_URL` (and `STAGING_ADMIN_URL`, + `PRODUCTION_ADMIN_URL` with the admin app): the public origins + `deploy-railway.sh` checks `/healthz` on. + - `RAILWAY_DEPLOY` = `true` last, once the rest is in place. + - Optional: `RAILWAY_PROJECT_ID`, used only by a deploy without a token + (a person logged in locally); a project token names its project itself. + - Optional: `CI_RUNNER` and `RELEASE_RUNNER`, a JSON runner label (default + `"ubuntu-latest"`). + +## Where it lives + +| File | Role | +|---|---| +| `.github/workflows/deploy-staging.yml` | Attestation, then the staging rollout on every push to `main` (or the dry run) | +| `.github/workflows/staging-catch-up.yml` | Hourly: dispatches `Deploy staging` when staging is behind `main` | +| `.github/workflows/draft-release.yml` | Drafts a release at the commit staging runs | +| `.github/workflows/deploy-production.yml` | Gate, then the production rollout | +| `.github/scripts/deploy-railway.sh` | The services in order, each waited for, then readiness checks | +| `.github/scripts/production-gate.sh` | The production gate | +| `.github/scripts/release-lib.sh` | Allowlist and staging proof shared by the gate and Draft release | +| `.github/scripts/release-settings.sh` | One-time environments and tag ruleset | +| `deploy/cmd/deploy-services` | Which services `deploy/platform.yaml` deploys, in order | + +The `Deploy staging` run title (`Deploy staging `) and its job name +(`Deploy main to Railway staging`) are what the gate reads as proof, so keep +them unchanged. diff --git a/docs/deploy.md b/docs/deploy.md index 487ddca..9a9e60f 100644 --- a/docs/deploy.md +++ b/docs/deploy.md @@ -16,7 +16,9 @@ The defaults come from the checkout: the project is named after your repo, and p `make railway-plan` shows what would change without touching anything (exit 2 when something would). `make railway-verify` checks `/healthz` on every public URL. `make railway-rotate-db` gives every Postgres a new superuser password (see below). Agents: see AGENTS.md "Deploy to Railway (for agents)" for `--yes`, `--json` and exit codes. -**Code source.** With `repo: auto` (the default: this checkout's `origin`), Railway builds each environment from its own branch once its GitHub app can read the repo: production from `branch` (`main`), and staging and qa from `dev` (or `branches.`), so a merge to `dev` ships to staging and a `dev` → `main` merge releases to production. A production-only project deploys `main`. Setup prints which environment deploys which branch, and if a branch does not exist on GitHub yet it says so and lists `git push origin main:dev` as a next step (Railway's trigger is set anyway and fires on the first push). Until then, setup still creates everything and deploys your checkout by upload (`railway up`), then prints the [Railway GitHub app](https://github.com/apps/railway-app) install link for your account. Install it, pick the repo, and re-run `make railway-setup` to switch every service to deploy-on-push (no duplicate services). With `repo: ""`, setup never connects GitHub: deploy new code with `make railway-up` (`PLATFORM_ENV=staging` for one environment). +**Code source.** With `repo: ""` (the default), setup never connects Railway's GitHub integration, so Railway never deploys on a push. GitHub Actions ship the code instead, by upload with environment-scoped project tokens: `Deploy staging` sends every attested merge to `main` to staging, and `Deploy production` sends a published release staging already ran to production ([RELEASES.md](RELEASES.md)). Setup itself deploys your checkout once by upload (`railway up`) when it creates the services; by hand, `make railway-up` (`PLATFORM_ENV=staging` for one environment). `repo: auto` (this checkout's `origin`) or `owner/name` turns Railway's deploy-on-push back on, from `branch` (`main`) for production and `branches.` (default `dev`) for the others, once the [Railway GitHub app](https://github.com/apps/railway-app) can read the repo. That bypasses the attestation and the release gate, so the template leaves it off. + +**Deploy only attested commits.** GitHub runs no verification on `main`: pull requests are verified on the agent's machine and attested by `make ci-local` before they merge ([CI](CI.md)). `Deploy staging` deploys a pushed commit only when `make ci-local` attested it, and production only takes a release whose commit staging ran ([RELEASES.md](RELEASES.md#what-is-gated)). Where GitHub Actions cannot run, nothing deploys on its own (docs/CI.md, "When Actions cannot run"). ## Profiles diff --git a/docs/development.md b/docs/development.md index 8b514b3..61921d7 100644 --- a/docs/development.md +++ b/docs/development.md @@ -40,9 +40,12 @@ Each checkout gets its own compose project and database ports, derived from its | `make test` | `go test -race ./...` (services, route handlers, and `packages/auth` against this checkout's throwaway tmpfs Postgres or `TEST_DATABASE_URL`), then per-package and per-test time budgets and per-boundary coverage floors | | `make lint` | gofmt + golangci-lint (layering and test-import rules) + test file locations | | `make lint-layering` / `make lint-tests` | prove the layering, test-location, coverage and test-time rules still reject planted violations | -| `make check-fast` | before every push: gofmt, vet, golangci-lint and plain tests for only what your changes vs origin/dev (else origin/main) affect (seconds) | +| `make check-fast` | before every push: gofmt, vet, golangci-lint and plain tests for only what your changes vs origin/main affect (seconds) | | `make hooks` / `make hooks-remove` | install / remove a pre-push hook that runs `make check-fast` (covers every worktree) | -| `make ci` | everything CI runs | +| `make ci` | every check, on the checkout as it is | +| `make ci-local RAN=""` | attest the pushed PR head after running what it needs: runs nothing, posts the `local-ci` status and fills the PR body's "Local verification" section (see [CI](CI.md)) | +| `make pr-merge PR="12 13"` | squash-merge PRs attested on their exact head; `ARGS="--check-deploy "` after a merge | +| `make secret-scan` | gitleaks (pinned) on what your branch changed; `ARGS=--all` for every tracked file | | `make generate` | sqlc, mocks (`go generate`) and templ code generation | | `make css` | rebuild both apps' `app.css` (pinned Tailwind standalone CLI) | | `make bench-devloop` | timing tables for the dev loop: build per binary (with size and cold start), generators, lint per linter, probes, tests per package with and without `-race`; `BENCH_COLD=1` adds cold caches, `BENCH_BASE=` an A/B of the test run against that ref on the same machine | @@ -67,4 +70,4 @@ Go stdlib `net/http` · `templ` + Datastar + Tailwind (standalone CLI) · `pgx` ## Working on it -Read [AGENTS.md](../AGENTS.md) (architecture, test boundaries, how to add a service, PR rules) and [CONTEXT.md](../CONTEXT.md) (shared vocabulary). +Read [AGENTS.md](../AGENTS.md) (architecture, test boundaries, how to add a service, PR rules), [CI](CI.md) (how a pull request is verified and merged) and [CONTEXT.md](../CONTEXT.md) (shared vocabulary). diff --git a/packages/uikit/README.md b/packages/uikit/README.md index 5ec2966..7601377 100644 --- a/packages/uikit/README.md +++ b/packages/uikit/README.md @@ -54,5 +54,5 @@ scripts, so pages stay CSP-clean. ## Building -`make generate` runs `templ generate` here too (CI checks the output); `make css` rebuilds both apps' `app.css`, which +`make generate` runs `templ generate` here too (`make check-generated` checks the output); `make css` rebuilds both apps' `app.css`, which include this package's classes through `base.css`'s `@source` lines. diff --git a/scripts/affected.sh b/scripts/affected.sh index c673ed3..a497481 100755 --- a/scripts/affected.sh +++ b/scripts/affected.sh @@ -1,12 +1,12 @@ #!/usr/bin/env bash -# Decides what CI must check for a change: the Go packages whose own files -# changed plus every package whose code or tests import them, transitively -# (Go's package graph via `go list -test -deps`). Changes to shared inputs -# (go.mod/go.sum, tooling, workflows, lint config, migrations, generator -# config) mean a full run. +# Decides what the root module's checks cover for a change (make check-fast, +# and the LINT_DIRS/TEST_PKGS to run before make ci-local): the Go packages whose own files changed plus every +# package whose code or tests import them, transitively (Go's package graph +# via `go list -test -deps`). Changes to shared inputs (go.mod/go.sum, +# tooling, scripts, lint config, migrations, generator config) mean a full run. # -# Usage: scripts/affected.sh [] (default: merge-base with origin/dev, -# or origin/main when there is no dev: scripts/branches.sh) +# Usage: scripts/affected.sh [] (default: merge-base with origin/main, +# scripts/branches.sh) # AFFECTED_FILES="a.go b.md" scripts/affected.sh (explicit file list) # Prints key=value lines (and appends them to $GITHUB_OUTPUT when set): # full=true|false run everything (./...) @@ -39,12 +39,12 @@ out() { } # Shared inputs: anything here can change every package's result. -# AFFECTED_FULL=1 forces a full run (pushes to dev/main, merge queue, manual runs). +# AFFECTED_FULL=1 forces a full run. full=false [ -n "${AFFECTED_FULL:-}" ] && full=true && echo "full run: AFFECTED_FULL" >&2 [ "$full" = true ] || for f in "${changed[@]}"; do case $f in - go.mod | go.sum | mise.toml | Makefile | .golangci.yml | tools/* | scripts/* | .github/* | \ + go.mod | go.sum | mise.toml | Makefile | .golangci.yml | tools/* | scripts/* | \ */migrations/* | packages/database/sqlc.yaml) full=true echo "full run: $f" >&2 diff --git a/scripts/branches.sh b/scripts/branches.sh index 3f124c1..1197b4c 100644 --- a/scripts/branches.sh +++ b/scripts/branches.sh @@ -1,10 +1,9 @@ -# The long-lived branches that deploy, staging first: dev deploys staging and -# main deploys production (AGENTS.md "Branches and releases"). Either may be -# missing (a fresh clone of the template has only main): everything here uses -# the ones that exist. Sourced by the scripts that need a base (check-fast, -# affected, check-migration-order, ci-check); the CI triggers in -# .github/workflows/ci.yml list the same names. -DEPLOY_BRANCHES=(dev main) +# The long-lived branches that deploy: main, which Deploy staging ships to +# staging on every attested merge; production runs releases cut from it +# (AGENTS.md "Branches and releases", docs/RELEASES.md). Everything here uses +# the ones this clone has fetched. Sourced by the scripts that need a base +# (check-fast, affected, the bench A/B); scripts/cilocal names the same. +DEPLOY_BRANCHES=(main) # deployed_refs prints origin/ for each deployed branch this clone # has fetched, staging first. @@ -15,9 +14,9 @@ deployed_refs() { done } -# deployed_names prints "origin/dev origin/main" (for messages). +# deployed_names prints "origin/main" (for messages). deployed_names() { printf 'origin/%s ' "${DEPLOY_BRANCHES[@]}" | sed 's/ $//'; } -# landing_ref prints the ref work lands on and is compared with: origin/dev -# when it exists, else origin/main. Empty when neither has been fetched. +# landing_ref prints the ref work lands on and is compared with: origin/main. +# Empty when it has not been fetched. landing_ref() { deployed_refs | head -n 1; } diff --git a/scripts/check-coverage.sh b/scripts/check-coverage.sh index ed0cdc8..d501812 100755 --- a/scripts/check-coverage.sh +++ b/scripts/check-coverage.sh @@ -17,6 +17,7 @@ floors=( "$mod/apps/admin/internal/services|90" "$mod/apps/admin/internal/api|85" "$mod/packages/auth|80" # measured 89.7% when set + "$mod/scripts/cilocal/verify|85" ) # --exist: fail if a floor package no longer exists (CI shards check floors diff --git a/scripts/check-fast.sh b/scripts/check-fast.sh index 34394ed..e2e89c8 100755 --- a/scripts/check-fast.sh +++ b/scripts/check-fast.sh @@ -4,14 +4,13 @@ # checkout's changes can affect, the generators whose inputs changed, and the # deploy/ module when it changed, and the migration order when a migration # changed. "Changes" are everything that differs from -# the merge base with origin/dev (origin/main when there is no dev; see -# scripts/branches.sh): commits, staged, unstaged and untracked -# files. scripts/affected.sh maps them to packages exactly as CI does, so a +# the merge base with origin/main (scripts/branches.sh): commits, staged, unstaged and untracked +# files. scripts/affected.sh maps them to packages (as make affected prints them), so a # change to a shared input (go.mod, tooling, lint config, migrations, # generator config) checks everything. # # No -race and no Postgres: database tests skip unless TEST_DATABASE_URL is -# set, and CI still runs the full suite. It catches what fails CI most often +# set; make test runs -race against Postgres. It catches what fails it most often # (format, lint, vet, a broken test, stale generated code) in seconds. # # Usage: scripts/check-fast.sh (CHECK_BASE= compares with another base) @@ -165,4 +164,4 @@ if [ "${#failed[@]}" -gt 0 ]; then echo "check-fast: FAIL (${failed[*]}) in $total" exit 1 fi -echo "check-fast: ok in $total (CI still runs -race, Postgres and the probes)" +echo "check-fast: ok in $total (make test runs -race, Postgres and the coverage floors)" diff --git a/scripts/check-layering.sh b/scripts/check-layering.sh index adf344d..976cb10 100755 --- a/scripts/check-layering.sh +++ b/scripts/check-layering.sh @@ -97,6 +97,8 @@ cases=( "apps/admin/ui|$mod/packages/auth" "apps/web/internal/server|$mod/apps/web/ui" "apps/admin/internal/server|$mod/apps/admin/ui" + # Local verification's rules stay pure (scripts/cilocal runs the commands). + "scripts/cilocal/verify|os/exec" ) # Test files. boundary dir | forbidden import diff --git a/scripts/check-migration-order.sh b/scripts/check-migration-order.sh index aa72436..7e624e4 100755 --- a/scripts/check-migration-order.sh +++ b/scripts/check-migration-order.sh @@ -1,4 +1,4 @@ -#!/usr/bin/env bash +#!/bin/sh # Fails when a goose migration this checkout adds is numbered at or below the # highest version a BASE already has in the same set. Goose refuses # out-of-order migrations, so once a deployed database has applied 00007, a @@ -9,85 +9,19 @@ # Usage: scripts/check-migration-order.sh [BASE...] # Each BASE is a ref whose migrations count as deployed (its tip, not the # merge base, so a branch that BASE has moved past fails too). Default: the -# deployed branches that exist (origin/dev and origin/main, scripts/ -# branches.sh), so a migration must sit above both. "Added" means +# deployed branch (origin/main, scripts/branches.sh). "Added" means # a migration in the working tree (committed, staged or untracked) that no # BASE has. A set is a directory named migrations holding NNNNN_*.sql files # (one goose version table each), so adding or deleting a set needs no change # here. Exit 0 ok, 1 a misnumbered migration, 2 an unknown BASE. -set -euo pipefail +# +# The check is `cilocal migration-order` (scripts/cilocal/verify/ +# migrations.go, tested there), built rather than run so its exit code +# survives (go run turns every failure into 1). Plain sh: no bash 4 needed. +set -eu # shellcheck disable=SC2046 unset $(git rev-parse --local-env-vars) cd "$(dirname "$0")/.." -. scripts/branches.sh - -readonly exit_misnumbered=1 # a migration needs a new number -readonly exit_usage=2 # a BASE git does not know - -bases=("$@") -[ "${#bases[@]}" -gt 0 ] || mapfile -t bases < <(deployed_refs) -if [ "${#bases[@]}" -eq 0 ]; then - echo "check-migration-order: none of $(deployed_names) fetched to compare with (git fetch origin, or pass a ref)" >&2 - exit "$exit_usage" -fi -for base in "${bases[@]}"; do - if ! git rev-parse --verify --quiet "$base^{commit}" >/dev/null; then - echo "check-migration-order: unknown base $base (git fetch origin, or pass a ref)" >&2 - exit "$exit_usage" - fi -done -label=${bases[0]} -for base in "${bases[@]:1}"; do label+=" or $base"; done - -migration_pattern='(^|/)migrations/[0-9]+_[^/]*\.sql$' -mapfile -t base_files < <(for base in "${bases[@]}"; do git ls-tree -r --name-only "$base"; done | grep -E "$migration_pattern" | sort -u || true) -mapfile -t tree_files < <(git ls-files --cached --others --exclude-standard | grep -E "$migration_pattern" | sort -u || true) - -# version_prefix prints a migration's number as written ("00007"). -version_prefix() { - local name=${1##*/} - echo "${name%%_*}" -} -# Per set (a migrations directory): the highest version on the bases, the -# width its numbers are written with, and the next free number. -declare -A on_base=() highest=() width=() next_free=() first_with=() -for file in "${base_files[@]}"; do - [ -n "$file" ] || continue - set_dir=${file%/*} prefix=$(version_prefix "$file") - on_base[$file]=1 width[$set_dir]=${#prefix} - if ((10#$prefix > ${highest[$set_dir]:-0})); then highest[$set_dir]=$((10#$prefix)); fi -done - -# Added files, and per set the highest number among the bases and them. -added=() -for file in "${tree_files[@]}"; do - [ -n "$file" ] && [ -e "$file" ] && [ -z "${on_base[$file]:-}" ] || continue - set_dir=${file%/*} prefix=$(version_prefix "$file") - added+=("$file") - : "${width[$set_dir]:=${#prefix}}" - top=${next_free[$set_dir]:-${highest[$set_dir]:-0}} - if ((10#$prefix > top)); then top=$((10#$prefix)); fi - next_free[$set_dir]=$top -done -for set_dir in "${!next_free[@]}"; do next_free[$set_dir]=$((next_free[$set_dir] + 1)); done - -status=0 -for file in "${added[@]}"; do - set_dir=${file%/*} prefix=$(version_prefix "$file") - version=$((10#$prefix)) set_highest=${highest[$set_dir]:-0} - if ((version <= set_highest)); then - printf '%s: version %d is not above the highest on %s (%d): renumber to %0*d\n' \ - "$file" "$version" "$label" "$set_highest" "${width[$set_dir]}" "${next_free[$set_dir]}" >&2 - elif [ -n "${first_with[$set_dir/$version]:-}" ]; then - printf '%s: version %d is also used by %s: renumber to %0*d\n' \ - "$file" "$version" "${first_with[$set_dir/$version]}" "${width[$set_dir]}" "${next_free[$set_dir]}" >&2 - else - first_with[$set_dir/$version]=$file - continue - fi - status=$exit_misnumbered next_free[$set_dir]=$((next_free[$set_dir] + 1)) -done -if [ "$status" -ne 0 ]; then - echo "check-migration-order: number a new migration above every one on $label, per set (AGENTS.md \"Migrations\")" >&2 -fi -exit "$status" +mkdir -p .cache +"${GO:-go}" build -o .cache/cilocal ./scripts/cilocal +exec .cache/cilocal migration-order "$@" diff --git a/scripts/check-test-boundaries.sh b/scripts/check-test-boundaries.sh index 4044c86..a544890 100755 --- a/scripts/check-test-boundaries.sh +++ b/scripts/check-test-boundaries.sh @@ -11,6 +11,7 @@ boundaries=( apps/admin/internal/services # admin business logic, same rules apps/admin/internal/api # admin route handlers, same rules packages/auth # backend auth: its own suite against real Postgres + scripts/cilocal/verify # local verification's rules: the section, refusals, merge and staging verdicts (pure) ) # No other tests: no UI, config, wiring, repository, secrets or deploy tests. diff --git a/scripts/check-test-probes.sh b/scripts/check-test-probes.sh index 581b67e..5cf33ef 100755 --- a/scripts/check-test-probes.sh +++ b/scripts/check-test-probes.sh @@ -39,6 +39,7 @@ planted=( packages/database/adminrepo/zz_probe_test.go packages/secrets/zz_probe_test.go deploy/platform/zz_probe_test.go + scripts/cilocal/zz_probe_test.go ) for f in "${planted[@]}"; do d=$(dirname "$f") diff --git a/scripts/ci-check.sh b/scripts/ci-check.sh deleted file mode 100755 index 3a7ea2a..0000000 --- a/scripts/ci-check.sh +++ /dev/null @@ -1,232 +0,0 @@ -#!/usr/bin/env bash -# The CI checks, run as parallel tasks inside one job (.github/actions/check): -# lint, migration order, generated code, tests, the release smoke, the deploy/ -# module and the mobile app each run in the background with their own log, then the probes -# (which plant files in the tree, so nothing else may be reading it) run once -# all of them are done. -# Each task's log is printed as one collapsed group, failures expanded. -# -# Env (from the check action; empty or false skips a task): -# MODE all | db | other (db: lint + DB tests, other: the rest; see ci.yml) -# DIRS affected package dirs for golangci-lint -# TESTS import paths to test (scripts/ci-shard.sh: tested packages only) -# DB true: some of TESTS need Postgres (started by the action) -# FULL true: shared inputs changed, also run the probes -# SQLC TEMPL GENERATE generators to rerun (scripts/affected.sh) -# MOBILE DEPLOY true: check apps/mobile-app / the deploy/ module -# SMOKE true: the release smoke (scripts/release-smoke.sh) -# EVENT TARGET the GitHub event and the branch it targets (the PR's -# base, else the pushed or dispatched ref): migration order -set -uo pipefail -cd "$(dirname "$0")/.." - -# The job's Postgres (started by the check action) and how long to wait for -# it: 2x its measured boot (12-22 s), at most 45 s. -readonly pg_server=postgres://postgres:postgres@127.0.0.1:5432 -readonly pg_wait_seconds=0.25 -readonly pg_wait_attempts=180 -# Every task but the timed test run runs at the lowest CPU priority, so the -# tests get the CPUs whenever they want them and the others use what is left -# (see test_build). -readonly background_niceness=19 -readonly build_poll_seconds=0.2 # how often the test binaries' build is checked - -tmp=${RUNNER_TEMP:-$(mktemp -d)}/ci-check -rm -rf "$tmp" && mkdir -p "$tmp" .cache -lint=false -[ "$MODE" != other ] && [ -n "${DIRS:-}" ] && lint=true - -names=() pids=() starts=() -start() { # start : run it in the background, log to $tmp/.log - names+=("$1") starts+=("$(date +%s.%N)") - # The inner subshell is the task (its set -e exits it, not the bookkeeping); - # its children inherit the outer subshell's priority. - ( - [ "$1" = test ] || renice -n "$background_niceness" -p "$BASHPID" >/dev/null - ("$2") >"$tmp/$1.log" 2>&1 - echo $? >"$tmp/$1.rc" - date +%s.%N >"$tmp/$1.end" - ) & - pids+=($!) -} -failed=0 -report() { # print every started task's log, in start order - local i n rc secs - for i in "${!names[@]}"; do - n=${names[$i]} rc=$(cat "$tmp/$n.rc" 2>/dev/null || echo unfinished) - secs=$(awk -v a="${starts[$i]}" -v b="$(cat "$tmp/$n.end" 2>/dev/null || date +%s.%N)" 'BEGIN { printf "%.1f", b - a }') - if [ "$rc" = 0 ]; then - echo "::group::ok ${secs}s $n"; cat "$tmp/$n.log"; echo "::endgroup::" - else - echo "::error::$n failed (${secs}s, exit $rc)"; cat "$tmp/$n.log"; failed=1 - fi - done -} -# A step timeout or cancel still shows what each task got to. -trap 'report; exit 1' INT TERM - -# ---- tasks ---- -lint_go() { - set -e - unformatted=$(gofmt -l .) - [ -z "$unformatted" ] || { echo "$unformatted"; echo 'gofmt: run gofmt -w .'; exit 1; } - # word splitting intended: DIRS is a list. --allow-parallel-runners: the - # deploy task lints at the same time (same cache, safe). - golangci-lint run --allow-parallel-runners $DIRS - go run ./scripts/namelint $DIRS # single-character names (AGENTS.md "Taste") - ./scripts/check-test-boundaries.sh - ./scripts/check-ui-api-paths.sh - if [ "$FULL" = true ]; then ./scripts/check-coverage.sh --exist; fi -} - -# In a separate worktree of HEAD, so regenerating never rewrites a file that -# lint or the tests are reading. -generated() { - set -e - gen=$tmp/gen - git worktree add -q --detach "$gen" HEAD - cd "$gen" - if [ "$SQLC" = true ]; then sqlc generate -f packages/database/sqlc.yaml; fi - for p in $GENERATE; do go generate "$p"; done - if [ "$TEMPL" = true ]; then - for ui in packages/uikit apps/web/ui apps/admin/ui; do if [ -d "$ui" ]; then go tool templ generate -path "$ui"; fi; done - fi - status=$(git status --porcelain) - [ -z "$status" ] || { git diff; echo "$status"; echo 'generated code is stale: run make generate and commit'; exit 1; } -} - -# The time budgets (AGENTS.md "Fast tests") must measure tests, not -# compilation on a 2-vCPU runner. So test_build first builds and links every -# test binary, untimed and at background priority like the other tasks, with -# the timed run's -race -cover instrumentation (each "runs" as `true`, so no -# test, TestMain or database is touched). Only then does test_go start, at -# normal priority: it just links and runs, and the other tasks (lint, -# generators, smoke, deploy/, mobile) get only the CPU time it leaves. -test_build() { - set -e - go test -race -cover -exec true $TESTS >/dev/null -} - -# DB-backed test binaries wait for Postgres through an -exec wrapper, so -# their own binary's link overlaps its boot. Readiness is checked inside the container over -# TCP: docker's port proxy accepts connections before Postgres listens, and -# the image's first-boot init server answers on the unix socket only. -test_go() { - set -e - exec_flag=() - if [ "$DB" = true ]; then - cat >"$tmp/wait-db" <<'SH' -#!/usr/bin/env bash -if grep -lq 'TEST_DATABASE_URL' ./*_test.go 2>/dev/null; then - # 2x the measured Postgres boot (12-22 s): never waits longer than 45 s. - for _ in $(seq 1 180); do - docker exec pg pg_isready -q -h 127.0.0.1 -U postgres 2>/dev/null && exec "$@" - sleep 0.25 - done - echo "postgres did not start:" >&2; cat "$RUNNER_TEMP/pg.log" >&2; docker logs pg >&2; exit 1 -fi -exec "$@" -SH - chmod +x "$tmp/wait-db" - exec_flag=(-exec "$tmp/wait-db") - fi - # testbudget prints plain go test output, the slowest tests, and fails a - # package or test over its time budget. - go test -race -json "${exec_flag[@]}" -coverprofile=.cache/coverage.out $TESTS | go run ./scripts/testbudget - TESTED="$TESTS" ./scripts/check-coverage.sh .cache/coverage.out -} - -# A new migration must sit above every migration the deployed branches have -# (scripts/check-migration-order.sh): staging (dev) and production (main). -# Bases: HEAD^1 on PRs, the merge queue and pushes (the target branch: the -# merge commit's base, or the branch's previous commit, which catches two PRs -# that raced for one number), plus the tip of each deployed branch other than -# the target itself. A deployed branch the repo does not have is skipped. -migration_order() { - set -e - . scripts/branches.sh - bases=() refspecs=() - case $EVENT in pull_request | merge_group | push) bases+=(HEAD^1) ;; esac - for branch in $(git ls-remote --heads origin "${DEPLOY_BRANCHES[@]}" | sed 's|.*refs/heads/||'); do - [ "$branch" = "$TARGET" ] && continue - refspecs+=("+refs/heads/$branch:refs/remotes/origin/$branch") bases+=("origin/$branch") - done - [ "${#refspecs[@]}" -eq 0 ] || git fetch -q --no-tags --depth=1 origin "${refspecs[@]}" - if [ "${#bases[@]}" -eq 0 ]; then echo "no base to compare with ($EVENT on $TARGET)"; return; fi - echo "bases: ${bases[*]}" - ./scripts/check-migration-order.sh "${bases[@]}" -} - -# Builds every Dockerfile's binary as its image does, migrates fresh -# databases on this job's Postgres and boots web + admin in production mode. -# Waits for Postgres like the DB tests do (it boots while the build runs). -smoke() { - set -e - local attempt - for attempt in $(seq 1 "$pg_wait_attempts"); do - docker exec pg pg_isready -q -h 127.0.0.1 -U postgres 2>/dev/null && break - sleep "$pg_wait_seconds" - done - echo "postgres ready after $attempt checks" - local database - for database in smoke_app smoke_admin; do docker exec pg createdb -U postgres "$database"; done - SMOKE_DATABASE_URL="$pg_server/smoke_app?sslmode=disable" \ - SMOKE_ADMIN_DATABASE_URL="$pg_server/smoke_admin?sslmode=disable" \ - ./scripts/release-smoke.sh -} - -deploy_mod() { - set -e - go build -o "$tmp/namelint" ./scripts/namelint - cd deploy - go vet ./... - (cd .railway && go vet railway.go) - golangci-lint run --allow-parallel-runners -c ../.golangci.yml ./... - "$tmp/namelint" ./... - cd .. && make --no-print-directory deploy-vars -} - -# node_modules comes from the cache keyed by the lockfile (see the action); -# installed only on a miss. -mobile() { - set -e - cd apps/mobile-app - [ -d node_modules ] || pnpm install --frozen-lockfile - pnpm exec tsc --noEmit - pnpm exec expo lint - pnpm exec expo install --check - pnpm test -} - -[ "$lint" = true ] && start lint lint_go -[ "$MODE" != other ] && [ -n "${EVENT:-}" ] && start migrations migration_order -[ "$MODE" != other ] && { [ "${SQLC:-}" = true ] || [ "${TEMPL:-}" = true ] || [ -n "${GENERATE:-}" ]; } && start generated generated -[ -n "${TESTS:-}" ] && start test-build test_build -[ "$MODE" != db ] && [ "${SMOKE:-}" = true ] && start smoke smoke -[ "${DEPLOY:-}" = true ] && start deploy deploy_mod -[ "${MOBILE:-}" = true ] && start mobile mobile -# The timed tests start once their binaries are built (see test_build). -if [ -n "${TESTS:-}" ]; then - while [ ! -e "$tmp/test-build.rc" ]; do sleep "$build_poll_seconds"; done - [ "$(cat "$tmp/test-build.rc")" != 0 ] || start test test_go -fi -wait "${pids[@]}" 2>/dev/null -report -[ -d "$tmp/gen" ] && git worktree remove --force "$tmp/gen" - -# ---- probes, after everything above: they plant files in the tree ---- -# They prove the lint and test rules still bite; their inputs (scripts/, -# .golangci.yml) always force a full run. The coverage probes reuse the test -# run's profile (a full run tests every probed package). -probes() { - set -e - if [ "$lint" = true ]; then GOLANGCI_LINT=golangci-lint ./scripts/check-layering.sh; fi - if [ "$MODE" != db ]; then PROFILE=.cache/coverage.out ./scripts/check-test-probes.sh; fi -} -if [ "$failed" = 0 ] && [ "$FULL" = true ]; then - names=() pids=() starts=() - start probes probes - wait "${pids[@]}" - report -fi -exit "$failed" diff --git a/scripts/ci-reuse.sh b/scripts/ci-reuse.sh deleted file mode 100755 index c3219c0..0000000 --- a/scripts/ci-reuse.sh +++ /dev/null @@ -1,73 +0,0 @@ -#!/usr/bin/env bash -# Decides whether a CI run can skip its checks (run by the ci job, see -# .github/workflows/ci.yml "Reuse or skip"): -# push: the pushed commit's tree is identical to the merge commit a PR's -# CI tested and passed. Every condition must hold: the PR's latest -# ci-ok status on its head is a success, posted by -# github-actions[bot], recording exactly this tree -# ("success tree="). Then ci-ok success is posted on the -# pushed commit with a pointer to that result. -# Also skipped when the pushed commit itself already has that -# success (dev or main fast-forwarded to a merge-queue commit CI tested). -# schedule: the build cache for this commit already exists. -# Anything else, or any doubt: skip=false and the run checks everything. -# Env: EVENT, MESSAGE (head commit message), GH_TOKEN, GITHUB_REPOSITORY, -# GITHUB_SHA, GITHUB_SERVER_URL, GITHUB_RUN_ID, RUNNER_OS -# Prints (and appends to $GITHUB_OUTPUT when set): skip=true|false -set -euo pipefail -cd "$(dirname "$0")/.." - -out() { - echo "skip=$1" - if [ -n "${GITHUB_OUTPUT:-}" ]; then echo "skip=$1" >>"$GITHUB_OUTPUT"; fi - exit 0 -} -repo=$GITHUB_REPOSITORY sha=$GITHUB_SHA - -case ${EVENT:-} in -schedule) - key="v7-go-${RUNNER_OS:-Linux}-$sha" - if [ "$(gh cache list --key "$key" --json key --jq "[.[] | select(.key == \"$key\")] | length" 2>/dev/null || echo 0)" != 0 ]; then - echo "cache $key already saved: nothing to refresh" - out true - fi - out false - ;; -push) ;; -*) out false ;; -esac - -tree=$(git rev-parse "$sha^{tree}") - -# The commit itself already passed (e.g. a branch fast-forwarded to a merge-queue -# commit that CI tested): nothing to do. -self=$(gh api "repos/$repo/commits/$sha/statuses" \ - --jq '[.[] | select(.context == "ci-ok")][0] | "\(.state)|\(.creator.login)|\(.description)"' 2>/dev/null || true) -if [ "$self" = "success|github-actions[bot]|success tree=$tree" ]; then - echo "commit ${sha:0:7} already has ci-ok success for tree $tree" - out true -fi -# PRs this commit came from: GitHub's association (squash/merge/rebase via the -# PR), plus "(#N)" in the first line, which GitHub adds to squash-merge titles. -prs=$( - { - gh api "repos/$repo/commits/$sha/pulls" --jq '.[].number' 2>/dev/null || true - head -n1 <<<"${MESSAGE:-}" | grep -oE '\(#[0-9]+\)' | tr -dc '0-9\n' || true - } | sort -un -) -for pr in $prs; do - head=$(gh api "repos/$repo/pulls/$pr" --jq .head.sha 2>/dev/null) || continue - # Newest first; only the latest ci-ok counts. - st=$(gh api "repos/$repo/commits/$head/statuses" \ - --jq '[.[] | select(.context == "ci-ok")][0] | "\(.state)|\(.creator.login)|\(.description)|\(.target_url)"' 2>/dev/null) || continue - IFS='|' read -r state creator desc url <<<"$st" - echo "PR #$pr head ${head:0:7}: ci-ok=$state by $creator ($desc)" - if [ "$state" = success ] && [ "$creator" = "github-actions[bot]" ] && [ "$desc" = "success tree=$tree" ]; then - gh api "repos/$repo/statuses/$sha" -f state=success -f context=ci-ok \ - -f description="reused PR #$pr (tree ${tree:0:12})" -f target_url="$url" >/dev/null - echo "tree $tree passed CI on PR #$pr: reusing that result" - out true - fi -done -echo "no PR result for tree $tree: running the checks" -out false diff --git a/scripts/ci-shard.sh b/scripts/ci-shard.sh deleted file mode 100755 index bc6c8a9..0000000 --- a/scripts/ci-shard.sh +++ /dev/null @@ -1,38 +0,0 @@ -#!/usr/bin/env bash -# Picks the packages one CI job tests, of those with test files (lint -# type-checks the rest; testing them would only compile them with coverage): -# "db" = packages whose tests use Postgres (TEST_DATABASE_URL in their -# _test.go files), "other" = the rest, "all" = both (a single job). -# Usage: PACKAGES="|./..." scripts/ci-shard.sh db|other|all -# Prints (and appends to $GITHUB_OUTPUT when set): -# packages= (empty: nothing to test) -# db=true|false some of them need Postgres -set -euo pipefail -cd "$(dirname "$0")/.." -shard=$1 - -if [ "${PACKAGES:-}" = ./... ]; then - mapfile -t all < <(go list ./...) -else - read -r -a all <<<"${PACKAGES:-}" -fi - -mod=$(go list -m) -mine=() db=false -for p in "${all[@]}"; do - [ -n "$p" ] || continue - d=${p/#$mod/.} - compgen -G "$d/*_test.go" >/dev/null || continue - if grep -lq 'TEST_DATABASE_URL' "$d"/*_test.go 2>/dev/null; then kind=db; else kind=other; fi - if [ "$shard" = all ] || [ "$kind" = "$shard" ]; then - mine+=("$p") - if [ "$kind" = db ]; then db=true; fi - fi -done - -out() { - printf '%s=%s\n' "$1" "$2" - if [ -n "${GITHUB_OUTPUT:-}" ]; then printf '%s=%s\n' "$1" "$2" >>"$GITHUB_OUTPUT"; fi -} -out packages "${mine[*]}" -out db "$db" diff --git a/scripts/cilocal/github.go b/scripts/cilocal/github.go new file mode 100644 index 0000000..017efc3 --- /dev/null +++ b/scripts/cilocal/github.go @@ -0,0 +1,300 @@ +package main + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "os" + "os/exec" + "path/filepath" + "slices" + "strconv" + "strings" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +// Everything here shells out to git and gh, the two tools the rules in +// verify are fed from. + +// output runs a command in `dir` (the current directory when empty) and +// returns its trimmed stdout; stderr is part of the error. +func output(ctx context.Context, dir, name string, args ...string) (string, error) { + command := exec.CommandContext(ctx, name, args...) + command.Dir = dir + var stderr bytes.Buffer + command.Stderr = &stderr + out, err := command.Output() + if err != nil { + return "", fmt.Errorf("%s %s: %w: %s", name, strings.Join(args, " "), err, strings.TrimSpace(stderr.String())) + } + return strings.TrimSpace(string(out)), nil +} + +func git(ctx context.Context, dir string, args ...string) (string, error) { + return output(ctx, dir, "git", args...) +} + +// github reads and writes one repository through gh, so it works wherever gh +// is signed in (a laptop, a runner with GH_TOKEN). `repo` is owner/name. +type github struct { + repo string +} + +func newGitHub(ctx context.Context) (github, error) { + if repo := os.Getenv("GITHUB_REPOSITORY"); repo != "" { + return github{repo: repo}, nil + } + repo, err := output(ctx, "", "gh", "repo", "view", "--json", "nameWithOwner", "-q", ".nameWithOwner") + if err != nil { + return github{}, err + } + return github{repo: repo}, nil +} + +// api GETs repos// into `into`. +func (hub github) api(ctx context.Context, path string, into any) error { + out, err := output(ctx, "", "gh", "api", "repos/"+hub.repo+"/"+path) + if err != nil { + return err + } + if err := json.Unmarshal([]byte(out), into); err != nil { + return fmt.Errorf("gh api %s: %w", path, err) + } + return nil +} + +// statuses is every status ever posted on `sha`, newest first. +func (hub github) statuses(ctx context.Context, sha string) ([]verify.Status, error) { + var statuses []verify.Status + err := hub.api(ctx, "commits/"+sha+"/statuses?per_page=100", &statuses) + return statuses, err +} + +// current is the `local-ci` entry of `sha`'s combined status, or nil. +func (hub github) current(ctx context.Context, sha string) (*verify.Status, error) { + var combined struct { + Statuses []verify.Status `json:"statuses"` + } + if err := hub.api(ctx, "commits/"+sha+"/status", &combined); err != nil { + return nil, err + } + for index := range combined.Statuses { + if combined.Statuses[index].Context == verify.StatusContext { + return &combined.Statuses[index], nil + } + } + return nil, nil +} + +// verification is what GitHub holds for a head: the given body, the +// head's `local-ci` in its combined status and every status ever posted on +// it, and `fork`, the repository of a head outside this one (verify.ForkOf). +func (hub github) verification(ctx context.Context, body, head, fork string) (verify.Verification, error) { + current, err := hub.current(ctx, head) + if err != nil { + return verify.Verification{}, err + } + statuses, err := hub.statuses(ctx, head) + if err != nil { + return verify.Verification{}, err + } + return verify.Verification{Body: body, Head: head, Status: current, Statuses: statuses, Fork: fork}, nil +} + +// pullRequest is the open pull request of a branch. +type pullRequest struct { + Number int `json:"number"` + URL string `json:"url"` + Body string `json:"body"` + HeadRefOid string `json:"headRefOid"` + HeadRefName string `json:"headRefName"` + BaseRefName string `json:"baseRefName"` + State string `json:"state"` + IsDraft bool `json:"isDraft"` + Mergeable string `json:"mergeable"` + Checks []verify.Check `json:"statusCheckRollup"` + // A head in a fork: its repository and owner (null once it is deleted). + CrossRepository bool `json:"isCrossRepository"` + HeadRepository *struct { + Name string `json:"name"` + } `json:"headRepository"` + HeadOwner *struct { + Login string `json:"login"` + } `json:"headRepositoryOwner"` +} + +const pullFields = "number,url,body,headRefOid,headRefName,baseRefName,state,isDraft,mergeable,statusCheckRollup,isCrossRepository,headRepository,headRepositoryOwner" + +// fork is the repository the pull request's head lives in when it is not +// `repo` (verify.ForkOf), or "". +func (pull pullRequest) fork(repo string) string { + if !pull.CrossRepository { + return "" + } + headRepo := "" + if pull.HeadRepository != nil && pull.HeadOwner != nil { + headRepo = pull.HeadOwner.Login + "/" + pull.HeadRepository.Name + } + return verify.ForkOf(repo, headRepo) +} + +// openPull is the branch's open pull request, or nil. +func (hub github) openPull(ctx context.Context, branch string) (*pullRequest, error) { + if branch == "" { + return nil, nil + } + out, err := output(ctx, "", "gh", "pr", "list", "-R", hub.repo, "--head", branch, "--state", "open", "--json", pullFields) + if err != nil { + return nil, err + } + var pulls []pullRequest + if err := json.Unmarshal([]byte(out), &pulls); err != nil { + return nil, fmt.Errorf("gh pr list: %w", err) + } + // A fork's pull request can share the branch's name; it is not this branch's. + for index := range pulls { + if !pulls[index].CrossRepository { + return &pulls[index], nil + } + } + return nil, nil +} + +func (hub github) pull(ctx context.Context, number int) (pullRequest, error) { + out, err := output(ctx, "", "gh", "pr", "view", strconv.Itoa(number), "-R", hub.repo, "--json", pullFields) + if err != nil { + return pullRequest{}, err + } + var pull pullRequest + if err := json.Unmarshal([]byte(out), &pull); err != nil { + return pullRequest{}, fmt.Errorf("gh pr view: %w", err) + } + for index, check := range pull.Checks { + if check.WorkflowName != verify.CIWorkflow || check.Conclusion != "FAILURE" || check.JobID() == 0 { + continue + } + if pull.Checks[index].NotStarted, err = hub.notStarted(ctx, check.JobID()); err != nil { + return pullRequest{}, err + } + } + return pull, nil +} + +// notStarted is whether GitHub failed the job without starting it: its +// annotations say so (verify.NotStartedMessage). +func (hub github) notStarted(ctx context.Context, job int) (bool, error) { + var annotations []annotation + if err := hub.api(ctx, "check-runs/"+strconv.Itoa(job)+"/annotations", &annotations); err != nil { + return false, err + } + return slices.ContainsFunc(annotations, func(note annotation) bool { return verify.NotStartedMessage(note.Message) }), nil +} + +// annotation is one annotation of a check run. +type annotation struct { + Message string `json:"message"` +} + +func (hub github) postStatus(ctx context.Context, sha, state, description, target string) error { + args := []string{"api", "--method", "POST", "repos/" + hub.repo + "/statuses/" + sha, + "-f", "state=" + state, "-f", "context=" + verify.StatusContext, "-f", "description=" + description} + if target != "" { + args = append(args, "-f", "target_url="+target) + } + _, err := output(ctx, "", "gh", args...) + return err +} + +func (hub github) editBody(ctx context.Context, number int, body string) error { + dir, err := os.MkdirTemp("", "cilocal-body-") + if err != nil { + return fmt.Errorf("body file: %w", err) + } + defer func() { _ = os.RemoveAll(dir) }() + file := filepath.Join(dir, "body.md") + if err := os.WriteFile(file, []byte(body), fileMode); err != nil { + return fmt.Errorf("body file: %w", err) + } + _, err = output(ctx, "", "gh", "pr", "edit", strconv.Itoa(number), "-R", hub.repo, "--body-file", file) + return err +} + +// attested is the staging gate's verdict on a commit of a deployed branch +// (verify.Attestation): who attested it, or why nobody did. +func (hub github) attested(ctx context.Context, sha string) (string, []string, error) { + current, err := hub.current(ctx, sha) + if err != nil { + return "", nil, err + } + state := "" + if current != nil { + state = current.State + } + var commit struct { + Commit struct { + Message string `json:"message"` + } `json:"commit"` + } + if err := hub.api(ctx, "commits/"+sha, &commit); err != nil { + return "", nil, err + } + type mergedPull struct { + Number int `json:"number"` + MergedAt string `json:"merged_at"` + MergeCommitSHA string `json:"merge_commit_sha"` + Body string `json:"body"` + Head struct { + SHA string `json:"sha"` + Repo *struct { + FullName string `json:"full_name"` + } `json:"repo"` + } `json:"head"` + } + var pulls []mergedPull + if err := hub.api(ctx, "commits/"+sha+"/pulls", &pulls); err != nil { + return "", nil, err + } + merged := []mergedPull{} + for _, pull := range pulls { + if pull.MergedAt != "" && pull.MergeCommitSHA == sha { + merged = append(merged, pull) + } + } + // The commit-to-PR index can trail a fresh merge; the squash title names it. + if number := verify.TitledPull(commit.Commit.Message); len(merged) == 0 && number != 0 { + var pull mergedPull + if err := hub.api(ctx, "pulls/"+strconv.Itoa(number), &pull); err != nil { + return "", nil, err + } + if pull.MergedAt != "" && pull.MergeCommitSHA == sha { + merged = append(merged, pull) + } + } + judged := []verify.PullAttestation{} + for _, pull := range merged { + headRepo := "" + if pull.Head.Repo != nil { + headRepo = pull.Head.Repo.FullName + } + verification, err := hub.verification(ctx, pull.Body, pull.Head.SHA, verify.ForkOf(hub.repo, headRepo)) + if err != nil { + return "", nil, err + } + judged = append(judged, verify.PullAttestation{Number: pull.Number, Verification: verification}) + } + by, reasons := verify.Attestation(sha, state, judged) + return by, reasons, nil +} + +var errNotVerified = errors.New("not verified") + +// Permissions of what the tool writes: its cache directory, PR bodies and +// the secret scan's copies, and the gitleaks binary. +const ( + dirMode = 0o750 + fileMode = 0o600 + binaryMode = 0o700 +) diff --git a/scripts/cilocal/main.go b/scripts/cilocal/main.go new file mode 100644 index 0000000..0d2469f --- /dev/null +++ b/scripts/cilocal/main.go @@ -0,0 +1,171 @@ +// Command cilocal is local verification (docs/CI.md, "Local verification"): +// the agent runs what its change needs on the exact pushed head, then +// attests it, recorded on GitHub as the `local-ci` commit status and the PR +// body's "Local verification" section. The rules live in +// scripts/cilocal/verify; this wires them to git, gh and HTTP. +// +// cilocal run --ran "" make ci-local: attest the pushed head (runs nothing; run -h) +// cilocal check [--claim] ci.yml's Local verification jobs (REPO, PR, HEAD_SHA, CLAIMED) +// cilocal merge ... make pr-merge: squash-merge attested PRs, check staging after +// cilocal attest the staging gate (Deploy staging, Staging catch-up) +// cilocal secret-scan ... gitleaks on changed files (secret-scan -h) +// cilocal migration-order ... scripts/check-migration-order.sh +package main + +import ( + "context" + "errors" + "fmt" + "os" + "os/signal" + "strings" + "syscall" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +// exitUsage is the exit code of a usage error or a refusal to attest. +const exitUsage = 2 + +// usageError is a wrong call or a refusal: it exits 2 and prints only itself. +type usageError struct{ message string } + +func (usage usageError) Error() string { return usage.message } + +func main() { + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) + args := os.Args[1:] + command := "run" + if len(args) > 0 && !strings.HasPrefix(args[0], "-") { + command, args = args[0], args[1:] + } + commands := map[string]func(context.Context, []string) error{ + "run": runCommand, "check": checkCommand, "merge": mergeCommand, "attest": attestCommand, + "secret-scan": secretScan, "migration-order": migrationOrder, + } + run, known := commands[command] + if !known { + fmt.Fprintf(os.Stderr, "cilocal: unknown command %q (run, check, merge, attest, secret-scan, migration-order)\n", command) + stop() + os.Exit(exitUsage) + } + err := run(ctx, args) + stop() + var usage usageError + switch { + case err == nil: + case errors.As(err, &usage): + fmt.Fprintf(os.Stderr, "cilocal %s: %s\n", command, usage.message) + os.Exit(exitUsage) + case errors.Is(err, errNotVerified): + os.Exit(1) + default: + fmt.Fprintf(os.Stderr, "cilocal %s: %v\n", command, err) + os.Exit(1) + } +} + +// checkCommand is ci.yml's two Local verification jobs. It reads the PR body +// and the head's statuses live from the API, not the event payload, so a +// re-run after make ci-local sees what it posted, and runs from the base's +// checkout (pull_request_target), so the head is only data. HEAD_REPO is the +// head's repository: a fork's head fails (verify.ForkOf). With --claim it +// writes claimed=true|false to $GITHUB_OUTPUT (verify.Claimed): ci.yml names the +// checking job "Local verification", the required check, only for a claimed +// head. Run with CLAIMED other than true, it says what to run and passes +// under its other name, so an unattested head's required check stays +// "Expected", never red. +func checkCommand(ctx context.Context, args []string) error { + claim := len(args) == 1 && args[0] == "--claim" + if len(args) > 0 && !claim { + return usageError{"usage: cilocal check [--claim] (env REPO, PR, HEAD_SHA, HEAD_REPO; CLAIMED, GITHUB_OUTPUT)"} + } + repo, number, head := os.Getenv("REPO"), os.Getenv("PR"), os.Getenv("HEAD_SHA") + if repo == "" || number == "" || head == "" { + return errors.New("check needs REPO, PR and HEAD_SHA") + } + hub := github{repo: repo} + fork := verify.ForkOf(repo, os.Getenv("HEAD_REPO")) + var pull struct { + Body string `json:"body"` + } + if err := hub.api(ctx, "pulls/"+number, &pull); err != nil { + return err + } + if claim { + statuses, err := hub.statuses(ctx, head) + if err != nil { + return err + } + return writeClaim(head, verify.Claimed(verify.Verification{Body: pull.Body, Head: head, Statuses: statuses, Fork: fork})) + } + if os.Getenv("CLAIMED") != "true" { + fmt.Printf("::notice::Awaiting make ci-local on this head: %s.\n", verify.NextStep) + return nil + } + verification, err := hub.verification(ctx, pull.Body, head, fork) + if err != nil { + return err + } + if problems := verify.LocalVerificationProblems(verification); len(problems) > 0 { + for _, problem := range problems { + fmt.Printf("::error::%s\n", problem) + } + fmt.Printf("::error::Not verified locally: %s.\n", verify.NextStep) + return errNotVerified + } + fmt.Printf("%s: make ci-local attested this head (%s)\n", head, verification.Status.Description) + return nil +} + +func writeClaim(head string, claimed bool) error { + path := os.Getenv("GITHUB_OUTPUT") + if path == "" { + return errors.New("check --claim needs GITHUB_OUTPUT") + } + file, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, fileMode) + if err != nil { + return fmt.Errorf("GITHUB_OUTPUT: %w", err) + } + if _, err := fmt.Fprintf(file, "claimed=%t\n", claimed); err != nil { + _ = file.Close() + return fmt.Errorf("GITHUB_OUTPUT: %w", err) + } + if err := file.Close(); err != nil { + return fmt.Errorf("GITHUB_OUTPUT: %w", err) + } + if claimed { + fmt.Printf("%s: make ci-local has claimed this head; Local verification checks it\n", head) + } else { + fmt.Printf("%s: make ci-local has not attested this head yet, so Local verification stays Expected\n", head) + } + return nil +} + +// attestCommand is the staging gate (deploy-staging.yml's plan, and +// staging-catch-up.yml before it dispatches): it passes a commit of main only +// when make ci-local attested it, on the commit itself or on the exact head of +// the pull request whose squash made it. +func attestCommand(ctx context.Context, args []string) error { + if len(args) != 1 { + return usageError{"usage: cilocal attest "} + } + hub, err := newGitHub(ctx) + if err != nil { + return err + } + sha := args[0] + by, reasons, err := hub.attested(ctx, sha) + if err != nil { + return err + } + if by != "" { + fmt.Printf("%s is attested: %s\n", sha, by) + return nil + } + for _, reason := range reasons { + fmt.Printf("::error::%s\n", reason) + } + fmt.Printf("::error::Not deploying %s: %s.\n", sha, verify.AttestNextStep(sha)) + return errNotVerified +} diff --git a/scripts/cilocal/merge.go b/scripts/cilocal/merge.go new file mode 100644 index 0000000..06061b3 --- /dev/null +++ b/scripts/cilocal/merge.go @@ -0,0 +1,322 @@ +package main + +import ( + "context" + "encoding/json" + "errors" + "flag" + "fmt" + "io" + "net/http" + "os" + "regexp" + "strconv" + "strings" + "time" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +// make pr-merge is the one supported way to land pull requests +// (.claude/skills/github-prs). It never waits: a PR that is a draft, not +// attested on its head, failing, pending or conflicting is reported with the +// next step and left alone (verify.Decide), so one stuck PR never holds up +// the others; a head behind main merges unless it conflicts. Before merging a +// PR it retargets to main every open PR stacked on its branch, so the stack +// survives. +// +// make pr-merge PR="21 12 20" merge what is green, report the rest +// make pr-merge PR=21 ARGS=--dry-run say what it would do +// make pr-merge PR="21 12" ARGS=--then-check-staging then one staging check for the burst +// make pr-merge ARGS="--check-deploy " after a merge: did staging deploy its squash +// make pr-merge ARGS=--check-staging the staging check of main's head alone +// +// Exits 0 when every listed PR merged (or was already merged) and staging, if +// checked, is healthy; 1 otherwise. Run it again later for the PRs it reported. + +const mergeUsage = `usage: make pr-merge PR="..." [ARGS="--dry-run --any-base --require-actions --then-check-staging"] + make pr-merge ARGS="--check-deploy " + make pr-merge ARGS=--check-staging` + +var commitSHA = regexp.MustCompile(`^[0-9a-f]{7,40}$`) + +// mergeFlags are make pr-merge's flags. +type mergeFlags struct { + dryRun, thenCheckStaging, checkStaging bool + checkDeploy string + options verify.MergeOptions +} + +func mergeCommand(ctx context.Context, args []string) error { + flags := flag.NewFlagSet("pr-merge", flag.ContinueOnError) + flags.SetOutput(io.Discard) + var opts mergeFlags + flags.BoolVar(&opts.dryRun, "dry-run", false, "say what it would do") + flags.BoolVar(&opts.options.AnyBase, "any-base", false, "merge PRs stacked on a branch other than main") + flags.BoolVar(&opts.options.RequireActions, "require-actions", false, "refuse when no GitHub CI job ran on the head") + flags.BoolVar(&opts.thenCheckStaging, "then-check-staging", false, "after merging, follow main's staging deploy until it settles (bounded)") + flags.BoolVar(&opts.checkStaging, "check-staging", false, "check main's head on staging once") + flags.StringVar(&opts.checkDeploy, "check-deploy", "", "check once that staging deployed this squash commit") + if err := flags.Parse(args); err != nil { + return usageError{fmt.Sprintf("%v\n%s", err, mergeUsage)} + } + numbers := []int{} + for _, arg := range flags.Args() { + number, err := strconv.Atoi(strings.TrimPrefix(arg, "#")) + if err != nil { + return usageError{fmt.Sprintf("%q is not a pull request number\n%s", arg, mergeUsage)} + } + numbers = append(numbers, number) + } + switch { + case opts.checkDeploy != "" && !commitSHA.MatchString(opts.checkDeploy): + return usageError{fmt.Sprintf("--check-deploy needs a commit sha, not %q\n%s", opts.checkDeploy, mergeUsage)} + case len(numbers) == 0 && !opts.checkStaging && opts.checkDeploy == "": + return usageError{mergeUsage} + } + hub, err := newGitHub(ctx) + if err != nil { + return err + } + if opts.checkDeploy != "" { + if err := checkStaging(ctx, hub, opts.checkDeploy, false); errors.Is(err, errStaging) { + return errNotVerified + } else if err != nil { + return err + } + return nil + } + failed := false + for _, number := range numbers { + verdict, err := mergeOne(ctx, hub, number, opts) + if err != nil { + verdict = verify.Verdict{Action: verify.ActionSkip, Reason: "gh failed: " + err.Error(), Next: "read the error; run again", Note: ""} + } + failed = failed || (verdict.Action != verify.ActionDone && verdict.Action != verify.ActionMerge && verdict.Action != merged) + fmt.Printf("#%d: %s - %s\n", number, verdict.Action, verdict.Reason) + if verdict.Next != "" { + fmt.Printf(" next: %s\n", verdict.Next) + } + } + if opts.checkStaging || (opts.thenCheckStaging && !opts.dryRun) { + if err := checkStaging(ctx, hub, "", opts.thenCheckStaging); err != nil { + if !errors.Is(err, errStaging) { + fmt.Printf("staging: could not read it: %v\n", err) + } + failed = true + } + } + if failed { + return errNotVerified + } + return nil +} + +// merged is the action of a PR this run squash-merged. +const merged = "merged" + +// mergeAttempts and mergePause bound the wait for GitHub to recompute +// mergeability after main moves. +const ( + mergeAttempts = 5 + mergePause = 3 * time.Second +) + +func mergeOne(ctx context.Context, hub github, number int, opts mergeFlags) (verify.Verdict, error) { + pull, err := hub.pull(ctx, number) + for attempt := 0; err == nil && pull.Mergeable == "UNKNOWN" && attempt < mergeAttempts; attempt++ { + if err := pause(ctx, mergePause); err != nil { + return verify.Verdict{}, err + } + pull, err = hub.pull(ctx, number) + } + if err != nil { + return verify.Verdict{}, err + } + verification, err := hub.verification(ctx, pull.Body, pull.HeadRefOid, pull.fork(hub.repo)) + if err != nil { + return verify.Verdict{}, err + } + verdict := verify.Decide(verify.Pull{ + Number: number, State: pull.State, Draft: pull.IsDraft, Base: pull.BaseRefName, Mergeable: pull.Mergeable, + Checks: pull.Checks, Verification: verification, + }, opts.options) + if verdict.Action != verify.ActionMerge { + return verdict, nil + } + fmt.Printf("#%d: %s\n", number, verdict.Note) + if err := retargetStack(ctx, hub, number, pull, opts.dryRun); err != nil { + return verify.Verdict{}, err + } + if opts.dryRun { + return verify.Verdict{Action: verify.ActionMerge, Reason: "would merge " + verify.Short(pull.HeadRefOid) + " (dry run)", Next: "", Note: ""}, nil + } + // Squash exactly the head that was attested; never --delete-branch (it + // closes the PRs stacked on the branch). + if _, err := output(ctx, "", "gh", "pr", "merge", strconv.Itoa(number), "-R", hub.repo, "--squash", "--match-head-commit", pull.HeadRefOid); err != nil { + return verify.Verdict{}, err + } + squash, err := output(ctx, "", "gh", "pr", "view", strconv.Itoa(number), "-R", hub.repo, "--json", "mergeCommit", "-q", ".mergeCommit.oid") + if err != nil { + return verify.Verdict{}, err + } + return verify.Verdict{ + Action: merged, + Reason: fmt.Sprintf("squashed %s as %s", verify.Short(pull.HeadRefOid), squash), + // A follow-up, never a gate: merge the next PR now (AGENTS.md, "Environments"). + Next: fmt.Sprintf(`once its deploy has run, make pr-merge ARGS="--check-deploy %s"; a failed or refused deploy is fixed forward before anything else merges`, squash), + Note: "", + }, nil +} + +// retargetStack points every open PR stacked on the PR's branch at main. +func retargetStack(ctx context.Context, hub github, number int, pull pullRequest, dryRun bool) error { + stacked, err := output(ctx, "", "gh", "pr", "list", "-R", hub.repo, "--base", pull.HeadRefName, "--state", "open", "--json", "number", "-q", ".[].number") + if err != nil { + return err + } + for _, child := range strings.Fields(stacked) { + if !dryRun { + if _, err := output(ctx, "", "gh", "pr", "edit", child, "-R", hub.repo, "--base", verify.Main); err != nil { + return err + } + } + fmt.Printf("#%d: retargeted stacked #%s to %s; after this merge rebase it: git rebase --onto origin/%s %s \n", + number, child, verify.Main, verify.Main, pull.HeadRefOid) + } + return nil +} + +func pause(ctx context.Context, wait time.Duration) error { + select { + case <-ctx.Done(): + return fmt.Errorf("interrupted: %w", ctx.Err()) + case <-time.After(wait): + return nil + } +} + +// How long --then-check-staging follows a deploy, and how often it reads. +const ( + stagingTimeout = 60 * time.Minute + stagingInterval = 90 * time.Second +) + +var errStaging = errors.New("staging is not healthy") + +// checkStaging is one staging check: of `commit` (a merge's squash), or of +// main's head for the whole burst, following it until it settles when +// `wait` is set. +func checkStaging(ctx context.Context, hub github, commit string, wait bool) error { + deadline := time.Now().Add(stagingTimeout) + for { + verdict, err := readStaging(ctx, hub, commit) + if err != nil { + return err + } + if verdict.Settled || !wait || time.Now().After(deadline) { + unsettled := "" + if !verdict.Settled { + unsettled = " (not settled; Staging catch-up deploys main within the hour; check once more later)" + } + fmt.Printf("staging: %s%s\n", verdict.Summary, unsettled) + if !verdict.OK { + return errStaging + } + return nil + } + if err := pause(ctx, stagingInterval); err != nil { + return err + } + } +} + +func readStaging(ctx context.Context, hub github, commit string) (verify.StagingVerdict, error) { + head := commit + if head == "" { + var err error + if head, err = output(ctx, "", "gh", "api", "repos/"+hub.repo+"/commits/"+verify.Main, "-q", ".sha"); err != nil { + return verify.StagingVerdict{}, err + } + } + // Every run on the commit, filtered here: --workflow fails while the + // default branch has no deploy-staging.yml. + out, err := output(ctx, "", "gh", "run", "list", "-R", hub.repo, "--commit", head, + "--json", "databaseId,workflowName,status,conclusion,url") + if err != nil { + return verify.StagingVerdict{}, err + } + var runs []struct { + verify.DeployRun + DatabaseID int64 `json:"databaseId"` + } + if err := json.Unmarshal([]byte(out), &runs); err != nil { + return verify.StagingVerdict{}, fmt.Errorf("gh run list: %w", err) + } + deploys := []verify.DeployRun{} + for _, run := range runs { + if run.Name != verify.DeployWorkflow { + continue + } + if run.Status == "completed" && run.Conclusion == "success" { + jobs, err := runJobs(ctx, hub, strconv.FormatInt(run.DatabaseID, 10)) + if err != nil { + return verify.StagingVerdict{}, err + } + run.DryRun = verify.DryRunJobs(jobs) + } + deploys = append(deploys, run.DeployRun) + } + return verify.Staging(head, deploys, readiness(ctx, hub)), nil +} + +func runJobs(ctx context.Context, hub github, run string) ([]verify.Job, error) { + out, err := output(ctx, "", "gh", "run", "view", run, "-R", hub.repo, "--json", "jobs") + if err != nil { + return nil, err + } + var view struct { + Jobs []verify.Job `json:"jobs"` + } + if err := json.Unmarshal([]byte(out), &view); err != nil { + return nil, fmt.Errorf("gh run view: %w", err) + } + return view.Jobs, nil +} + +// Bounds of the /healthz read. +const ( + readyTimeout = 20 * time.Second + readyBody = 1 << 16 // 64 KiB: a /healthz answer is a few bytes +) + +// readiness GETs /healthz: from the env, else the +// repository variable; unset when neither has it (the template has no +// default staging URL). +func readiness(ctx context.Context, hub github) verify.Readiness { + url := os.Getenv("STAGING_WEB_URL") + if url == "" { + url, _ = output(ctx, "", "gh", "variable", "get", "STAGING_WEB_URL", "-R", hub.repo) + } + url = strings.TrimRight(strings.TrimSpace(url), "/") + if url == "" { + return verify.Readiness{} + } + ctx, cancel := context.WithTimeout(ctx, readyTimeout) + defer cancel() + request, err := http.NewRequestWithContext(ctx, http.MethodGet, url+"/healthz", nil) + if err != nil { + return verify.Readiness{URL: url, Err: err.Error()} + } + request.Header.Set("Accept", "application/json") + response, err := http.DefaultClient.Do(request) + if err != nil { + return verify.Readiness{URL: url, Err: err.Error()} + } + defer func() { _ = response.Body.Close() }() + body, err := io.ReadAll(io.LimitReader(response.Body, readyBody)) + if err != nil { + return verify.Readiness{URL: url, Code: response.StatusCode, Err: err.Error()} + } + return verify.ReadinessOf(url, response.StatusCode, body) +} diff --git a/scripts/cilocal/migrations.go b/scripts/cilocal/migrations.go new file mode 100644 index 0000000..bdeaec8 --- /dev/null +++ b/scripts/cilocal/migrations.go @@ -0,0 +1,66 @@ +package main + +import ( + "context" + "fmt" + "os" + "strings" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +// exitUnknownBase is check-migration-order.sh's exit code for a base git +// does not know (1 is a misnumbered migration). +const exitUnknownBase = 2 + +// migrationOrder is `cilocal migration-order [BASE...]` +// (scripts/check-migration-order.sh): each BASE is a ref whose migrations +// count as deployed, by default the deployed branches origin has +// (origin/main). "Added" means a migration in the working +// tree (committed, staged or untracked) that no BASE has. +func migrationOrder(ctx context.Context, args []string) error { + bases := args + if len(bases) == 0 { + for _, branch := range []string{"main"} { // scripts/branches.sh + if _, err := git(ctx, "", "rev-parse", "--verify", "--quiet", "refs/remotes/origin/"+branch); err == nil { + bases = append(bases, "origin/"+branch) + } + } + } + if len(bases) == 0 { + fmt.Fprintln(os.Stderr, "check-migration-order: origin/main is not fetched to compare with (git fetch origin, or pass a ref)") + os.Exit(exitUnknownBase) + } + baseFiles := []string{} + for _, base := range bases { + if _, err := git(ctx, "", "rev-parse", "--verify", "--quiet", base+"^{commit}"); err != nil { + fmt.Fprintf(os.Stderr, "check-migration-order: unknown base %s (git fetch origin, or pass a ref)\n", base) + os.Exit(exitUnknownBase) + } + listed, err := git(ctx, "", "ls-tree", "-r", "--name-only", base) + if err != nil { + return err + } + baseFiles = append(baseFiles, strings.Split(listed, "\n")...) + } + listed, err := git(ctx, "", "ls-files", "--cached", "--others", "--exclude-standard") + if err != nil { + return err + } + treeFiles := []string{} + for _, file := range strings.Split(listed, "\n") { + if _, err := os.Stat(file); err == nil { // deleted in the working tree + treeFiles = append(treeFiles, file) + } + } + label := strings.Join(bases, " or ") + problems := verify.MigrationProblems(baseFiles, treeFiles, label) + for _, problem := range problems { + fmt.Fprintln(os.Stderr, problem) + } + if len(problems) > 0 { + fmt.Fprintf(os.Stderr, "check-migration-order: number a new migration above every one on %s, per set (AGENTS.md \"Migrations\")\n", label) + return errNotVerified + } + return nil +} diff --git a/scripts/cilocal/run.go b/scripts/cilocal/run.go new file mode 100644 index 0000000..08b44b7 --- /dev/null +++ b/scripts/cilocal/run.go @@ -0,0 +1,317 @@ +package main + +import ( + "context" + "encoding/json" + "flag" + "fmt" + "io" + "os" + "strconv" + "strings" + "time" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +// make ci-local runs no checks. The agent runs what its change needs first +// (docs/CI.md, "What to run"), commits and pushes; this then refuses a dirty +// or unpushed tree, or a head that already failed, posts the `local-ci` +// commit status on that exact SHA and fills the PR body's "Local +// verification" section with the statement and the head. It takes seconds. +// +// make ci-local RAN="make lint test TEST_PKGS=./x" +// make ci-local FAILED="make test: TestX fails" [RAN="…"] +// make ci-local COMMIT= RAN="…" attest a main commit checked out here +// +// A head attested as failed is never attested again: the failure is resolved +// by a new commit, in this PR or forward off main (AGENTS.md, "Verifying"). + +// options are make ci-local's flags. +type options struct { + ran, failed, commit string + failedSet bool +} + +const runUsage = `usage: make ci-local RAN="" [FAILED=""] [COMMIT=
] + go run ./scripts/cilocal run --ran "" [--failed ""] [--commit ] +make ci-local runs nothing: run what the change needs first (docs/CI.md, "What to run")` + +func parseOptions(args []string) (options, error) { + flags := flag.NewFlagSet("ci-local", flag.ContinueOnError) + flags.SetOutput(io.Discard) + var opts options + flags.StringVar(&opts.ran, "ran", "", "what you ran on this head (required unless --failed)") + flags.StringVar(&opts.failed, "failed", "", "what failed: posts failure, final for this head") + flags.StringVar(&opts.commit, "commit", "", "attest this main commit, checked out detached here") + if err := flags.Parse(args); err != nil { + return options{}, usageError{fmt.Sprintf("%v\n%s", err, runUsage)} + } + if flags.NArg() > 0 { + return options{}, usageError{fmt.Sprintf("unexpected argument %q\n%s", flags.Arg(0), runUsage)} + } + flags.Visit(func(set *flag.Flag) { opts.failedSet = opts.failedSet || set.Name == "failed" }) + return opts, nil +} + +func logf(format string, args ...any) { + fmt.Fprintf(os.Stderr, "ci-local: "+format+"\n", args...) +} + +// state is the checkout and its pull request as the attestation found them. +type state struct { + head verify.Head + tree string + pull *pullRequest +} + +func repositoryState(ctx context.Context, hub github, opts options) (state, error) { + var found state + if _, err := git(ctx, "", "fetch", "--quiet", "origin", verify.Main); err != nil { + return found, err + } + sha, err := git(ctx, "", "rev-parse", "HEAD") + if err != nil { + return found, err + } + branch, _ := git(ctx, "", "symbolic-ref", "--quiet", "--short", "HEAD") + commit := "" + if opts.commit != "" { + if commit, err = git(ctx, "", "rev-parse", "--verify", opts.commit+"^{commit}"); err != nil { + return found, err + } + } + mainCommit := commit != "" || branch == verify.Main + pushed := "" + if branch != "" && !mainCommit { + remote, _ := git(ctx, "", "ls-remote", "origin", "refs/heads/"+branch) + if fields := strings.Fields(remote); len(fields) > 0 { + pushed = fields[0] + } + if found.pull, err = hub.openPull(ctx, branch); err != nil { + return found, err + } + } + _, notAncestor := git(ctx, "", "merge-base", "--is-ancestor", sha, "origin/"+verify.Main) + dirty, err := git(ctx, "", "status", "--porcelain") + if err != nil { + return found, err + } + if mainCommit { + branch = "" + } + found.head = verify.Head{SHA: sha, Branch: branch, Dirty: dirty, Commit: commit, MainCommit: mainCommit, Pushed: pushed, OnBase: notAncestor == nil, Base: verify.Main} + found.tree, err = git(ctx, "", "rev-parse", "HEAD^{tree}") + return found, err +} + +// runCommand is make ci-local. +func runCommand(ctx context.Context, args []string) error { + opts, err := parseOptions(args) + if err != nil { + return err + } + ranText := opts.ran + if ranText == "" && opts.failedSet { + ranText = "see Failed below" + } + ran, problem := verify.StatementOf(ranText) + failed := "" + if problem == "" && opts.failedSet { + failed, problem = verify.StatementOf(opts.failed) + } + if problem != "" { + return usageError{problem + "\n" + runUsage} + } + hub, err := newGitHub(ctx) + if err != nil { + return err + } + found, err := repositoryState(ctx, hub, opts) + if err != nil { + return err + } + if refusal := found.head.Refusal(); refusal != "" { + return usageError{"refusing to attest: " + refusal} + } + statuses, err := hub.statuses(ctx, found.head.SHA) + if err != nil { + return err + } + if why := verify.FailedHeadRefusal(found.head.SHA, statuses); why != "" { + return usageError{"refusing to attest: " + why} + } + if found.head.MainCommit && !opts.failedSet { + last, err := lastAttested(ctx, hub, found.head.SHA) + if err != nil { + return err + } + if why := verify.FirstAttestationProblem(found.head.SHA, last, ran); why != "" { + return usageError{"refusing to attest: " + why} + } + if last != "" { + logf("the last attested main commit is %s: the statement covers %s..%s", verify.Short(last), verify.Short(last), verify.Short(found.head.SHA)) + } + } + author, _ := git(ctx, "", "config", "user.name") + if author == "" { + author = "unknown" + } + hostname, _ := os.Hostname() + report := verify.Report{ + Head: found.head.SHA, Tree: found.tree, Author: author, Hostname: hostname, + FinishedAt: time.Now().UTC().Format(time.RFC3339), Result: verify.Passed, Ran: ran, Failed: failed, + } + if opts.failedSet { + report.Result = verify.Failed + } + if err := publish(ctx, hub, opts, report); err != nil { + return err + } + if report.Result != verify.Passed { + return errNotVerified + } + return nil +} + +// lastAttested is the nearest of a main commit's first-parent ancestors +// (verify.AttestedAncestorLimit of them) the staging gate accepts, or "". A +// root commit, or the first commit of a shallow clone, has none. +func lastAttested(ctx context.Context, hub github, sha string) (string, error) { + list, err := git(ctx, "", "rev-list", "--first-parent", fmt.Sprintf("--max-count=%d", verify.AttestedAncestorLimit+1), sha) + if err != nil { + return "", err + } + ancestors := strings.Fields(list)[1:] // the first is the commit itself + var lookupErr error + last := verify.LastAttested(ancestors, func(ancestor string) bool { + by, _, err := hub.attested(ctx, ancestor) + lookupErr = err + return err != nil || by != "" + }) + return last, lookupErr +} + +// publish reads the checkout again and posts `report` only for exactly that +// clean, pushed head and tree (verify.PublishRefusal): the `local-ci` status +// on the head first, then the PR body's section. The body edit starts the +// PR's CI run (`edited`), whose Local verification job then reads both; when +// the body already holds this section, the PR's newest CI run is re-run +// instead. +func publish(ctx context.Context, hub github, opts options, report verify.Report) error { + found, err := repositoryState(ctx, hub, opts) + if err != nil { + return err + } + if why := verify.PublishRefusal(report, found.head, found.tree); why != "" { + return usageError{"refusing to attest: " + why} + } + passed := report.Result == verify.Passed + section := verify.RenderSection(report) + current := found.pull != nil && found.pull.HeadRefOid == report.Head + status, target := "failure", "" + if passed { + status = "success" + } + if current { + target = found.pull.URL + "#local-verification" + } + if err := hub.postStatus(ctx, report.Head, status, verify.StatusDescription(report, section), target); err != nil { + return err + } + logf("posted %s=%s on %s", verify.StatusContext, status, report.Head) + if !current { + switch { + case found.head.MainCommit: + logf("main commit %s: the staging gate reads %s on it; dispatch Deploy staging if staging is behind", verify.Short(report.Head), verify.StatusContext) + case found.pull != nil: + logf("%s is at %s, not %s: its body was left alone", found.pull.URL, verify.Short(found.pull.HeadRefOid), verify.Short(report.Head)) + default: + logf("no open PR for %s: open it from the template (.github/pull_request_template.md), then run this again", found.head.Branch) + } + return nil + } + body := verify.WithSection(found.pull.Body, section) + if body == found.pull.Body { + if passed { + rerunWaitingCI(ctx, hub, report.Head) + } + return nil + } + if err := hub.editBody(ctx, found.pull.Number, body); err != nil { + return err + } + logf("filled the Local verification section of %s; its PR CI run starts now", found.pull.URL) + return nil +} + +// rerunWaitingCI re-runs the PR's newest CI run on `head` when the body was +// already current (so no `edited` run starts): its Local verification job +// read the PR before this attestation. It re-runs the failed jobs, or the +// whole run when it never ran the job named `Local verification` (it ran as +// "Awaiting make ci-local", so its claim job must read the head again). It +// waits (bounded) for a run still in progress. +func rerunWaitingCI(ctx context.Context, hub github, head string) { + const attempts, pause = 40, 15 * time.Second + for attempt := range attempts { + out, err := output(ctx, "", "gh", "run", "list", "-R", hub.repo, "--workflow", "ci.yml", "--commit", head, "--event", verify.TrustedTrigger, + "--limit", "1", "--json", "databaseId,status,conclusion,url") + var runs []struct { + DatabaseID int64 `json:"databaseId"` + Status string `json:"status"` + Conclusion string `json:"conclusion"` + URL string `json:"url"` + } + if err != nil || json.Unmarshal([]byte(out), &runs) != nil || len(runs) == 0 { + logf("no PR CI run on this head yet; it starts on the next edit of the PR") + return + } + latest := runs[0] + if latest.Status == "completed" { + id := strconv.FormatInt(latest.DatabaseID, 10) + checked, err := ranGate(ctx, hub, id) + if err != nil { + logf("could not read the jobs of %s: %v", latest.URL, err) + return + } + if latest.Conclusion == "success" && checked { + logf("PR CI is green: %s", latest.URL) + return + } + rerun := []string{"run", "rerun", id, "-R", hub.repo} + what := "" + if checked { + rerun, what = append(rerun, "--failed"), "the failed jobs of " + } + if _, err := output(ctx, "", "gh", rerun...); err != nil { + logf("could not re-run %s: %v", latest.URL, err) + return + } + logf("re-ran %s%s", what, latest.URL) + return + } + if attempt == 0 { + logf("waiting for PR CI %s to finish before re-running it", latest.URL) + } + select { + case <-ctx.Done(): + return + case <-time.After(pause): + } + } + logf("PR CI is still running; re-run it with gh run rerun --failed if it fails") +} + +// ranGate is whether a run has a job named `Local verification`. +func ranGate(ctx context.Context, hub github, run string) (bool, error) { + jobs, err := runJobs(ctx, hub, run) + if err != nil { + return false, err + } + for _, job := range jobs { + if job.Name == verify.Gate { + return true, nil + } + } + return false, nil +} diff --git a/scripts/cilocal/secretscan.go b/scripts/cilocal/secretscan.go new file mode 100644 index 0000000..76a9cdd --- /dev/null +++ b/scripts/cilocal/secretscan.go @@ -0,0 +1,446 @@ +package main + +import ( + "archive/tar" + "bufio" + "compress/gzip" + "context" + "crypto/sha256" + "encoding/hex" + "errors" + "flag" + "fmt" + "io" + "net/http" + "os" + "os/exec" + "path/filepath" + "runtime" + "slices" + "strconv" + "strings" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +// The secret scan (docs/CI.md, "Secret scan"): the files a change touches, +// copied out and scanned with gitleaks (pinned, checksum-verified), using +// .gitleaks.toml. make secret-scan runs it on the branch's changed files; the +// Secret scan job in ci.yml runs it on the pull request's file list, so it +// does not depend on the agent's machine. A finding fails, printing the +// rule, file and line with the value redacted. +// +// cilocal secret-scan --base files changed since the commit +// cilocal secret-scan --pr the pull request's files (GitHub's list) +// cilocal secret-scan --all every tracked file +// cilocal secret-scan ... just these files +// +// --head (the Secret scan job) reads those files from that commit's +// objects instead of the checkout: the job checks out the base, so the +// scanner is the base's and the pull request's head is only data +// (verify.ScanTargets), never checked out or run. +// +// Policy (verify.ScannerPolicy: .gitleaks.toml and .gitleaksignore) comes +// only from the trusted commit: the --base commit, or else the checkout's +// HEAD (the base, in the job). It is written outside the scanned directory, +// a change's own policy files are copied in as content under another name +// (verify.ScanCopyPath), inline gitleaks:allow comments and GITLEAKS_* env +// are ignored, and a change that edits policy fails for review +// (verify.PolicyProblem) after the scan. + +const gitleaksVersion = "8.30.1" + +// gitleaksSHA256 are the release archives' checksums +// (gitleaks__checksums.txt), by GOOS_GOARCH. +var gitleaksSHA256 = map[string]string{ + "linux_amd64": "551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb", + "linux_arm64": "e4a487ee7ccd7d3a7f7ec08657610aa3606637dab924210b3aee62570fb4b080", + "darwin_amd64": "dfe101a4db2255fc85120ac7f3d25e4342c3c20cf749f2c20a18081af1952709", + "darwin_arm64": "b40ab0ae55c505963e365f271a8d3846efbc170aa17f2607f13df610a9aeb6a5", +} + +// gitleaks is the pinned binary, downloaded once into the user's cache. +func gitleaks(ctx context.Context) (string, error) { + platform := runtime.GOOS + "_" + runtime.GOARCH + want, known := gitleaksSHA256[platform] + if !known { + return "", fmt.Errorf("no pinned gitleaks for %s", platform) + } + cache, err := os.UserCacheDir() + if err != nil { + return "", fmt.Errorf("gitleaks: %w", err) + } + dir := filepath.Join(cache, "cilocal", "gitleaks-"+gitleaksVersion) + binary := filepath.Join(dir, "gitleaks") + if _, err := os.Stat(binary); err == nil { + return binary, nil + } + asset := strings.Replace(platform, "amd64", "x64", 1) + url := fmt.Sprintf("https://github.com/gitleaks/gitleaks/releases/download/v%s/gitleaks_%s_%s.tar.gz", gitleaksVersion, gitleaksVersion, asset) + request, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil) + if err != nil { + return "", fmt.Errorf("gitleaks: %w", err) + } + response, err := http.DefaultClient.Do(request) + if err != nil { + return "", fmt.Errorf("downloading gitleaks: %w", err) + } + defer func() { _ = response.Body.Close() }() + if response.StatusCode != http.StatusOK { + return "", fmt.Errorf("downloading %s: %s", url, response.Status) + } + archive, err := io.ReadAll(response.Body) + if err != nil { + return "", fmt.Errorf("downloading gitleaks: %w", err) + } + if sum := sha256.Sum256(archive); hex.EncodeToString(sum[:]) != want { + return "", fmt.Errorf("%s has sha256 %s, expected %s", url, hex.EncodeToString(sum[:]), want) + } + if err := os.MkdirAll(dir, dirMode); err != nil { + return "", fmt.Errorf("gitleaks: %w", err) + } + if err := untarOne(archive, "gitleaks", binary); err != nil { + return "", err + } + return binary, nil +} + +func untarOne(archive []byte, name, target string) error { + unzipped, err := gzip.NewReader(strings.NewReader(string(archive))) + if err != nil { + return fmt.Errorf("gitleaks archive: %w", err) + } + reader := tar.NewReader(unzipped) + for { + header, err := reader.Next() + if errors.Is(err, io.EOF) { + return fmt.Errorf("gitleaks archive has no %s", name) + } + if err != nil { + return fmt.Errorf("gitleaks archive: %w", err) + } + if header.Name != name { + continue + } + partial := target + ".partial" + file, err := os.OpenFile(partial, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, binaryMode) + if err != nil { + return fmt.Errorf("gitleaks: %w", err) + } + const maxBinary = 1 << 28 // 256 MiB, far above gitleaks' size + if _, err := io.CopyN(file, reader, maxBinary); err != nil && !errors.Is(err, io.EOF) { + _ = file.Close() + return fmt.Errorf("gitleaks: %w", err) + } + if err := file.Close(); err != nil { + return fmt.Errorf("gitleaks: %w", err) + } + if err := os.Rename(partial, target); err != nil { + return fmt.Errorf("gitleaks: %w", err) + } + return nil + } +} + +// scanFiles is which files the arguments name (nil for every tracked file), +// the commit to read them from ("" for the checkout), and the base commit +// ("" without --base). +func scanFiles(ctx context.Context, args []string) ([]string, string, string, error) { + flags := flag.NewFlagSet("secret-scan", flag.ContinueOnError) + base := flags.String("base", "", "scan the files changed since this commit") + pull := flags.String("pr", "", "scan this pull request's files (GitHub's list)") + all := flags.Bool("all", false, "scan every tracked file") + head := flags.String("head", "", "read the files from this commit's objects, not the checkout") + if err := flags.Parse(args); err != nil { + return nil, "", "", fmt.Errorf("secret-scan: %w", err) + } + files, err := namedFiles(ctx, flags, *base, *pull, *all) + return files, *head, *base, err +} + +func namedFiles(ctx context.Context, flags *flag.FlagSet, base, pull string, all bool) ([]string, error) { + switch { + case all: + return nil, nil + case base != "": + out, err := git(ctx, "", "diff", "--name-only", "--no-renames", "--diff-filter=d", base, "HEAD") + if err != nil { + return nil, err + } + return strings.Fields(out), nil + case pull != "": + hub, err := newGitHub(ctx) + if err != nil { + return nil, err + } + out, err := output(ctx, "", "gh", "api", "--paginate", "repos/"+hub.repo+"/pulls/"+pull+"/files?per_page=100", + "-q", `.[] | select(.status != "removed") | .filename`) + files := strings.Split(out, "\n") + if err != nil || len(files) >= gitHubFileLimit { + fmt.Println("Scanning every file: could not list the pull request's files") + return nil, nil //nolint:nilerr // fall back to scanning everything + } + return files, nil + } + return flags.Args(), nil +} + +// gitHubFileLimit is the most files GitHub lists for a pull request. +const gitHubFileLimit = 3000 + +func secretScan(ctx context.Context, args []string) error { + files, head, base, err := scanFiles(ctx, args) + if err != nil { + return err + } + // The policy is the trusted commit's: --base, or the checkout's HEAD (the + // base, in the Secret scan job). The change compared with it is the head + // read as data, or HEAD against --base; --all and named files scan none. + trusted, scanned := "HEAD", head + if base != "" { + trusted = base + if scanned == "" { + scanned = "HEAD" + } + } + changed, err := policyChanges(ctx, trusted, scanned) + if err != nil { + return err + } + work, err := os.MkdirTemp("", "secret-scan-") + if err != nil { + return fmt.Errorf("secret-scan: %w", err) + } + defer func() { _ = os.RemoveAll(work) }() + // The trusted policy and the copies live side by side, never one inside + // the other: gitleaks loads policy from the root of what it scans. + policy, scan := filepath.Join(work, "policy"), filepath.Join(work, "scan") + for _, dir := range []string{policy, scan} { + if err := os.Mkdir(dir, dirMode); err != nil { + return fmt.Errorf("secret-scan: %w", err) + } + } + if err := writePolicy(ctx, trusted, policy); err != nil { + return err + } + copyFiles := copyCheckout + if head != "" { + copyFiles = func(ctx context.Context, files []string, scan string) (int, error) { + return copyCommit(ctx, head, files, scan) + } + } + copied, err := copyFiles(ctx, files, scan) + if err != nil { + return err + } + if copied == 0 { + fmt.Println("No changed files to scan.") + } else if err := runGitleaks(ctx, scan, policy, copied); err != nil { + return err + } + if problem := verify.PolicyProblem(changed); problem != "" { + return errors.New(problem) + } + return nil +} + +// policyChanges is the scanner policy files that differ between the trusted +// commit and the scanned one (verify.PolicyChanges); none when `scanned` is "". +func policyChanges(ctx context.Context, trusted, scanned string) ([]string, error) { + if scanned == "" { + return nil, nil + } + if err := fetchCommit(ctx, scanned); err != nil { + return nil, err + } + trees := [2][]verify.TreeEntry{} + for index, rev := range []string{trusted, scanned} { + listing, err := git(ctx, "", "ls-tree", "-z", "--full-tree", rev) + if err != nil { + return nil, err + } + trees[index] = verify.ParseTree(listing) + } + return verify.PolicyChanges(trees[0], trees[1]), nil +} + +// trustedDefault is the config when the trusted commit has no +// .gitleaks.toml: gitleaks' own rules, nothing allowed. +const trustedDefault = "[extend]\nuseDefault = true\n" + +// writePolicy writes the trusted commit's .gitleaks.toml and .gitleaksignore +// into `dir` (gitleaks' defaults and an empty ignore list where it has none). +func writePolicy(ctx context.Context, rev, dir string) error { + defaults := map[string]string{".gitleaks.toml": trustedDefault, ".gitleaksignore": ""} + for _, name := range verify.ScannerPolicy { + content := defaults[name] + if _, err := git(ctx, "", "cat-file", "-e", rev+":"+name); err == nil { + command := exec.CommandContext(ctx, "git", "cat-file", "blob", rev+":"+name) + out, err := command.Output() + if err != nil { + return fmt.Errorf("git cat-file %s:%s: %w", rev, name, err) + } + content = string(out) + } + if err := os.WriteFile(filepath.Join(dir, name), []byte(content), fileMode); err != nil { + return fmt.Errorf("secret-scan: %w", err) + } + } + return nil +} + +// fetchCommit fetches `rev` when the checkout lacks it: a pull request's +// head, a fork's included, is reachable from the base repository. +func fetchCommit(ctx context.Context, rev string) error { + if _, err := git(ctx, "", "cat-file", "-e", rev+"^{commit}"); err == nil { + return nil + } + _, err := git(ctx, "", "fetch", "--quiet", "--no-tags", "--depth=1", "origin", rev) + return err +} + +// copyCheckout copies the named files (every tracked one when nil) from the +// checkout into `scan`. +func copyCheckout(ctx context.Context, files []string, scan string) (int, error) { + if files == nil { + out, err := git(ctx, "", "ls-files") + if err != nil { + return 0, err + } + files = strings.Split(out, "\n") + } + root, err := git(ctx, "", "rev-parse", "--show-toplevel") + if err != nil { + return 0, err + } + copied := 0 + for _, file := range files { + if file == "" { + continue + } + // Symlinks and submodules are not content of their own. + info, err := os.Lstat(filepath.Join(root, file)) + if err != nil || !info.Mode().IsRegular() { + continue + } + if err := copyFile(filepath.Join(root, file), filepath.Join(scan, verify.ScanCopyPath(file))); err != nil { + return 0, err + } + copied++ + } + return copied, nil +} + +// copyCommit writes the named files (every one when nil) of commit `head`'s +// tree into `scan`, from git's objects: the commit is fetched if the +// checkout lacks it (a pull request's head, a fork's included, is reachable +// from the base repository), and nothing of it is checked out. +func copyCommit(ctx context.Context, head string, files []string, scan string) (int, error) { + if err := fetchCommit(ctx, head); err != nil { + return 0, err + } + listing, err := git(ctx, "", "ls-tree", "-r", "-z", "--full-tree", head) + if err != nil { + return 0, err + } + targets, problem := verify.ScanTargets(verify.ParseTree(listing), files) + if problem != "" { + return 0, errors.New("secret-scan: " + problem) + } + if len(targets) == 0 { + return 0, nil + } + var request strings.Builder + for _, target := range targets { + request.WriteString(target.Blob + "\n") + } + command := exec.CommandContext(ctx, "git", "cat-file", "--batch") + command.Stdin = strings.NewReader(request.String()) + out, err := command.StdoutPipe() + if err != nil { + return 0, fmt.Errorf("secret-scan: %w", err) + } + if err := command.Start(); err != nil { + return 0, fmt.Errorf("secret-scan: %w", err) + } + reader := bufio.NewReader(out) + for _, target := range targets { + if err := writeBlob(reader, target, filepath.Join(scan, filepath.FromSlash(verify.ScanCopyPath(target.Path)))); err != nil { + _ = command.Wait() + return 0, err + } + } + if err := command.Wait(); err != nil { + return 0, fmt.Errorf("git cat-file: %w", err) + } + return len(targets), nil +} + +// writeBlob reads one `git cat-file --batch` answer (` blob `, +// the content, a newline) into `to`. +func writeBlob(reader *bufio.Reader, target verify.TreeEntry, to string) error { + header, err := reader.ReadString('\n') + if err != nil { + return fmt.Errorf("git cat-file: %w", err) + } + fields := strings.Fields(header) + const headerFields = 3 // sha, type, size + if len(fields) != headerFields || fields[0] != target.Blob || fields[1] != "blob" { + return fmt.Errorf("git cat-file: %q for %s", strings.TrimSpace(header), target.Path) + } + size, err := strconv.ParseInt(fields[2], 10, 64) + if err != nil { + return fmt.Errorf("git cat-file: %w", err) + } + if err := os.MkdirAll(filepath.Dir(to), dirMode); err != nil { + return fmt.Errorf("secret-scan: %w", err) + } + file, err := os.OpenFile(to, os.O_CREATE|os.O_EXCL|os.O_WRONLY, fileMode) + if err != nil { + return fmt.Errorf("secret-scan: %w", err) + } + if _, err := io.CopyN(file, reader, size); err != nil { + _ = file.Close() + return fmt.Errorf("git cat-file: %w", err) + } + if err := file.Close(); err != nil { + return fmt.Errorf("secret-scan: %w", err) + } + if _, err := reader.Discard(1); err != nil { // the newline after the content + return fmt.Errorf("git cat-file: %w", err) + } + return nil +} + +// runGitleaks scans the copies in `scan` with the pinned gitleaks and the +// trusted policy in `policy` (verify.GitleaksArgs), with no GITLEAKS_* env, +// which would name another config. +func runGitleaks(ctx context.Context, scan, policy string, copied int) error { + binary, err := gitleaks(ctx) + if err != nil { + return err + } + fmt.Printf("Scanning %d file(s) with gitleaks %s.\n", copied, gitleaksVersion) + command := exec.CommandContext(ctx, binary, verify.GitleaksArgs(filepath.Join(policy, ".gitleaks.toml"), filepath.Join(policy, ".gitleaksignore"))...) + command.Dir = scan + command.Env = slices.DeleteFunc(os.Environ(), func(variable string) bool { return strings.HasPrefix(variable, "GITLEAKS_") }) + command.Stdout, command.Stderr = os.Stdout, os.Stderr + if err := command.Run(); err != nil { + return fmt.Errorf("the secret scan found credential-shaped values (above, redacted): remove a real secret and rotate it; a deliberate fixture gets an obviously fake value and an exact allowlist entry in .gitleaks.toml (docs/CI.md, \"Secret scan\"): %w", err) + } + return nil +} + +func copyFile(from, to string) error { + if err := os.MkdirAll(filepath.Dir(to), dirMode); err != nil { + return fmt.Errorf("secret-scan: %w", err) + } + content, err := os.ReadFile(from) + if err != nil { + return fmt.Errorf("secret-scan: %w", err) + } + if err := os.WriteFile(to, content, fileMode); err != nil { + return fmt.Errorf("secret-scan: %w", err) + } + return nil +} diff --git a/scripts/cilocal/verify/allowlist.go b/scripts/cilocal/verify/allowlist.go new file mode 100644 index 0000000..a1061fd --- /dev/null +++ b/scripts/cilocal/verify/allowlist.go @@ -0,0 +1,106 @@ +package verify + +import ( + "fmt" + "strings" +) + +// AllowlistProblem is why a gitleaks config's allowlist is too broad: it +// exempts a path without also requiring a value, so every credential-shaped +// value in that path (a live key in a test, a copied example env file, the +// bench script) would pass the scan. A path may only narrow a value +// exception: `paths` with `regexes` or `stopwords`, and `condition = "AND"`, +// because gitleaks ORs an allowlist's conditions by default. Value-only +// allowlists are always fine. +func AllowlistProblem(config string) string { + for _, section := range allowlistSections(config) { + if !allowlistHasEntries(section, "paths") { + continue + } + values := allowlistHasEntries(section, "regexes") || allowlistHasEntries(section, "stopwords") + if !values { + return fmt.Sprintf("%s exempts `paths` without `regexes` or `stopwords`: a credential in one of those paths would pass the scan", section.name) + } + if !strings.EqualFold(unquote(allowlistValue(section, "condition")), "AND") { + return fmt.Sprintf("%s exempts `paths` with values but not `condition = \"AND\"`: gitleaks ORs them, so a credential in one of those paths would pass the scan", section.name) + } + } + return "" +} + +// allowlistSection is one allowlist table of a gitleaks config and the lines +// after its header, up to the next table. +type allowlistSection struct { + name string + lines []string +} + +// allowlistSections reads the `[allowlist]`, `[rules.allowlist]` and +// `[[rules.allowlists]]` tables of a gitleaks config. +func allowlistSections(config string) []allowlistSection { + sections := []allowlistSection{} + current := allowlistSection{} + flush := func() { + if current.name != "" { + sections = append(sections, current) + } + } + for line := range strings.SplitSeq(config, "\n") { + trimmed := strings.TrimSpace(line) + if strings.HasPrefix(trimmed, "[") && strings.HasSuffix(trimmed, "]") { + flush() + current = allowlistSection{name: tableName(trimmed)} + continue + } + if current.name != "" && !strings.HasPrefix(trimmed, "#") { + current.lines = append(current.lines, line) + } + } + flush() + return sections +} + +// tableName is the table a header line names, or "" for one that is not an +// allowlist. `[[rules.allowlists]]` and `[[rules]]` keep their inner name. +func tableName(header string) string { + name := strings.TrimSpace(strings.TrimSuffix(strings.TrimPrefix(header, "["), "]")) + name = strings.TrimSpace(strings.TrimSuffix(strings.TrimPrefix(name, "["), "]")) + switch name { + case "allowlist", "rules.allowlist", "rules.allowlists": + return name + } + return "" +} + +// allowlistValue is the value assigned to `key` in the section: the whole +// array when it spans lines, else the rest of the line. +func allowlistValue(section allowlistSection, key string) string { + for index := 0; index < len(section.lines); index++ { + name, value, found := strings.Cut(strings.TrimSpace(section.lines[index]), "=") + if !found || strings.TrimSpace(name) != key { + continue + } + opened, closed := strings.Count(value, "["), strings.Count(value, "]") + for opened > closed && index+1 < len(section.lines) { + index++ + value += "\n" + section.lines[index] + opened, closed = strings.Count(value, "["), strings.Count(value, "]") + } + return value + } + return "" +} + +// allowlistHasEntries reports whether the key's array has any entry: `[]`, +// `[""]` and a missing key hold none. +func allowlistHasEntries(section allowlistSection, key string) bool { + value := strings.TrimSpace(allowlistValue(section, key)) + value = strings.TrimSuffix(strings.TrimPrefix(value, "["), "]") + for _, char := range value { + if strings.ContainsRune(" \t\r\n,", char) || char == '\'' || char == '"' { + continue + } + return true + } + return false +} diff --git a/scripts/cilocal/verify/allowlist_test.go b/scripts/cilocal/verify/allowlist_test.go new file mode 100644 index 0000000..0e6c0b3 --- /dev/null +++ b/scripts/cilocal/verify/allowlist_test.go @@ -0,0 +1,76 @@ +package verify_test + +import ( + "os" + "path/filepath" + "strings" + "testing" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +// A path in an allowlist may only narrow a value exception: exemption by +// path alone would let any credential in a test, testdata, an example env +// file or the bench script pass the scan. Randy's re-review of #134: the same +// synthetic token passed in all five of those locations. +func TestAllowlistProblem(test *testing.T) { + test.Parallel() + cases := map[string]struct { + config string + want string + }{ + "path only": { + config: "[extend]\nuseDefault = true\n[allowlist]\npaths = [\n '''_test\\.go$''',\n '''(^|/)testdata/''',\n]\n", + want: "`paths` without `regexes` or `stopwords`", + }, + "path and values are ORed without condition": { + config: "[extend]\nuseDefault = true\n[allowlist]\npaths = ['''^scripts/bench-devloop\\.sh$''']\nregexes = ['''^bench-secret$''']\n", + want: "not `condition = \"AND\"`", + }, + "path only in a rule allowlist": { + config: "[extend]\nuseDefault = true\n[[rules]]\nid = \"generic-api-key\"\n[[rules.allowlists]]\ncondition = \"AND\"\npaths = ['''\\.env\\.example$''']\n", + want: "`paths` without `regexes` or `stopwords`", + }, + "path narrowed by a value and AND": { + config: "[extend]\nuseDefault = true\n[[rules]]\nid = \"generic-api-key\"\n[[rules.allowlists]]\ncondition = \"AND\"\npaths = ['''^scripts/bench-devloop\\.sh$''']\nregexes = ['''^bench-secret$''']\n", + }, + "path narrowed by stopwords and AND": { + config: "[extend]\nuseDefault = true\n[allowlist]\ncondition = \"AND\"\npaths = ['''(^|/)docs/''']\nstopwords = ['example']\n", + }, + "value only": { + config: "[extend]\nuseDefault = true\n[allowlist]\nregexes = ['''^bench-secret$''']\n", + }, + "empty paths": { + config: "[extend]\nuseDefault = true\n[allowlist]\npaths = []\n", + }, + "rule table after an allowlist ends it": { + config: "[extend]\nuseDefault = true\n[allowlist]\nregexes = ['''^bench-secret$''']\n[[rules]]\nid = \"generic-api-key\"\n", + }, + } + for name, testCase := range cases { + problem := verify.AllowlistProblem(testCase.config) + if testCase.want == "" { + if problem != "" { + test.Errorf("%s: AllowlistProblem = %q, want none", name, problem) + } + continue + } + if !strings.Contains(problem, testCase.want) { + test.Errorf("%s: AllowlistProblem = %q, want it to name %q", name, problem, testCase.want) + } + } +} + +// The repository's own policy keeps that rule: the scan's allowlist names +// synthetic values, never a path on its own. A blanket path allowlist fails +// here before it can make a real credential invisible. +func TestScannerPolicyAllowlistsValuesOnly(test *testing.T) { + test.Parallel() + config, err := os.ReadFile(filepath.Join("..", "..", "..", ".gitleaks.toml")) + if err != nil { + test.Fatal(err) + } + if problem := verify.AllowlistProblem(string(config)); problem != "" { + test.Fatalf(".gitleaks.toml allows a whole path instead of a fixture value: %s", problem) + } +} diff --git a/scripts/cilocal/verify/doc.go b/scripts/cilocal/verify/doc.go new file mode 100644 index 0000000..afb2c48 --- /dev/null +++ b/scripts/cilocal/verify/doc.go @@ -0,0 +1,11 @@ +// Package verify holds the rules of local verification (docs/CI.md, "Local +// verification"): the agent runs what its change needs, then `make ci-local +// RAN=""` attests it on the exact pushed head. This package +// renders the PR body section that attestation fills and decides, for the +// Local verification job in ci.yml, `make pr-merge` and the staging gate, +// whether a head is attested. It also holds the merge decision, the staging +// verdict after a merge, the migration order and the post-merge report's +// workflow list. It is pure: git, gh and HTTP run in scripts/cilocal, which +// passes their results in. Its tests are a test boundary (AGENTS.md "Test +// boundaries"). +package verify diff --git a/scripts/cilocal/verify/head.go b/scripts/cilocal/verify/head.go new file mode 100644 index 0000000..035e889 --- /dev/null +++ b/scripts/cilocal/verify/head.go @@ -0,0 +1,171 @@ +package verify + +import ( + "fmt" + "regexp" + "slices" + "strconv" + "strings" +) + +// Head is the checkout make ci-local was started in. +type Head struct { + SHA string + Branch string // empty when detached + Dirty string // `git status --porcelain`, empty when clean + Commit string // the resolved COMMIT sha, or empty + MainCommit bool // attesting a commit of the base branch (COMMIT, or on it) + Pushed string // the remote branch's head, or empty when it has none + OnBase bool // SHA is reachable from origin/ + Base string // the branch it lands on: main (scripts/branches.sh) +} + +// Refusal is why make ci-local may not attest the head, or "": only a clean +// checkout whose HEAD is exactly the pushed branch head (or, with COMMIT, a +// commit of the base branch checked out detached) is attested. +func (head Head) Refusal() string { + sha := Short(head.SHA) + if head.Dirty != "" { + return "the worktree has uncommitted changes; commit and push them first:\n" + head.Dirty + } + if head.MainCommit { + if head.Commit != "" && head.Commit != head.SHA { + return fmt.Sprintf("HEAD is %s, not %s; check it out in a clean worktree (git worktree add --detach %s)", sha, Short(head.Commit), Short(head.Commit)) + } + if !head.OnBase { + return fmt.Sprintf("%s is not on origin/%s; COMMIT attests only a commit of the base branch", sha, head.Base) + } + return "" + } + if head.Branch == "" { + return "HEAD is detached; check out the PR branch, or pass COMMIT= for a commit of the base branch" + } + if head.Pushed != head.SHA { + pushed := "missing" + if head.Pushed != "" { + pushed = Short(head.Pushed) + } + return fmt.Sprintf("origin/%s is %s, not HEAD %s; push first", head.Branch, pushed, sha) + } + return "" +} + +// PublishRefusal is why make ci-local may not post `report`, or "": it reads +// the checkout again just before it posts (`head`, `tree`), and the report +// must be for exactly that clean, pushed head and its tree. One rule for +// every way a report is posted, so none can publish for a worktree that +// changed or a HEAD that moved after the report was made. +func PublishRefusal(report Report, head Head, tree string) string { + if refusal := head.Refusal(); refusal != "" { + return refusal + } + if report.Head != head.SHA { + return fmt.Sprintf("the report is for %s, but HEAD is now %s", Short(report.Head), Short(head.SHA)) + } + if report.Tree != tree { + return fmt.Sprintf("the report is for tree %s, but HEAD's tree is %s", Short(report.Tree), Short(tree)) + } + return "" +} + +// AttestedAncestorLimit is how many first-parent ancestors of a main commit +// make ci-local reads for the last attested one. +const AttestedAncestorLimit = 50 + +// LastAttested is the nearest of a main commit's first-parent `ancestors` +// (nearest first) the staging gate accepts, or "" when none is: a new +// project's first attestation, or a longer run of unattested commits. +func LastAttested(ancestors []string, attested func(string) bool) string { + for _, ancestor := range ancestors { + if attested(ancestor) { + return ancestor + } + } + return "" +} + +// FullCheck is the check over the whole tree a first attestation runs. +const FullCheck = "make ci" + +// RanFullCheck is whether a statement names FullCheck: a command (parts +// joined with `;` or `&&`) that runs make with the target ci, directly or +// through a wrapper (`davit box run -- make ci`). +func RanFullCheck(ran string) bool { + for _, part := range strings.FieldsFunc(strings.ReplaceAll(ran, "&&", ";"), func(char rune) bool { return char == ';' }) { + fields := strings.Fields(part) + for index, field := range fields { + if field == "make" && slices.Contains(fields[index+1:], "ci") { + return true + } + } + } + return false +} + +// FirstAttestationProblem is why a main commit's statement `ran` may not +// attest it, or "": with no attested main commit before it (`last` empty) +// nothing narrows what changed, so it must have run the whole check. +func FirstAttestationProblem(sha, last, ran string) string { + if last != "" || RanFullCheck(ran) { + return "" + } + return fmt.Sprintf("no attested main commit among the %d before %s (a new project's first attestation, or a longer unattested run), so its statement covers the whole tree: run %s on it, then make ci-local COMMIT=%s RAN=%q", + AttestedAncestorLimit, Short(sha), FullCheck, Short(sha), FullCheck) +} + +// PullAttestation is a merged pull request whose merge commit is the commit +// being judged, with what GitHub holds for its merged head. +type PullAttestation struct { + Number int + Verification +} + +// Attestation is the staging gate's verdict on a commit of a deployed +// branch: attested when `local-ci` passed on the commit itself (`status`, +// its state there, or ""), or when it is the merge of a pull request whose +// merged head holds a valid local verification. A direct push, or the merge +// of an unattested head, is not. +func Attestation(sha, status string, pulls []PullAttestation) (string, []string) { + if status == "success" { + return fmt.Sprintf("%s on %s itself", StatusContext, sha), nil + } + reasons := []string{} + if status != "" { + reasons = append(reasons, fmt.Sprintf("%s has %s %s", sha, StatusContext, status)) + } + for _, pull := range pulls { + problems := LocalVerificationProblems(pull.Verification) + if len(problems) == 0 { + return fmt.Sprintf("%s on #%d's merged head %s", StatusContext, pull.Number, pull.Head), nil + } + reasons = append(reasons, fmt.Sprintf("#%d's merged head %s is not attested: %s", pull.Number, pull.Head, strings.Join(problems, "; "))) + } + if len(pulls) == 0 { + reasons = append(reasons, fmt.Sprintf("%s has no passed %s status and no merged pull request (a direct push)", sha, StatusContext)) + } + return "", reasons +} + +// AttestNextStep is how to attest an unattested main commit, so the staging +// gate passes it. +func AttestNextStep(sha string) string { + return fmt.Sprintf("in a clean worktree of it (git worktree add --detach %s), run what the commits since the last attested main commit need (%s when there is none, as in a new project), then make ci-local COMMIT=%s RAN=\"\" there: it posts local-ci on %s. Then dispatch Deploy staging (Staging catch-up also dispatches it within the hour)", + sha, FullCheck, sha, Short(sha)) +} + +var squashTitle = regexp.MustCompile(`\(#(\d+)\)$`) + +// TitledPull is the pull request a squash commit's title names (`... (#12)`), +// or 0. GitHub's commit-to-PR index can trail a fresh merge. +func TitledPull(message string) int { + title, _, _ := strings.Cut(message, "\n") + match := squashTitle.FindStringSubmatch(strings.TrimSpace(title)) + if match == nil { + return 0 + } + number, err := strconv.Atoi(match[1]) + if err != nil { + return 0 + } + return number +} diff --git a/scripts/cilocal/verify/head_test.go b/scripts/cilocal/verify/head_test.go new file mode 100644 index 0000000..30b7dfe --- /dev/null +++ b/scripts/cilocal/verify/head_test.go @@ -0,0 +1,221 @@ +package verify_test + +import ( + "strings" + "testing" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +func TestHeadRefusal(test *testing.T) { + test.Parallel() + branch := verify.Head{SHA: headSHA, Branch: "feat/x", Pushed: headSHA, Base: "main"} + mainCommit := verify.Head{SHA: headSHA, MainCommit: true, OnBase: true, Base: "main"} + for _, testCase := range []struct { + name string + head func(verify.Head) verify.Head + from verify.Head + want string // "" when it may attest + }{ + {"clean pushed branch", func(head verify.Head) verify.Head { return head }, branch, ""}, + {"dirty", func(head verify.Head) verify.Head { head.Dirty = " M main.go"; return head }, branch, "uncommitted changes; commit and push them first:\n M main.go"}, + {"dirty main commit", func(head verify.Head) verify.Head { head.Dirty = "?? new"; return head }, mainCommit, "uncommitted changes"}, + {"detached", func(head verify.Head) verify.Head { head.Branch = ""; return head }, branch, "HEAD is detached"}, + {"not pushed", func(head verify.Head) verify.Head { head.Pushed = ""; return head }, branch, "origin/feat/x is missing, not HEAD aaaaaaaaaaaa; push first"}, + {"pushed elsewhere", func(head verify.Head) verify.Head { head.Pushed = otherSHA; return head }, branch, "origin/feat/x is bbbbbbbbbbbb, not HEAD aaaaaaaaaaaa"}, + {"main commit on base", func(head verify.Head) verify.Head { return head }, mainCommit, ""}, + {"main commit matching --commit", func(head verify.Head) verify.Head { head.Commit = headSHA; return head }, mainCommit, ""}, + {"main commit not on base", func(head verify.Head) verify.Head { head.OnBase = false; return head }, mainCommit, "aaaaaaaaaaaa is not on origin/main"}, + {"--commit mismatch", func(head verify.Head) verify.Head { head.Commit = otherSHA; return head }, mainCommit, "HEAD is aaaaaaaaaaaa, not bbbbbbbbbbbb"}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + got := testCase.head(testCase.from).Refusal() + if testCase.want == "" { + if got != "" { + test.Fatalf("Refusal = %q, want none", got) + } + return + } + if !strings.Contains(got, testCase.want) { + test.Fatalf("Refusal = %q, want it to mention %q", got, testCase.want) + } + }) + } +} + +func TestAttestation(test *testing.T) { + test.Parallel() + const sha = "deadbeef" + unverified := verified() + unverified.Status = nil + for _, testCase := range []struct { + name string + status string + pulls []verify.PullAttestation + verdict string + mentions []string + }{ + {"status on the commit", "success", nil, "local-ci on deadbeef itself", nil}, + {"attested merged pull", "", []verify.PullAttestation{{Number: 7, Verification: verified()}}, "local-ci on #7's merged head " + headSHA, nil}, + {"second pull attested", "", []verify.PullAttestation{{Number: 6, Verification: unverified}, {Number: 7, Verification: verified()}}, "#7's merged head", nil}, + {"direct push", "", nil, "", []string{"deadbeef has no passed local-ci status and no merged pull request (a direct push)"}}, + {"failed direct push", "failure", nil, "", []string{"deadbeef has local-ci failure", "a direct push"}}, + {"unattested merged pull", "pending", []verify.PullAttestation{{Number: 6, Verification: unverified}}, "", + []string{"deadbeef has local-ci pending", "#6's merged head " + headSHA + " is not attested: no local-ci status"}}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + verdict, reasons := verify.Attestation(sha, testCase.status, testCase.pulls) + if testCase.verdict != "" { + if !strings.Contains(verdict, testCase.verdict) || reasons != nil { + test.Fatalf("Attestation = %q, %q; want attested %q", verdict, reasons, testCase.verdict) + } + return + } + if verdict != "" { + test.Fatalf("Attestation attested %q", verdict) + } + for _, mention := range testCase.mentions { + requireMention(test, reasons, mention) + } + if len(reasons) != len(testCase.mentions) { + test.Fatalf("reasons = %q, want %d", reasons, len(testCase.mentions)) + } + }) + } +} + +func TestTitledPull(test *testing.T) { + test.Parallel() + for _, testCase := range []struct { + message string + want int + }{ + {"fix(config): refuse an unset FROM_EMAIL (#103)", 103}, + {"fix(config): refuse (#103) \n\nbody (#5)", 103}, + {"feat: no number", 0}, + {"feat: (#12) not at the end", 0}, + {"feat: x\n\nmerged (#12)", 0}, + {"feat: overflow (#99999999999999999999999)", 0}, + {"", 0}, + } { + test.Run(testCase.message, func(test *testing.T) { + test.Parallel() + if got := verify.TitledPull(testCase.message); got != testCase.want { + test.Fatalf("TitledPull(%q) = %d, want %d", testCase.message, got, testCase.want) + } + }) + } +} + +func TestAttestNextStep(test *testing.T) { + test.Parallel() + got := verify.AttestNextStep(headSHA) + for _, want := range []string{"git worktree add --detach " + headSHA, "make ci-local COMMIT=" + headSHA + ` RAN=""`, "dispatch Deploy staging"} { + if !strings.Contains(got, want) { + test.Fatalf("AttestNextStep = %q, want it to mention %q", got, want) + } + } +} + +// Randy's review of #134 (TestRejectedDirtyTreeReportCanBePublished): a run +// refused for worktree changes still published its report as success. The +// checkout is read again just before posting, and a report is posted only +// for exactly that clean, pushed head and tree. +func TestPublishRefusal(test *testing.T) { + test.Parallel() + report := passingReport() + tree := report.Tree + clean := verify.Head{SHA: headSHA, Branch: "feat/x", Pushed: headSHA, Base: "main"} + for _, testCase := range []struct { + name string + head func(verify.Head) verify.Head + tree string + want string // "" when it may post + }{ + {"clean and unchanged", func(head verify.Head) verify.Head { return head }, tree, ""}, + {"a file changed since the report", func(head verify.Head) verify.Head { head.Dirty = " M README.md"; return head }, tree, "uncommitted changes"}, + {"HEAD moved", func(head verify.Head) verify.Head { head.SHA, head.Pushed = otherSHA, otherSHA; return head }, tree, "the report is for aaaaaaaaaaaa, but HEAD is now bbbbbbbbbbbb"}, + {"not pushed any more", func(head verify.Head) verify.Head { head.Pushed = otherSHA; return head }, tree, "push first"}, + {"another tree", func(head verify.Head) verify.Head { return head }, strings.Repeat("f", 40), "the report is for tree dddddddddddd, but HEAD's tree is ffffffffffff"}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + got := verify.PublishRefusal(report, testCase.head(clean), testCase.tree) + if testCase.want == "" { + if got != "" { + test.Fatalf("PublishRefusal = %q, want none", got) + } + return + } + if !strings.Contains(got, testCase.want) { + test.Fatalf("PublishRefusal = %q, want it to mention %q", got, testCase.want) + } + }) + } +} + +func TestLastAttested(test *testing.T) { + test.Parallel() + attestedSet := map[string]bool{"m2": true, "m4": true} + isAttested := func(sha string) bool { return attestedSet[sha] } + for _, testCase := range []struct { + name string + ancestors []string + want string + }{ + {"nearest attested ancestor", []string{"m1", "m2", "m3", "m4"}, "m2"}, + {"none attested", []string{"m1", "m3"}, ""}, + {"a root commit", nil, ""}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + if got := verify.LastAttested(testCase.ancestors, isAttested); got != testCase.want { + test.Fatalf("LastAttested = %q, want %q", got, testCase.want) + } + }) + } +} + +func TestRanFullCheck(test *testing.T) { + test.Parallel() + for _, testCase := range []struct { + ran string + want bool + }{ + {"make ci", true}, + {"make lint; make ci", true}, + {"go build ./... && make ci", true}, + {"davit box run -- make ci", true}, + {"make -j4 ci", true}, + {"make ci-local", false}, + {"make lint test", false}, + {"ci", false}, + {"", false}, + } { + if got := verify.RanFullCheck(testCase.ran); got != testCase.want { + test.Errorf("RanFullCheck(%q) = %v, want %v", testCase.ran, got, testCase.want) + } + } +} + +// Randy's review of #134 (TestNoAttestedAncestorHasNoBootstrapBase): a new +// project has no attested main commit, and the first attestation was +// refused before anything ran. It is allowed, and it must cover the whole +// tree: with no attested main commit before it, the statement names make ci. +func TestFirstAttestationProblem(test *testing.T) { + test.Parallel() + if got := verify.FirstAttestationProblem(headSHA, "", "make ci"); got != "" { + test.Fatalf("a first attestation that ran make ci is refused: %q", got) + } + if got := verify.FirstAttestationProblem(headSHA, otherSHA, "make lint"); got != "" { + test.Fatalf("an attestation since an attested main commit is refused: %q", got) + } + got := verify.FirstAttestationProblem(headSHA, "", "make lint test") + for _, want := range []string{"no attested main commit among the 50 before aaaaaaaaaaaa", "run make ci on it", `make ci-local COMMIT=aaaaaaaaaaaa RAN="make ci"`} { + if !strings.Contains(got, want) { + test.Errorf("FirstAttestationProblem = %q, want it to mention %q", got, want) + } + } +} diff --git a/scripts/cilocal/verify/helpers_test.go b/scripts/cilocal/verify/helpers_test.go new file mode 100644 index 0000000..bb41cc2 --- /dev/null +++ b/scripts/cilocal/verify/helpers_test.go @@ -0,0 +1,65 @@ +package verify_test + +import ( + "slices" + "strings" + "testing" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +// Commit shas as GitHub reports them: 40 lowercase hex characters. +var ( + headSHA = strings.Repeat("a", 40) + otherSHA = strings.Repeat("b", 40) +) + +func success(description string) verify.Status { + return verify.Status{Context: verify.StatusContext, State: "success", Description: description, CreatedAt: "2026-10-31T10:00:00Z"} +} + +// passingReport is an attestation of headSHA. +func passingReport() verify.Report { + return verify.Report{ + Head: headSHA, + Tree: strings.Repeat("d", 40), + Author: "JJ Eaton", + Hostname: "laptop", + FinishedAt: "2026-10-31T10:00:00Z", + Result: verify.Passed, + Ran: "make lint test TEST_PKGS=./scripts/cilocal/verify", + Failed: "", + } +} + +// verified is a pull request whose head headSHA was attested and whose body +// and status hold for it. +func verified() verify.Verification { + report := passingReport() + section := verify.RenderSection(report) + status := success(verify.StatusDescription(report, section)) + return verify.Verification{ + Body: verify.WithSection("## Problem\n", section), + Head: headSHA, + Status: &status, + Statuses: []verify.Status{status}, + } +} + +func containsText(list []string, text string) bool { + return slices.ContainsFunc(list, func(entry string) bool { return strings.Contains(entry, text) }) +} + +func requireNone(test *testing.T, problems []string) { + test.Helper() + if len(problems) != 0 { + test.Fatalf("want no problems, got %q", problems) + } +} + +func requireMention(test *testing.T, problems []string, text string) { + test.Helper() + if !containsText(problems, text) { + test.Fatalf("want a problem mentioning %q, got %q", text, problems) + } +} diff --git a/scripts/cilocal/verify/merge.go b/scripts/cilocal/verify/merge.go new file mode 100644 index 0000000..af2e5cc --- /dev/null +++ b/scripts/cilocal/verify/merge.go @@ -0,0 +1,265 @@ +package verify + +import ( + "fmt" + "regexp" + "strconv" + "strings" +) + +// The merge decision (make pr-merge, docs/CI.md "Merging on local +// verification"): each listed pull request merges now or is reported with +// its next step. Nothing waits: a draft, unattested, failing, pending or +// conflicting PR is left alone so it never holds up the others, and a head +// behind main merges unless it conflicts. + +// Main is the branch pull requests land on (scripts/branches.sh). +const Main = "main" + +// The checks the main ruleset requires (ci.yml). +const ( + // Gate is the check of the make ci-local attestation. + Gate = "Local verification" + // AlwaysRuns runs on every pull request, docs included. + AlwaysRuns = "Secret scan" +) + +// Check is one entry of a pull request's statusCheckRollup (gh pr view +// --json statusCheckRollup): a check run of a workflow job, or a commit +// status (`local-ci`). +type Check struct { + Typename string `json:"__typename"` // CheckRun or StatusContext + Name string `json:"name"` + Context string `json:"context"` // a StatusContext's name + State string `json:"state"` // a StatusContext's result + WorkflowName string `json:"workflowName"` + Status string `json:"status"` + Conclusion string `json:"conclusion"` + DetailsURL string `json:"detailsUrl"` + StartedAt string `json:"startedAt"` + // NotStarted is a job GitHub failed without starting it (Actions + // blocked by billing or a spending limit): it never ran, so it counts + // as no run, not as a failure. Set from the job's annotations + // (NotStartedMessage). + NotStarted bool `json:"-"` +} + +// notStarted is how GitHub's annotation on such a job begins: "The job was +// not started because recent account payments have failed or your spending +// limit needs to be increased." +const notStarted = "The job was not started" + +// NotStartedMessage is whether an annotation of a failed job says GitHub +// never started it. +func NotStartedMessage(message string) bool { return strings.HasPrefix(message, notStarted) } + +var ( + jobID = regexp.MustCompile(`/job/(\d+)`) + runID = regexp.MustCompile(`/actions/runs/(\d+)/`) +) + +func number(pattern *regexp.Regexp, text string) int { + match := pattern.FindStringSubmatch(text) + if match == nil { + return 0 + } + value, _ := strconv.Atoi(match[1]) + return value +} + +// JobID is the check run id of a workflow job's check (its details URL ends +// in /job/), or 0. +func (check Check) JobID() int { return number(jobID, check.DetailsURL) } + +func (check Check) run() int { return number(runID, check.DetailsURL) } + +// CheckName is what the check is called: the job's name or the status's context. +func (check Check) CheckName() string { + switch { + case check.Name != "": + return check.Name + case check.Context != "": + return check.Context + } + return "unnamed check" +} + +// result is the check's outcome as one word: a status's state, a finished +// job's conclusion, or PENDING. +func (check Check) result() string { + switch { + case check.Typename == "StatusContext": + return check.State + case check.Status == "COMPLETED": + return check.Conclusion + } + return "PENDING" +} + +// CIWorkflow is the name of ci.yml, whose jobs check a pull request. +const CIWorkflow = "CI" + +// LatestChecks keeps what counts of a head's rollup: it holds every run on +// the head, so a run cancelled by a newer edit sits next to the run that +// counts. Only the newest run of each workflow is current, and within it +// only the newest attempt of each job (a re-run adds a copy). +func LatestChecks(rollup []Check) []Check { + newest := map[string]int{} + for _, check := range rollup { + if check.WorkflowName != "" { + newest[check.WorkflowName] = max(newest[check.WorkflowName], check.run()) + } + } + latest := []Check{} + index := map[string]int{} + for _, check := range rollup { + if check.WorkflowName != "" && check.run() != newest[check.WorkflowName] { + continue + } + key := check.WorkflowName + "\x00" + check.CheckName() + seen, found := index[key] + switch { + case !found: + index[key] = len(latest) + latest = append(latest, check) + case check.StartedAt >= latest[seen].StartedAt: + latest[seen] = check + } + } + return latest +} + +var ( + passedResults = map[string]bool{"SUCCESS": true, "NEUTRAL": true, "SKIPPED": true, "success": true} + failedResults = map[string]bool{ + "FAILURE": true, "TIMED_OUT": true, "CANCELLED": true, "ACTION_REQUIRED": true, + "STARTUP_FAILURE": true, "ERROR": true, "failure": true, "error": true, + } +) + +// Checks is a head's rollup sorted: failed and pending check names, whether +// any workflow job ran at all, and whether the green is hollow (the gate +// passed but the secret scan was skipped). +type Checks struct { + Failed []string + Pending []string + Ran bool + Hollow bool +} + +// ClassifyChecks sorts the newest result of each check (LatestChecks). A +// job GitHub never started counts as not run. Without the gate, a head with +// nothing failed is pending. +func ClassifyChecks(rollup []Check) Checks { + var sorted Checks + gate, alwaysRuns := "", "" + for _, check := range LatestChecks(rollup) { + if check.NotStarted { + continue + } + name, result := check.CheckName(), check.result() + if check.Typename != "StatusContext" { + sorted.Ran = true + } + switch name { + case Gate: + gate = result + case AlwaysRuns: + alwaysRuns = result + } + switch { + case failedResults[result]: + sorted.Failed = append(sorted.Failed, name) + case !passedResults[result]: + sorted.Pending = append(sorted.Pending, name) + } + } + if sorted.Ran && gate == "" && len(sorted.Failed) == 0 { + sorted.Pending = append(sorted.Pending, Gate) + } + sorted.Hollow = gate == "SUCCESS" && alwaysRuns == "SKIPPED" + return sorted +} + +// Pull is what the merge script reads of a pull request. +type Pull struct { + Number int + State string // OPEN, CLOSED, MERGED + Draft bool + Base string + Mergeable string // MERGEABLE, CONFLICTING, UNKNOWN + Checks []Check + Verification +} + +// Merge options. +type MergeOptions struct { + AnyBase bool // merge a PR stacked on another branch + RequireActions bool // refuse when no GitHub CI job ran on the head +} + +// Actions of a verdict. +const ( + ActionDone = "done" + ActionMerge = "merge" + ActionSkip = "skip" +) + +// Verdict is what to do with one pull request now: done, merge, or skip +// with the reason and the next step. Never wait. +type Verdict struct { + Action string + Reason string + Next string + Note string // how GitHub's own checks counted, for a merge +} + +func skip(reason, next string) Verdict { + return Verdict{Action: ActionSkip, Reason: reason, Next: next, Note: ""} +} + +// Decide is the merge decision for one pull request. It merges on local +// verification of the exact head (LocalVerificationProblems) and on GitHub's +// checks where they ran. When no CI job ran on the head (a repository whose +// Actions are off, or blocked so GitHub fails every job without starting +// it), `local-ci` alone decides, with a note, unless RequireActions. +func Decide(pull Pull, options MergeOptions) Verdict { + switch { + case pull.State == "MERGED": + return Verdict{Action: ActionDone, Reason: "already merged", Next: "", Note: ""} + case pull.State != "OPEN": + return skip("is "+strings.ToLower(pull.State), "nothing; never reopen, open a new PR") + case pull.Draft: + return skip("is a draft", fmt.Sprintf("gh pr ready %d", pull.Number)) + case pull.Base != Main && !options.AnyBase: + return skip("is stacked on "+pull.Base, + fmt.Sprintf("merge its parent first (make pr-merge retargets it to %s), or gh pr edit %d --base %s", Main, pull.Number, Main)) + case pull.Mergeable == "CONFLICTING": + // A head behind main still merges when GitHub can apply it cleanly: + // only a conflict stops it. + return skip("conflicts with "+pull.Base, fmt.Sprintf("git rebase origin/%s, push, %s", pull.Base, NextStep)) + } + if problems := LocalVerificationProblems(pull.Verification); len(problems) > 0 { + return skip("is not verified locally: "+strings.Join(problems, "; "), NextStep) + } + checks := ClassifyChecks(pull.Checks) + note := "GitHub's checks are green" + switch { + case !checks.Ran && options.RequireActions: + return skip("has no GitHub CI job that ran on the head (Actions off, or blocked so GitHub never started them)", + "edit the PR body once to start CI, or drop --require-actions where Actions cannot run") + case !checks.Ran: + note = "no GitHub CI job ran on the head (Actions off, or blocked so GitHub never started them): merging on local-ci alone" + case len(checks.Failed) > 0: + return skip("has failing checks: "+strings.Join(checks.Failed, ", "), + "fix and push, or for a lost runner gh run rerun --failed once") + case checks.Hollow: + return skip("is green only because CI skipped every job, the secret scan included", "push a new commit and attest it to start a full PR run") + case len(checks.Pending) > 0: + return skip("has checks still running: "+strings.Join(checks.Pending, ", "), "run this again when they finish; nothing else waits on it") + } + if pull.Mergeable != "MERGEABLE" { + return skip("mergeability is still being computed", "run this again in a minute") + } + return Verdict{Action: ActionMerge, Reason: "attested on " + Short(pull.Head), Next: "", Note: note} +} diff --git a/scripts/cilocal/verify/merge_test.go b/scripts/cilocal/verify/merge_test.go new file mode 100644 index 0000000..bacc00d --- /dev/null +++ b/scripts/cilocal/verify/merge_test.go @@ -0,0 +1,148 @@ +package verify_test + +import ( + "slices" + "strings" + "testing" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +func ciCheck(name, run, status, conclusion string) verify.Check { + return verify.Check{ + Typename: "CheckRun", Name: name, WorkflowName: verify.CIWorkflow, Status: status, Conclusion: conclusion, + DetailsURL: "https://github.com/jayleaton/go-fast-stack/actions/runs/" + run + "/job/1", StartedAt: "2026-10-31T10:00:00Z", + } +} + +func notStarted(name string) verify.Check { + check := ciCheck(name, "205", "COMPLETED", "FAILURE") + check.NotStarted = true + return check +} + +// green is a PR run where every job passed. +func green() []verify.Check { + return []verify.Check{ + ciCheck(verify.AlwaysRuns, "205", "COMPLETED", "SUCCESS"), + ciCheck("Local verification claim", "205", "COMPLETED", "SUCCESS"), + ciCheck(verify.Gate, "205", "COMPLETED", "SUCCESS"), + {Typename: "StatusContext", Context: verify.StatusContext, State: "SUCCESS"}, + } +} + +func TestNotStartedReadsGitHubsAnnotationAndTheJobID(test *testing.T) { + test.Parallel() + if !verify.NotStartedMessage("The job was not started because recent account payments have failed or your spending limit needs to be increased. Please check the 'Billing & plans' section in your settings") { + test.Fatal("the billing annotation is not recognised") + } + if verify.NotStartedMessage("Process completed with exit code 1.") { + test.Fatal("a job's own failure counts as not started") + } + if got := ciCheck("x", "205", "COMPLETED", "FAILURE").JobID(); got != 1 { + test.Fatalf("JobID = %d, want 1", got) + } + if got := (verify.Check{DetailsURL: "https://github.com/o/r/pull/1#local-verification"}).JobID(); got != 0 { + test.Fatalf("JobID without a job = %d, want 0", got) + } + if got := (verify.Check{}).CheckName(); got != "unnamed check" { + test.Fatalf("CheckName of an empty check = %q", got) + } +} + +func TestLatestChecksKeepsTheNewestRunAndAttempt(test *testing.T) { + test.Parallel() + rerun := ciCheck("lint", "205", "COMPLETED", "SUCCESS") + rerun.StartedAt = "2026-10-31T11:00:00Z" + rollup := []verify.Check{ + ciCheck("build", "100", "COMPLETED", "CANCELLED"), + ciCheck("build", "205", "COMPLETED", "SUCCESS"), + ciCheck("lint", "205", "COMPLETED", "FAILURE"), + rerun, + ciCheck("lint", "99", "COMPLETED", "FAILURE"), + {Typename: "CheckRun", Name: "deploy", WorkflowName: "Deploy", Status: "COMPLETED", DetailsURL: "https://github.com/o/r/actions/runs/999/job/1"}, + {Typename: "StatusContext", Context: verify.StatusContext, State: "SUCCESS"}, + } + names := []string{} + for _, check := range verify.LatestChecks(rollup) { + names = append(names, check.CheckName()+":"+check.Conclusion+check.State) + } + if !slices.Equal(names, []string{"build:SUCCESS", "lint:SUCCESS", "deploy:", "local-ci:SUCCESS"}) { + test.Fatalf("LatestChecks = %q", names) + } + if got := verify.LatestChecks(nil); len(got) != 0 { + test.Fatalf("LatestChecks(nil) = %+v", got) + } +} + +func TestDecide(test *testing.T) { + test.Parallel() + ready := verify.Pull{Number: 12, State: "OPEN", Base: verify.Main, Mergeable: "MERGEABLE", Checks: green(), Verification: verified()} + unattested := verified() + unattested.Status = nil + for _, testCase := range []struct { + name string + pull func(verify.Pull) verify.Pull + options verify.MergeOptions + action string + reason string // or the note, for a merge + next string + }{ + {"green", func(pull verify.Pull) verify.Pull { return pull }, verify.MergeOptions{}, verify.ActionMerge, "GitHub's checks are green", ""}, + {"merged", func(pull verify.Pull) verify.Pull { pull.State = "MERGED"; return pull }, verify.MergeOptions{}, verify.ActionDone, "already merged", ""}, + {"closed", func(pull verify.Pull) verify.Pull { pull.State = "CLOSED"; return pull }, verify.MergeOptions{}, verify.ActionSkip, "is closed", "never reopen"}, + {"draft", func(pull verify.Pull) verify.Pull { pull.Draft = true; return pull }, verify.MergeOptions{}, verify.ActionSkip, "is a draft", "gh pr ready 12"}, + {"stacked", func(pull verify.Pull) verify.Pull { pull.Base = "feat/a"; return pull }, verify.MergeOptions{}, verify.ActionSkip, "is stacked on feat/a", "gh pr edit 12 --base main"}, + {"stacked, any base", func(pull verify.Pull) verify.Pull { pull.Base = "feat/a"; return pull }, verify.MergeOptions{AnyBase: true}, verify.ActionMerge, "green", ""}, + {"conflicting", func(pull verify.Pull) verify.Pull { pull.Mergeable = "CONFLICTING"; return pull }, verify.MergeOptions{}, verify.ActionSkip, "conflicts with main", "git rebase origin/main"}, + {"unattested", func(pull verify.Pull) verify.Pull { pull.Verification = unattested; return pull }, verify.MergeOptions{}, verify.ActionSkip, "is not verified locally: no local-ci status", "make ci-local RAN="}, + {"failed check", func(pull verify.Pull) verify.Pull { + pull.Checks = append(green(), ciCheck(verify.AlwaysRuns, "206", "COMPLETED", "FAILURE")) + return pull + }, verify.MergeOptions{}, verify.ActionSkip, "has failing checks: Secret scan", "gh run rerun"}, + {"hollow", func(pull verify.Pull) verify.Pull { + pull.Checks = []verify.Check{ciCheck(verify.AlwaysRuns, "205", "COMPLETED", "SKIPPED"), ciCheck(verify.Gate, "205", "COMPLETED", "SUCCESS")} + return pull + }, verify.MergeOptions{}, verify.ActionSkip, "skipped every job", "push a new commit"}, + {"running", func(pull verify.Pull) verify.Pull { + pull.Checks = []verify.Check{ciCheck(verify.AlwaysRuns, "205", "IN_PROGRESS", ""), ciCheck(verify.Gate, "205", "COMPLETED", "SUCCESS")} + return pull + }, verify.MergeOptions{}, verify.ActionSkip, "still running: Secret scan", "run this again"}, + {"awaiting: no gate yet", func(pull verify.Pull) verify.Pull { + pull.Checks = []verify.Check{ciCheck(verify.AlwaysRuns, "205", "COMPLETED", "SUCCESS"), ciCheck("Awaiting make ci-local", "205", "COMPLETED", "SUCCESS")} + return pull + }, verify.MergeOptions{}, verify.ActionSkip, "still running: Local verification", ""}, + {"unknown mergeable", func(pull verify.Pull) verify.Pull { pull.Mergeable = "UNKNOWN"; return pull }, verify.MergeOptions{}, verify.ActionSkip, "still being computed", "in a minute"}, + {"no CI run merges on local-ci", func(pull verify.Pull) verify.Pull { + pull.Checks = []verify.Check{{Typename: "StatusContext", Context: verify.StatusContext, State: "SUCCESS"}} + return pull + }, verify.MergeOptions{}, verify.ActionMerge, "merging on local-ci alone", ""}, + {"no CI run, actions required", func(pull verify.Pull) verify.Pull { pull.Checks = nil; return pull }, verify.MergeOptions{RequireActions: true}, verify.ActionSkip, "no GitHub CI job that ran", "--require-actions"}, + // Billing-blocked Actions fail every job in seconds without starting + // it. Those jobs never ran, so local-ci decides. + {"jobs GitHub never started merge on local-ci", func(pull verify.Pull) verify.Pull { + pull.Checks = []verify.Check{notStarted(verify.AlwaysRuns), notStarted("Local verification claim")} + return pull + }, verify.MergeOptions{}, verify.ActionMerge, "merging on local-ci alone", ""}, + {"jobs never started, actions required", func(pull verify.Pull) verify.Pull { + pull.Checks = []verify.Check{notStarted(verify.AlwaysRuns)} + return pull + }, verify.MergeOptions{RequireActions: true}, verify.ActionSkip, "no GitHub CI job that ran", ""}, + {"a job that ran and failed still refuses beside one never started", func(pull verify.Pull) verify.Pull { + pull.Checks = []verify.Check{notStarted(verify.AlwaysRuns), ciCheck(verify.Gate, "205", "COMPLETED", "FAILURE")} + return pull + }, verify.MergeOptions{}, verify.ActionSkip, "has failing checks: Local verification", ""}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + verdict := verify.Decide(testCase.pull(ready), testCase.options) + said := verdict.Reason + if verdict.Action == verify.ActionMerge { + said = verdict.Note + } + if verdict.Action != testCase.action || !strings.Contains(said, testCase.reason) || !strings.Contains(verdict.Next, testCase.next) { + test.Fatalf("Decide = %+v, want %s %q next %q", verdict, testCase.action, testCase.reason, testCase.next) + } + }) + } +} diff --git a/scripts/cilocal/verify/migrations.go b/scripts/cilocal/verify/migrations.go new file mode 100644 index 0000000..9055b07 --- /dev/null +++ b/scripts/cilocal/verify/migrations.go @@ -0,0 +1,97 @@ +package verify + +import ( + "fmt" + "path" + "regexp" + "sort" + "strconv" + "strings" +) + +// The migration order (AGENTS.md "Migrations"; scripts/check-migration-order.sh +// runs `cilocal migration-order`): a goose migration a checkout adds must be +// numbered above the highest version every deployed base already has in the +// same set. Goose refuses out-of-order migrations, so once a deployed +// database has applied 00007, a later-merged 00005 stops `migrate` (and +// every deploy) there. A set is a directory named migrations holding +// NNNNN_*.sql files (one goose version table each). + +var migrationVersion = regexp.MustCompile(`(^|/)migrations/[0-9]+_[^/]*\.sql$`) + +// IsMigration is whether `file` is a numbered goose migration. +func IsMigration(file string) bool { return migrationVersion.MatchString(file) } + +func versionPrefix(file string) string { + prefix, _, _ := strings.Cut(path.Base(file), "_") + return prefix +} + +type migrationSet struct { + highest, width, nextFree int +} + +// MigrationProblems is one line per misnumbered migration, naming the number +// to renumber it to: `baseFiles` are the migrations on the deployed bases +// (their tips, not the merge base, so a branch a base has moved past fails +// too), `treeFiles` those in the checkout, `label` names the bases. +func MigrationProblems(baseFiles, treeFiles []string, label string) []string { + sets := map[string]*migrationSet{} + setOf := func(file string) *migrationSet { + dir := path.Dir(file) + if sets[dir] == nil { + sets[dir] = &migrationSet{highest: 0, width: 0, nextFree: 0} + } + return sets[dir] + } + onBase := map[string]bool{} + for _, file := range baseFiles { + if !IsMigration(file) { + continue + } + prefix := versionPrefix(file) + version, _ := strconv.Atoi(prefix) + set := setOf(file) + onBase[file], set.width = true, len(prefix) + set.highest = max(set.highest, version) + } + added := []string{} + for _, file := range treeFiles { + if !IsMigration(file) || onBase[file] { + continue + } + added = append(added, file) + } + sort.Strings(added) + for _, file := range added { + prefix := versionPrefix(file) + version, _ := strconv.Atoi(prefix) + set := setOf(file) + if set.width == 0 { + set.width = len(prefix) + } + set.nextFree = max(set.nextFree, set.highest, version) + } + for _, set := range sets { + set.nextFree++ + } + problems := []string{} + firstWith := map[string]string{} + for _, file := range added { + prefix := versionPrefix(file) + version, _ := strconv.Atoi(prefix) + set := setOf(file) + key := fmt.Sprintf("%s/%d", path.Dir(file), version) + switch { + case version <= set.highest: + problems = append(problems, fmt.Sprintf("%s: version %d is not above the highest on %s (%d): renumber to %0*d", file, version, label, set.highest, set.width, set.nextFree)) + case firstWith[key] != "": + problems = append(problems, fmt.Sprintf("%s: version %d is also used by %s: renumber to %0*d", file, version, firstWith[key], set.width, set.nextFree)) + default: + firstWith[key] = file + continue + } + set.nextFree++ + } + return problems +} diff --git a/scripts/cilocal/verify/migrations_test.go b/scripts/cilocal/verify/migrations_test.go new file mode 100644 index 0000000..069076f --- /dev/null +++ b/scripts/cilocal/verify/migrations_test.go @@ -0,0 +1,57 @@ +package verify_test + +import ( + "slices" + "testing" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +func TestMigrationProblemsNameTheNumberToRenumberTo(test *testing.T) { + test.Parallel() + const app, auth = "packages/database/migrations/", "packages/auth/migrations/" + base := []string{app + "00001_init.sql", app + "00007_users.sql", auth + "00003_sessions.sql", "README.md"} + for _, testCase := range []struct { + name string + base []string + tree []string + wants []string + }{ + {"nothing added", base, base, []string{}}, + {"above the highest", base, append(slices.Clone(base), app+"00008_more.sql"), []string{}}, + {"a gap is fine", base, append(slices.Clone(base), app+"00020_more.sql"), []string{}}, + { + "below the highest", base, append(slices.Clone(base), app+"00005_late.sql"), + []string{app + "00005_late.sql: version 5 is not above the highest on origin/main (7): renumber to 00008"}, + }, + { + "equal to the highest", base, append(slices.Clone(base), app+"00007_clash.sql"), + []string{app + "00007_clash.sql: version 7 is not above the highest on origin/main (7): renumber to 00008"}, + }, + { + "two added with one number", base, append(slices.Clone(base), app+"00008_a.sql", app+"00008_b.sql"), + []string{app + "00008_b.sql: version 8 is also used by " + app + "00008_a.sql: renumber to 00009"}, + }, + { + "each misnumbered one gets its own next number", base, append(slices.Clone(base), app+"00002_a.sql", app+"00003_b.sql"), + []string{ + app + "00002_a.sql: version 2 is not above the highest on origin/main (7): renumber to 00008", + app + "00003_b.sql: version 3 is not above the highest on origin/main (7): renumber to 00009", + }, + }, + {"sets are separate", base, append(slices.Clone(base), auth+"00004_next.sql"), []string{}}, + { + "a new set keeps its own width", nil, []string{"apps/x/migrations/001_a.sql", "apps/x/migrations/001_b.sql"}, + []string{"apps/x/migrations/001_b.sql: version 1 is also used by apps/x/migrations/001_a.sql: renumber to 002"}, + }, + {"not a numbered migration", base, append(slices.Clone(base), app+"notes.sql", "docs/migrations/0001_x.md"), []string{}}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + got := verify.MigrationProblems(testCase.base, testCase.tree, "origin/main") + if !slices.Equal(got, testCase.wants) { + test.Errorf("problems = %q, want %q", got, testCase.wants) + } + }) + } +} diff --git a/scripts/cilocal/verify/problems.go b/scripts/cilocal/verify/problems.go new file mode 100644 index 0000000..a061295 --- /dev/null +++ b/scripts/cilocal/verify/problems.go @@ -0,0 +1,160 @@ +package verify + +import ( + "fmt" + "regexp" +) + +// Status is a commit status as GitHub returns it (`commits//statuses` +// lists every one ever posted, newest first). +type Status struct { + Context string `json:"context"` + State string `json:"state"` + Description string `json:"description"` + CreatedAt string `json:"created_at"` +} + +var ( + sectionHead = regexp.MustCompile("(?m)^Head `([0-9a-f]{40})`") + ranLine = regexp.MustCompile(`(?m)^\*\*Ran:\*\* \S`) + bodyDigest = regexp.MustCompile(`body ([0-9a-f]{12})`) +) + +// headOf is the head commit a section names, or "". +func headOf(section string) string { + if match := sectionHead.FindStringSubmatch(section); match != nil { + return match[1] + } + return "" +} + +// FailedBefore is the first failed `local-ci` among a commit's statuses, or +// nil. A head whose author reported a failure is never attested again: a +// failure is resolved by a new commit (the fix, or a rebase on a fix merged +// off main), never by attesting until green. +func FailedBefore(statuses []Status) *Status { + for index := range statuses { + status := statuses[index] + if status.Context == StatusContext && (status.State == "failure" || status.State == "error") { + return &status + } + } + return nil +} + +// FailedHeadRefusal is why make ci-local refuses to attest `head`, or "" +// when no `local-ci` failed there. +func FailedHeadRefusal(head string, statuses []Status) string { + failed := FailedBefore(statuses) + if failed == nil { + return "" + } + return fmt.Sprintf("%s already failed on %s (%s). A failed head is resolved by a new commit: fix it in this PR if the failure is in what it changed, otherwise fix it forward in its own PR off main with a regression test, rebase on it and attest again.", + StatusContext, Short(head), failed.CreatedAt) +} + +// ClaimsHead is whether make ci-local has claimed `head`: a `local-ci` +// status was posted on it (`statuses`, every status ever posted there), or +// the body's section names it. Until then nothing is attested, so ci.yml +// names its checking job "Awaiting make ci-local" and the required check +// `Local verification` stays "Expected" rather than red; once claimed, every +// problem fails it (docs/CI.md, "What runs"). +func ClaimsHead(body, head string, statuses []Status) bool { + for _, status := range statuses { + if status.Context == StatusContext { + return true + } + } + section, _ := SectionOf(body) + return headOf(section) == head +} + +// BodyProblems is why the PR body's section does not attest `head`: +// missing, for another head, or no statement of what ran. Empty when it holds. +func BodyProblems(body, head string) []string { + section, found := SectionOf(body) + if !found { + return []string{"the PR body has no Local verification section"} + } + problems := []string{} + switch named := headOf(section); { + case named == "": + problems = append(problems, "the Local verification section names no head") + case named != head: + problems = append(problems, fmt.Sprintf("the Local verification section is for %s, not the PR head %s", Short(named), Short(head))) + } + if !ranLine.MatchString(section) { + problems = append(problems, "the Local verification section does not say what ran") + } + return problems +} + +// Verification is what GitHub holds for a pull request's head: its body, +// the `local-ci` status in the combined status (nil when there is none), +// every status ever posted on the head, and the fork the head lives in. +type Verification struct { + Body string + Head string + Status *Status + Statuses []Status + // Fork is the repository of a head outside this one (owner/name, or + // ForkGone), "" for a branch of this repository. + Fork string +} + +// ForkGone stands for the fork of a pull request whose fork was deleted. +const ForkGone = "a deleted fork" + +// ForkOf is the fork a head lives in: "" when `headRepo` (owner/name) is +// `repo` itself, ForkGone when GitHub names no repository. +func ForkOf(repo, headRepo string) string { + switch headRepo { + case repo: + return "" + case "": + return ForkGone + } + return headRepo +} + +// Claimed is whether the Local verification job checks this head under its +// required name (ClaimsHead). A fork's head always is, so it fails red +// rather than waiting as "Expected". +func Claimed(verification Verification) bool { + return verification.Fork != "" || ClaimsHead(verification.Body, verification.Head, verification.Statuses) +} + +// LocalVerificationProblems is every reason a head is not attested; empty +// when it is: the head is a branch of this repository, the section names the +// head and says what ran, `local-ci` is success there with the section's +// digest, and no `local-ci` ever failed on it. The Local verification job, +// the merge script and the staging gate all decide with it. A head behind +// main is not a problem here. +// +// A fork's head is never attested: make ci-local attests only a pushed +// branch of this repository, and a fork's author controls its body, so a +// section and status copied from another pull request must not pass. +func LocalVerificationProblems(verification Verification) []string { + head := verification.Head + problems := BodyProblems(verification.Body, head) + if verification.Fork != "" { + problems = append(problems, fmt.Sprintf("the head is in %s, not this repository: make ci-local attests only branches pushed here (push the commits to a branch of this repository and open the PR from it)", verification.Fork)) + } + if failed := FailedBefore(verification.Statuses); failed != nil { + problems = append(problems, fmt.Sprintf("%s failed on %s at %s: a failed head is resolved by a new commit (fix it here, or fix it forward off main and rebase), never by attesting again", + StatusContext, Short(head), failed.CreatedAt)) + } + section, found := SectionOf(verification.Body) + switch status := verification.Status; { + case status == nil: + problems = append(problems, fmt.Sprintf("no %s status on %s", StatusContext, Short(head))) + case status.State != "success": + problems = append(problems, fmt.Sprintf("%s is %s on %s", StatusContext, status.State, Short(head))) + case found: + match := bodyDigest.FindStringSubmatch(status.Description) + if match == nil || match[1] != SectionDigest(section) { + problems = append(problems, fmt.Sprintf("the Local verification section is not the one %s attested (edited by hand?)", StatusContext)) + } + } + return problems +} diff --git a/scripts/cilocal/verify/problems_test.go b/scripts/cilocal/verify/problems_test.go new file mode 100644 index 0000000..a326203 --- /dev/null +++ b/scripts/cilocal/verify/problems_test.go @@ -0,0 +1,161 @@ +package verify_test + +import ( + "strings" + "testing" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +func TestBodyProblems(test *testing.T) { + test.Parallel() + section := verify.RenderSection(passingReport()) + noHead := strings.Replace(section, "Head `", "Head: `", 1) + noRan := strings.Replace(section, "**Ran:** ", "Ran: ", 1) + emptyRan := strings.Replace(section, "**Ran:** "+passingReport().Ran, "**Ran:** ", 1) + for _, testCase := range []struct { + name string + body string + head string + want string + }{ + {"no section", "just a body", headSHA, "has no Local verification section"}, + {"wrong head", verify.WithSection("", section), otherSHA, "is for aaaaaaaaaaaa, not the PR head bbbbbbbbbbbb"}, + {"no head", verify.WithSection("", noHead), headSHA, "names no head"}, + {"no statement", verify.WithSection("", noRan), headSHA, "does not say what ran"}, + {"empty statement", verify.WithSection("", emptyRan), headSHA, "does not say what ran"}, + {"the template's placeholder", verify.WithSection("", "Not attested yet."), headSHA, "names no head"}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + requireMention(test, verify.BodyProblems(testCase.body, testCase.head), testCase.want) + }) + } +} + +func TestFailedHeadRefusal(test *testing.T) { + test.Parallel() + for _, testCase := range []struct { + name string + statuses []verify.Status + refuses bool + }{ + {"no statuses", nil, false}, + {"success", []verify.Status{success("ok")}, false}, + {"pending", []verify.Status{{Context: verify.StatusContext, State: "pending"}}, false}, + {"other context failed", []verify.Status{{Context: "ci/other", State: "failure"}, {Context: "ci/other", State: "error"}}, false}, + {"failure", []verify.Status{success("ok"), {Context: verify.StatusContext, State: "failure", CreatedAt: "2026-10-30T09:00:00Z"}}, true}, + {"error", []verify.Status{{Context: verify.StatusContext, State: "error", CreatedAt: "2026-10-30T09:00:00Z"}}, true}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + failed := verify.FailedBefore(testCase.statuses) + refusal := verify.FailedHeadRefusal(headSHA, testCase.statuses) + if (failed != nil) != testCase.refuses || (refusal != "") != testCase.refuses { + test.Fatalf("FailedBefore = %+v, refusal = %q; want refuses %v", failed, refusal, testCase.refuses) + } + if testCase.refuses && (!strings.Contains(refusal, "aaaaaaaaaaaa") || !strings.Contains(refusal, failed.CreatedAt) || !strings.Contains(refusal, "attest again")) { + test.Fatalf("refusal = %q", refusal) + } + }) + } +} + +func TestClaimsHead(test *testing.T) { + test.Parallel() + attested := verified() + template := verify.WithSection("", "Not attested yet: run what the change needs, then make ci-local.") + for _, testCase := range []struct { + name string + body string + statuses []verify.Status + want bool + }{ + {"the template, no status", template, nil, false}, + {"a section for an older head", verify.WithSection("", strings.ReplaceAll(verify.RenderSection(passingReport()), headSHA, otherSHA)), nil, false}, + {"another context only", template, []verify.Status{{Context: "ci/other", State: "success"}}, false}, + {"a local-ci status on the head", template, []verify.Status{{Context: verify.StatusContext, State: "failure"}}, true}, + {"the section names the head", attested.Body, nil, true}, + {"no section", "", nil, false}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + if got := verify.ClaimsHead(testCase.body, headSHA, testCase.statuses); got != testCase.want { + test.Fatalf("ClaimsHead = %v, want %v", got, testCase.want) + } + }) + } +} + +func TestLocalVerificationProblems(test *testing.T) { + test.Parallel() + requireNone(test, verify.LocalVerificationProblems(verified())) + edit := func(change func(*verify.Verification)) verify.Verification { + verification := verified() + change(&verification) + return verification + } + for _, testCase := range []struct { + name string + verification verify.Verification + want string + }{ + {"section edited after", edit(func(verification *verify.Verification) { + verification.Body = strings.Replace(verification.Body, "make lint", "make lint (edited)", 1) + }), "is not the one local-ci attested"}, + {"status without body digest", edit(func(verification *verify.Verification) { + status := success("passed") + verification.Status = &status + }), "is not the one local-ci attested"}, + {"no status", edit(func(verification *verify.Verification) { verification.Status = nil }), "no local-ci status on aaaaaaaaaaaa"}, + {"status failed", edit(func(verification *verify.Verification) { + verification.Status = &verify.Status{Context: verify.StatusContext, State: "failure"} + }), "local-ci is failure on aaaaaaaaaaaa"}, + {"failure in history", edit(func(verification *verify.Verification) { + verification.Statuses = append(verification.Statuses, verify.Status{Context: verify.StatusContext, State: "failure", CreatedAt: "earlier"}) + }), "local-ci failed on aaaaaaaaaaaa at earlier"}, + {"other head", edit(func(verification *verify.Verification) { verification.Head = otherSHA }), "not the PR head bbbbbbbbbbbb"}, + {"no section", edit(func(verification *verify.Verification) { verification.Body = "## Problem" }), "has no Local verification section"}, + // Randy's review of #134: a fork's author controls its body, so a + // section and status copied from an attested head must not pass. + {"fork", edit(func(verification *verify.Verification) { verification.Fork = "someone/go-fast-stack" }), "the head is in someone/go-fast-stack, not this repository"}, + {"deleted fork", edit(func(verification *verify.Verification) { verification.Fork = verify.ForkGone }), "the head is in a deleted fork"}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + requireMention(test, verify.LocalVerificationProblems(testCase.verification), testCase.want) + }) + } +} + +func TestForkOf(test *testing.T) { + test.Parallel() + for _, testCase := range []struct{ headRepo, want string }{ + {"acme/app", ""}, + {"someone/app", "someone/app"}, + {"", verify.ForkGone}, + } { + if got := verify.ForkOf("acme/app", testCase.headRepo); got != testCase.want { + test.Errorf("ForkOf(acme/app, %q) = %q, want %q", testCase.headRepo, got, testCase.want) + } + } +} + +// A fork's head is checked under the required name, so it fails red instead +// of waiting as "Expected"; any other head is claimed as ClaimsHead says. +func TestClaimed(test *testing.T) { + test.Parallel() + fork := verify.Verification{Body: "", Head: headSHA, Status: nil, Statuses: nil, Fork: "someone/app"} + if !verify.Claimed(fork) { + test.Error("a fork's unattested head is not claimed") + } + if problems := verify.LocalVerificationProblems(fork); !containsText(problems, "not this repository") { + test.Errorf("a fork's head passes: %q", problems) + } + if verify.Claimed(verify.Verification{Body: "", Head: headSHA, Status: nil, Statuses: nil, Fork: ""}) { + test.Error("an unattested branch head is claimed") + } + if !verify.Claimed(verified()) { + test.Error("an attested head is not claimed") + } +} diff --git a/scripts/cilocal/verify/section.go b/scripts/cilocal/verify/section.go new file mode 100644 index 0000000..d1844a0 --- /dev/null +++ b/scripts/cilocal/verify/section.go @@ -0,0 +1,147 @@ +package verify + +import ( + "crypto/sha256" + "encoding/hex" + "fmt" + "strings" +) + +// The PR body's "Local verification" section: rendered by make ci-local from +// the agent's statement of what it ran, and checked by the Local +// verification job in ci.yml (cilocal check), the merge script (cilocal +// merge) and the staging gate (cilocal attest). One module for both sides of +// the rule. make ci-local runs nothing: it tells the PR that the agent ran +// what CI would have checked, and says what that was. + +// StatusContext is the commit status make ci-local posts on the head it attests. +const StatusContext = "local-ci" + +// The section's markers in the PR body. +const ( + SectionStart = "" + SectionEnd = "" +) + +// NextStep is the one fix for every problem the checks report. +const NextStep = "run what the change needs, then `make ci-local RAN=\"\"` on the pushed head (a new push or rebase needs a new attestation); it posts local-ci and fills the PR body" + +// digestLength is how many hex characters of a sha256 the status carries. +const digestLength = 12 + +// shortLength is how many characters of a commit sha messages show. +const shortLength = 12 + +// descriptionLimit is the most GitHub keeps of a status description, in characters. +const descriptionLimit = 140 + +// Results of an attestation. +const ( + Passed = "passed" + Failed = "failed" +) + +// Short is a commit sha as messages show it. +func Short(sha string) string { + if len(sha) > shortLength { + return sha[:shortLength] + } + return sha +} + +// StatementOf is a statement of what ran as one line (RAN may span lines, +// joined with "; "), or why it cannot be one: empty, or holding an HTML +// comment that could rewrite the section around it. +func StatementOf(text string) (string, string) { + parts := []string{} + for _, part := range strings.Split(text, "\n") { + if part = strings.TrimSpace(part); part != "" { + parts = append(parts, part) + } + } + line := strings.Join(parts, "; ") + switch { + case line == "": + return "", `say what you ran: make ci-local RAN=""` + case strings.Contains(line, "", "", "may not hold an HTML comment"}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + line, problem := verify.StatementOf(testCase.text) + if line != testCase.line || !strings.Contains(problem, testCase.problem) || (testCase.problem == "") != (problem == "") { + test.Fatalf("StatementOf(%q) = %q, %q; want %q, %q", testCase.text, line, problem, testCase.line, testCase.problem) + } + }) + } +} + +func TestRenderSection(test *testing.T) { + test.Parallel() + want := strings.Join([]string{ + "> Filled by `make ci-local`, 2026-10-31T10:00:00Z. Never edit this section by hand.", + "", + "Head `" + headSHA + "` · tree `dddddddddddd` · passed", + "Attested by `JJ Eaton` on `laptop`", + "", + "**Ran:** make lint test TEST_PKGS=./scripts/cilocal/verify", + }, "\n") + if got := verify.RenderSection(passingReport()); got != want { + test.Fatalf("RenderSection =\n%s\nwant\n%s", got, want) + } + failed := passingReport() + failed.Result, failed.Failed = verify.Failed, "make test: TestX" + got := verify.RenderSection(failed) + if !strings.Contains(got, "· failed\n") || !strings.HasSuffix(got, "\n**Failed:** make test: TestX") { + test.Fatalf("a failed section = %q", got) + } +} + +func TestSectionRoundTripAttestsTheHead(test *testing.T) { + test.Parallel() + section := verify.RenderSection(passingReport()) + for _, testCase := range []struct { + name string + body string + }{ + {"empty body", ""}, + {"template body", "## Problem\n\nWhat changed.\n\n"}, + {"existing section", "## Problem\n\n## Local verification\n\n" + verify.SectionStart + "\nstale\n" + verify.SectionEnd + "\n\n## Notes\nkept"}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + body := verify.WithSection(testCase.body, section) + got, found := verify.SectionOf(body) + if !found || got != strings.TrimSpace(section) { + test.Fatalf("SectionOf = %q, %v; want the rendered section", got, found) + } + requireNone(test, verify.BodyProblems(body, headSHA)) + if strings.Contains(body, "stale") { + test.Fatal("WithSection kept the old section") + } + if strings.Count(body, verify.SectionStart) != 1 { + test.Fatalf("body has %d sections", strings.Count(body, verify.SectionStart)) + } + if strings.Contains(testCase.body, "## Notes") && !strings.HasSuffix(body, "## Notes\nkept") { + test.Fatalf("WithSection dropped the text after the section: %q", body) + } + }) + } +} + +// A body whose section lost its end marker (an edit that cut it) gets a new +// section appended, and SectionOf must read that new section back: make +// ci-local posts the digest of what it rendered, and the check digests what +// SectionOf reads. +func TestWithSectionRoundTripsABodyWithADanglingStartMarker(test *testing.T) { + test.Parallel() + section := verify.RenderSection(passingReport()) + body := verify.WithSection("## Problem\n\n"+verify.SectionStart+"\nstale, end marker deleted", section) + got, found := verify.SectionOf(body) + if !found || got != strings.TrimSpace(section) { + test.Fatalf("SectionOf(WithSection(body, section)) = %q, want the rendered section", got) + } +} + +func TestSectionOfWithoutMarkers(test *testing.T) { + test.Parallel() + for _, body := range []string{"", "no markers", verify.SectionStart + " no end", verify.SectionEnd + " end only"} { + test.Run(body, func(test *testing.T) { + test.Parallel() + if section, found := verify.SectionOf(body); found || section != "" { + test.Fatalf("SectionOf(%q) = %q, %v", body, section, found) + } + }) + } +} + +func TestStatusDescription(test *testing.T) { + test.Parallel() + report := passingReport() + section := verify.RenderSection(report) + want := "passed · body " + verify.SectionDigest(section) + " · by JJ Eaton on laptop" + if got := verify.StatusDescription(report, section); got != want { + test.Fatalf("StatusDescription = %q, want %q", got, want) + } + if verify.SectionDigest(section) != verify.SectionDigest("\n"+section+"\n ") || len(verify.SectionDigest(section)) != 12 { + test.Fatal("SectionDigest is not a 12-character digest of the trimmed section") + } + report.Hostname = strings.Repeat("é", 200) + long := verify.StatusDescription(report, section) + if utf8.RuneCountInString(long) != 140 || !utf8.ValidString(long) || !strings.HasPrefix(long, "passed · body ") { + test.Fatalf("a long description = %q (%d characters)", long, utf8.RuneCountInString(long)) + } +} diff --git a/scripts/cilocal/verify/staging.go b/scripts/cilocal/verify/staging.go new file mode 100644 index 0000000..da174e2 --- /dev/null +++ b/scripts/cilocal/verify/staging.go @@ -0,0 +1,143 @@ +package verify + +import ( + "encoding/json" + "fmt" + "strings" +) + +// After a merge (make pr-merge ARGS="--check-deploy "): did staging +// deploy that commit, and is it ready? One read of the commit's "Deploy +// staging" runs and of the staging web's /healthz. A follow-up, never a +// gate: the next PR merges meanwhile. + +// DeployWorkflow is deploy-staging.yml's name. +const DeployWorkflow = "Deploy staging" + +// DeployRun is one run of Deploy staging for a commit (gh run list). +type DeployRun struct { + Name string `json:"workflowName"` + Status string `json:"status"` + Conclusion string `json:"conclusion"` + URL string `json:"url"` + // DryRun is a run whose deploy job was skipped and whose dry-run job + // ran (RAILWAY_DEPLOY is not true): it deployed nothing. + DryRun bool `json:"-"` +} + +// The jobs of deploy-staging.yml that DryRunJobs reads. +const ( + deployJob = "Deploy main to Railway staging" + dryRunJob = "Dry run (RAILWAY_DEPLOY is not true)" +) + +// Job is one job of a workflow run (gh run view --json jobs). +type Job struct { + Name string `json:"name"` + Conclusion string `json:"conclusion"` +} + +// DryRunJobs is whether a Deploy staging run's jobs are a dry run: the +// dry-run job ran and the deploy job did not. +func DryRunJobs(jobs []Job) bool { + dry, deployed := false, false + for _, job := range jobs { + switch job.Name { + case dryRunJob: + dry = job.Conclusion == "success" + case deployJob: + deployed = job.Conclusion != "skipped" && job.Conclusion != "" + } + } + return dry && !deployed +} + +// Readiness is the staging web's /healthz answer: unset URL, a transport +// error, or the status code and its `data.status`. +type Readiness struct { + URL string + Code int + Status string + Err string +} + +// readyCode is the /healthz status code of a ready server. +const readyCode = 200 + +// ReadinessOf reads a /healthz answer: `{"data":{"status":"ok"}}` with 200 +// is ready. +func ReadinessOf(url string, code int, body []byte) Readiness { + var answer struct { + Data struct { + Status string `json:"status"` + } `json:"data"` + } + _ = json.Unmarshal(body, &answer) // any other body is simply not ready + return Readiness{URL: url, Code: code, Status: answer.Data.Status, Err: ""} +} + +// Ready is whether staging answered ready. +func (ready Readiness) Ready() bool { return ready.Code == readyCode && ready.Status == "ok" } + +func (ready Readiness) String() string { + switch { + case ready.URL == "": + return "readiness unknown: STAGING_WEB_URL is unset (the repository variable, or the env)" + case ready.Err != "": + return fmt.Sprintf("%s/healthz unreachable: %s", ready.URL, ready.Err) + case ready.Ready(): + return fmt.Sprintf("%s/healthz 200 ok", ready.URL) + } + return fmt.Sprintf("%s/healthz answered %d with data.status %q", ready.URL, ready.Code, ready.Status) +} + +// StagingVerdict is the verdict on one main commit. +type StagingVerdict struct { + Settled bool // nothing is still running + OK bool + Summary string +} + +// CheckStagingStep is the command that checks main's head instead. +const CheckStagingStep = `make pr-merge ARGS=--check-staging` + +// Staging is the staging verdict for main commit `head` from its Deploy +// staging runs and the readiness answer. A run cancelled by a newer main +// commit's deploy is superseded: that deploy carries this commit, so main's +// head is the one to check. +func Staging(head string, deploys []DeployRun, ready Readiness) StagingVerdict { + at := "main " + Short(head) + superseded, running, failed, dry := []string{}, 0, []string{}, 0 + for _, run := range deploys { + switch { + case run.Status != "completed": + running++ + case run.Conclusion == "cancelled": + superseded = append(superseded, run.URL) + case run.Conclusion != "success" && run.Conclusion != "skipped": + failed = append(failed, fmt.Sprintf("%s %s %s", run.Name, run.Conclusion, run.URL)) + case run.DryRun: + dry++ + } + } + switch { + case len(superseded) > 0: + return StagingVerdict{Settled: true, OK: false, + Summary: fmt.Sprintf("%s: Deploy staging cancelled by a newer main deploy (%s); check that one: %s", at, strings.Join(superseded, ", "), CheckStagingStep)} + case running > 0 || len(deploys) == 0: + state := "not started" + if running > 0 { + state = "running" + } + return StagingVerdict{Settled: false, OK: false, Summary: fmt.Sprintf("%s: deploy %s", at, state)} + case len(failed) > 0: + return StagingVerdict{Settled: true, OK: false, + Summary: fmt.Sprintf("%s: %s. A failed staging deploy is the top priority: fix it forward before merging anything else", at, strings.Join(failed, "; "))} + case dry > 0: + return StagingVerdict{Settled: true, OK: true, + Summary: at + ": dry run: nothing deployed (set the RAILWAY_DEPLOY repository variable to true to deploy, docs/RELEASES.md)"} + case !ready.Ready(): + return StagingVerdict{Settled: true, OK: false, Summary: fmt.Sprintf("%s: deployed; %s (top priority)", at, ready)} + } + return StagingVerdict{Settled: true, OK: true, Summary: fmt.Sprintf("%s: deployed; %s", at, ready)} +} diff --git a/scripts/cilocal/verify/staging_test.go b/scripts/cilocal/verify/staging_test.go new file mode 100644 index 0000000..0f4b015 --- /dev/null +++ b/scripts/cilocal/verify/staging_test.go @@ -0,0 +1,95 @@ +package verify_test + +import ( + "strings" + "testing" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +func TestDryRunJobs(test *testing.T) { + test.Parallel() + plan := verify.Job{Name: "Plan the staging services", Conclusion: "success"} + deploy := func(conclusion string) verify.Job { + return verify.Job{Name: "Deploy main to Railway staging", Conclusion: conclusion} + } + dry := func(conclusion string) verify.Job { + return verify.Job{Name: "Dry run (RAILWAY_DEPLOY is not true)", Conclusion: conclusion} + } + for _, testCase := range []struct { + name string + jobs []verify.Job + want bool + }{ + {"dry run", []verify.Job{plan, deploy("skipped"), dry("success")}, true}, + {"dry run, deploy job not listed", []verify.Job{plan, dry("success")}, true}, + {"deployed", []verify.Job{plan, deploy("success"), dry("skipped")}, false}, + {"dry run failed", []verify.Job{plan, deploy("skipped"), dry("failure")}, false}, + {"plan refused", []verify.Job{{Name: "Plan the staging services", Conclusion: "failure"}, deploy("skipped"), dry("skipped")}, false}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + if got := verify.DryRunJobs(testCase.jobs); got != testCase.want { + test.Fatalf("DryRunJobs = %v, want %v", got, testCase.want) + } + }) + } +} + +func TestReadiness(test *testing.T) { + test.Parallel() + const url = "https://staging.example.com" + for _, testCase := range []struct { + name string + ready verify.Readiness + ok bool + says string + }{ + {"ok", verify.ReadinessOf(url, 200, []byte(`{"data":{"status":"ok"}}`)), true, url + "/healthz 200 ok"}, + {"503", verify.ReadinessOf(url, 503, []byte(`{"error":{"message":"database unavailable"}}`)), false, "answered 503"}, + {"200 but not ok", verify.ReadinessOf(url, 200, []byte(`not json`)), false, `answered 200 with data.status ""`}, + {"unreachable", verify.Readiness{URL: url, Err: "timeout"}, false, "unreachable: timeout"}, + {"unset", verify.Readiness{}, false, "STAGING_WEB_URL is unset"}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + if testCase.ready.Ready() != testCase.ok || !strings.Contains(testCase.ready.String(), testCase.says) { + test.Fatalf("Readiness %+v: Ready %v, %q; want %v, %q", testCase.ready, testCase.ready.Ready(), testCase.ready, testCase.ok, testCase.says) + } + }) + } +} + +func TestStaging(test *testing.T) { + test.Parallel() + ready := verify.ReadinessOf("https://staging.example.com", 200, []byte(`{"data":{"status":"ok"}}`)) + down := verify.ReadinessOf("https://staging.example.com", 503, nil) + run := func(status, conclusion string) verify.DeployRun { + return verify.DeployRun{Name: verify.DeployWorkflow, Status: status, Conclusion: conclusion, URL: "https://github.com/o/r/actions/runs/7"} + } + dry := run("completed", "success") + dry.DryRun = true + for _, testCase := range []struct { + name string + deploys []verify.DeployRun + ready verify.Readiness + settled, ok bool + says string + }{ + {"deployed and ready", []verify.DeployRun{run("completed", "success")}, ready, true, true, "main aaaaaaaaaaaa: deployed; https://staging.example.com/healthz 200 ok"}, + {"deployed, not ready", []verify.DeployRun{run("completed", "success")}, down, true, false, "answered 503 with data.status \"\" (top priority)"}, + {"not started", nil, ready, false, false, "deploy not started"}, + {"running", []verify.DeployRun{run("in_progress", "")}, ready, false, false, "deploy running"}, + {"cancelled by a newer deploy", []verify.DeployRun{run("completed", "cancelled")}, ready, true, false, "make pr-merge ARGS=--check-staging"}, + {"failed", []verify.DeployRun{run("completed", "failure")}, ready, true, false, "Deploy staging failure https://github.com/o/r/actions/runs/7. A failed staging deploy is the top priority"}, + {"dry run", []verify.DeployRun{dry}, verify.Readiness{}, true, true, "dry run: nothing deployed"}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + verdict := verify.Staging(headSHA, testCase.deploys, testCase.ready) + if verdict.Settled != testCase.settled || verdict.OK != testCase.ok || !strings.Contains(verdict.Summary, testCase.says) { + test.Fatalf("Staging = %+v, want settled %v ok %v saying %q", verdict, testCase.settled, testCase.ok, testCase.says) + } + }) + } +} diff --git a/scripts/cilocal/verify/template_test.go b/scripts/cilocal/verify/template_test.go new file mode 100644 index 0000000..0f839b6 --- /dev/null +++ b/scripts/cilocal/verify/template_test.go @@ -0,0 +1,34 @@ +package verify_test + +import ( + "os" + "path/filepath" + "strings" + "testing" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +// The PR template holds the section make ci-local fills, between its markers, +// with a placeholder that claims no head: a new PR's Local verification job +// then runs as "Awaiting make ci-local" (ClaimsHead) and its body problems +// say how to attest. +func TestTemplateHoldsAnUnattestedSection(test *testing.T) { + test.Parallel() + template, err := os.ReadFile(filepath.Join("..", "..", "..", ".github", "pull_request_template.md")) + if err != nil { + test.Fatal(err) + } + body := string(template) + section, found := verify.SectionOf(body) + if !found || strings.Count(body, verify.SectionStart) != 1 || strings.Count(body, verify.SectionEnd) != 1 { + test.Fatal("the PR template has no single Local verification section between its markers") + } + if !strings.Contains(section, `make ci-local RAN=`) { + test.Errorf("the template's placeholder %q does not say how to attest", section) + } + if verify.ClaimsHead(body, headSHA, nil) { + test.Error("the template's section claims a head") + } + requireMention(test, verify.BodyProblems(body, headSHA), "names no head") +} diff --git a/scripts/cilocal/verify/tree.go b/scripts/cilocal/verify/tree.go new file mode 100644 index 0000000..e6296e2 --- /dev/null +++ b/scripts/cilocal/verify/tree.go @@ -0,0 +1,122 @@ +package verify + +import ( + "fmt" + pathpkg "path" + "path/filepath" + "slices" + "strings" +) + +// The Secret scan job reads a pull request's head as data: the files of its +// tree, from git's objects, never a checkout of it (docs/CI.md, "Secret +// scan"). The code that scans, and its .gitleaks.toml, are the base's. + +// TreeEntry is one entry of `git ls-tree -r -z `. +type TreeEntry struct { + Mode, Blob, Path string +} + +// ParseTree reads `git ls-tree -r -z` output. +func ParseTree(output string) []TreeEntry { + entries := []TreeEntry{} + for line := range strings.SplitSeq(output, "\x00") { + meta, file, found := strings.Cut(line, "\t") + fields := strings.Fields(meta) + const lsFields = 3 // mode, type, object + if !found || len(fields) != lsFields { + continue + } + entries = append(entries, TreeEntry{Mode: fields[0], Blob: fields[2], Path: file}) + } + return entries +} + +// regularMode is the mode prefix of a file with content (100644, 100755); +// symlinks (120000) and submodules (160000) have none of their own. +const regularMode = "100" + +// ScanTargets is the files of a tree the secret scan copies out: the regular +// files among `entries` named in `wanted`, or all of them when `wanted` is +// nil. A path that would leave the scan's directory (absolute, or with a +// `..` element) is a problem, not a file to skip: git never checks one out, +// so only a crafted tree holds one, and the scan fails closed on it. +func ScanTargets(entries []TreeEntry, wanted []string) ([]TreeEntry, string) { + named := map[string]bool{} + for _, file := range wanted { + named[file] = true + } + targets := []TreeEntry{} + for _, entry := range entries { + if !filepath.IsLocal(filepath.FromSlash(entry.Path)) { + return nil, fmt.Sprintf("the head's tree has the path %q, which leaves the scan's directory", entry.Path) + } + if strings.HasPrefix(entry.Mode, regularMode) && (wanted == nil || named[entry.Path]) { + targets = append(targets, entry) + } + } + return targets, "" +} + +// ScannerPolicy are the files gitleaks reads as policy rather than content: +// its config, and the ignore list it loads from the root of the directory it +// scans whatever --gitleaks-ignore-path says (gitleaks 8.30.1). The scan +// takes them from the trusted side only, written outside the scanned +// directory, never from the change it scans. +var ScannerPolicy = []string{".gitleaks.toml", ".gitleaksignore"} + +// scannedSuffix marks a policy-named file copied in as content. +const scannedSuffix = ".scanned" + +// ScanCopyPath is where the scan copies a file of the change: a file named +// like scanner policy, at any depth, gets scannedSuffix, so gitleaks scans +// it as content and never loads it as policy. +func ScanCopyPath(path string) string { + if slices.Contains(ScannerPolicy, pathpkg.Base(path)) { + return path + scannedSuffix + } + return path +} + +// PolicyChanges is the scanner policy files (at the tree's root, where +// gitleaks looks) that differ between the trusted tree and the scanned one: +// added, removed or changed. +func PolicyChanges(trusted, scanned []TreeEntry) []string { + blobs := func(entries []TreeEntry) map[string]string { + found := map[string]string{} + for _, entry := range entries { + if slices.Contains(ScannerPolicy, entry.Path) { + found[entry.Path] = entry.Blob + } + } + return found + } + before, after := blobs(trusted), blobs(scanned) + changed := []string{} + for _, name := range ScannerPolicy { + if before[name] != after[name] { + changed = append(changed, name) + } + } + return changed +} + +// PolicyProblem is why a scan of a change that edits scanner policy fails +// even when it finds nothing: the scan ran with the trusted policy, so the +// new one is unchecked until someone reviews it. +func PolicyProblem(changed []string) string { + if len(changed) == 0 { + return "" + } + return fmt.Sprintf("the change edits the secret scan's policy (%s), which the scan never takes from the change it scans: it ran with the trusted base's. A maintainer reviews the policy change and merges it on its own (docs/CI.md, \"Secret scan\")", + strings.Join(changed, ", ")) +} + +// GitleaksArgs is the gitleaks command line for a scan of the current +// directory: the trusted `config` and `ignore` file (outside the scanned +// directory), and inline `gitleaks:allow` comments ignored, since they are +// content of the change too. +func GitleaksArgs(config, ignore string) []string { + return []string{"dir", ".", "--config", config, "--gitleaks-ignore-path", ignore, "--ignore-gitleaks-allow", + "--redact", "--no-banner", "--no-color", "--exit-code", "1", "--verbose"} +} diff --git a/scripts/cilocal/verify/tree_test.go b/scripts/cilocal/verify/tree_test.go new file mode 100644 index 0000000..96dcf8e --- /dev/null +++ b/scripts/cilocal/verify/tree_test.go @@ -0,0 +1,141 @@ +package verify_test + +import ( + "slices" + "strings" + "testing" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +func TestParseTree(test *testing.T) { + test.Parallel() + listing := "100644 blob " + headSHA + "\tREADME.md\x00" + + "100755 blob " + otherSHA + "\tscripts/a b.sh\x00" + + "160000 commit " + strings.Repeat("c", 40) + "\tvendor/sub\x00" + + "not a tree line\x00" + want := []verify.TreeEntry{ + {Mode: "100644", Blob: headSHA, Path: "README.md"}, + {Mode: "100755", Blob: otherSHA, Path: "scripts/a b.sh"}, + {Mode: "160000", Blob: strings.Repeat("c", 40), Path: "vendor/sub"}, + } + if got := verify.ParseTree(listing); !slices.Equal(got, want) { + test.Fatalf("ParseTree = %+v, want %+v", got, want) + } +} + +// The Secret scan job reads a pull request's head as data: the files GitHub +// lists, or every one, of the head's tree. A fork rewriting .gitleaks.toml +// or scripts/cilocal changes only files that are scanned, never the scanner. +func TestScanTargets(test *testing.T) { + test.Parallel() + entries := []verify.TreeEntry{ + {Mode: "100644", Blob: "b1", Path: ".gitleaks.toml"}, + {Mode: "100644", Blob: "b2", Path: "scripts/cilocal/bypass.go"}, + {Mode: "100755", Blob: "b3", Path: "scripts/run.sh"}, + {Mode: "120000", Blob: "b4", Path: "link"}, + {Mode: "160000", Blob: "b5", Path: "vendor/sub"}, + {Mode: "100644", Blob: "b6", Path: "docs/old.md"}, + } + paths := func(targets []verify.TreeEntry) []string { + names := []string{} + for _, target := range targets { + names = append(names, target.Path) + } + return names + } + every, problem := verify.ScanTargets(entries, nil) + if want := []string{".gitleaks.toml", "scripts/cilocal/bypass.go", "scripts/run.sh", "docs/old.md"}; problem != "" || !slices.Equal(paths(every), want) { + test.Fatalf("ScanTargets(every) = %q, %q; want %q", paths(every), problem, want) + } + listed, problem := verify.ScanTargets(entries, []string{".gitleaks.toml", "scripts/cilocal/bypass.go", "link", "deleted.go"}) + if want := []string{".gitleaks.toml", "scripts/cilocal/bypass.go"}; problem != "" || !slices.Equal(paths(listed), want) { + test.Fatalf("ScanTargets(listed) = %q, %q; want %q", paths(listed), problem, want) + } + for _, path := range []string{"../outside", "/etc/passwd", "a/../../b"} { + crafted := append(slices.Clone(entries), verify.TreeEntry{Mode: "100644", Blob: "b7", Path: path}) + targets, problem := verify.ScanTargets(crafted, nil) + if targets != nil || !strings.Contains(problem, "leaves the scan's directory") { + test.Errorf("ScanTargets with %q = %q, %q; want it to fail closed", path, paths(targets), problem) + } + } +} + +// Randy's re-review of #134 (2f1cca7): with the base's .gitleaks.toml, a +// synthetic credential failed the scan, and a child commit adding only +// .gitleaksignore with that finding's fingerprint passed it, because +// gitleaks loads .gitleaksignore from the root of the directory it scans. +// Both heads must fail: the first on the finding, the second on the finding +// (its .gitleaksignore is copied in as content, never as policy, and the +// trusted ignore file sits outside the scanned directory) and on the policy +// change it makes. +func TestHeadPolicyIsNeverScannerPolicy(test *testing.T) { + test.Parallel() + base := []verify.TreeEntry{{Mode: "100644", Blob: "cfg", Path: ".gitleaks.toml"}, {Mode: "100644", Blob: "go", Path: "main.go"}} + credential := append(slices.Clone(base), verify.TreeEntry{Mode: "100644", Blob: "leak", Path: "credential.txt"}) + ignored := append(slices.Clone(credential), verify.TreeEntry{Mode: "100644", Blob: "fingerprint", Path: ".gitleaksignore"}) + + requireNoPolicyChange := func(head []verify.TreeEntry) { + test.Helper() + if changed := verify.PolicyChanges(base, head); len(changed) != 0 || verify.PolicyProblem(changed) != "" { + test.Fatalf("PolicyChanges = %q, want none", changed) + } + } + requireNoPolicyChange(credential) + + changed := verify.PolicyChanges(base, ignored) + if !slices.Equal(changed, []string{".gitleaksignore"}) { + test.Fatalf("PolicyChanges = %q, want [.gitleaksignore]", changed) + } + if problem := verify.PolicyProblem(changed); !strings.Contains(problem, "edits the secret scan's policy (.gitleaksignore)") { + test.Fatalf("PolicyProblem = %q", problem) + } + targets, problem := verify.ScanTargets(ignored, nil) + if problem != "" { + test.Fatal(problem) + } + for _, target := range targets { + if copied := verify.ScanCopyPath(target.Path); slices.Contains(verify.ScannerPolicy, copied) { + test.Errorf("%s is copied into the scan as %s, where gitleaks loads it as policy", target.Path, copied) + } + } + + edited := slices.Clone(base) + edited[0].Blob = "allow everything" + removed := base[1:] + for name, head := range map[string][]verify.TreeEntry{"edited config": edited, "removed config": removed} { + if got := verify.PolicyChanges(base, head); !slices.Equal(got, []string{".gitleaks.toml"}) { + test.Errorf("%s: PolicyChanges = %q, want [.gitleaks.toml]", name, got) + } + } + nested := append(slices.Clone(base), verify.TreeEntry{Mode: "100644", Blob: "x", Path: "docs/.gitleaksignore"}) + requireNoPolicyChange(nested) // gitleaks reads only the root's +} + +func TestScanCopyPath(test *testing.T) { + test.Parallel() + for path, want := range map[string]string{ + ".gitleaksignore": ".gitleaksignore.scanned", + ".gitleaks.toml": ".gitleaks.toml.scanned", + "sub/.gitleaksignore": "sub/.gitleaksignore.scanned", + "main.go": "main.go", + "docs/gitleaks.toml": "docs/gitleaks.toml", + ".gitleaksignore.orig": ".gitleaksignore.orig", + } { + if got := verify.ScanCopyPath(path); got != want { + test.Errorf("ScanCopyPath(%q) = %q, want %q", path, got, want) + } + } +} + +// gitleaks gets the trusted config and ignore file by path and ignores +// inline gitleaks:allow comments, which a change could add to its own lines. +func TestGitleaksArgs(test *testing.T) { + test.Parallel() + args := strings.Join(verify.GitleaksArgs("/trusted/.gitleaks.toml", "/trusted/.gitleaksignore"), " ") + for _, want := range []string{"dir . ", "--config /trusted/.gitleaks.toml", "--gitleaks-ignore-path /trusted/.gitleaksignore", "--ignore-gitleaks-allow", "--redact", "--exit-code 1"} { + if !strings.Contains(args, want) { + test.Errorf("GitleaksArgs = %q, want %q", args, want) + } + } +} diff --git a/scripts/cilocal/verify/workflows.go b/scripts/cilocal/verify/workflows.go new file mode 100644 index 0000000..38a734c --- /dev/null +++ b/scripts/cilocal/verify/workflows.go @@ -0,0 +1,251 @@ +package verify + +import ( + "fmt" + "path" + "regexp" + "slices" + "strings" +) + +// The post-merge report (.github/workflows/post-merge-report.yml) comments +// on the merged pull request when a workflow that runs on a push to main +// fails there. It lists those workflows by name, so a new one must be added +// to it; ReportGaps names the ones it misses. The reader below knows only the +// workflow YAML this repository writes (block mappings, inline or dash lists). + +var ( + onKey = regexp.MustCompile(`^['"]?on['"]?:\s*(.*)$`) + nameKey = regexp.MustCompile(`^name:\s*(.+)$`) +) + +// yamlLine is a line without its comment, and its indent. +type yamlLine struct { + indent int + text string +} + +func yamlLines(text string) []yamlLine { + lines := []yamlLine{} + for _, raw := range strings.Split(text, "\n") { + if hash := strings.Index(raw, " #"); hash != -1 { + raw = raw[:hash] + } + trimmed := strings.TrimSpace(raw) + if trimmed == "" || strings.HasPrefix(trimmed, "#") { + continue + } + lines = append(lines, yamlLine{indent: len(raw) - len(strings.TrimLeft(raw, " ")), text: trimmed}) + } + return lines +} + +// children is the lines nested under lines[at]. +func children(lines []yamlLine, at int) []yamlLine { + end := at + 1 + for end < len(lines) && lines[end].indent > lines[at].indent { + end++ + } + return lines[at+1 : end] +} + +// child is the index of `key:` among the direct children in `lines` (the +// least indented), and its inline value. +func child(lines []yamlLine, key string) (int, string, bool) { + if len(lines) == 0 { + return 0, "", false + } + indent := lines[0].indent + for index, line := range lines { + if line.indent != indent { + continue + } + if value, found := strings.CutPrefix(line.text, key+":"); found { + return index, strings.TrimSpace(value), true + } + } + return 0, "", false +} + +func unquote(text string) string { return strings.Trim(strings.TrimSpace(text), `'"`) } + +// list is a key's value as a list: inline (`[a, b]` or a scalar) or the +// dash items nested under it. +func list(lines []yamlLine, at int, inline string) []string { + items := []string{} + if inline != "" { + for _, item := range strings.Split(strings.Trim(inline, "[]"), ",") { + if item = unquote(item); item != "" { + items = append(items, item) + } + } + return items + } + for _, line := range children(lines, at) { + if item, found := strings.CutPrefix(line.text, "- "); found { + items = append(items, unquote(item)) + } + } + return items +} + +// WorkflowName is a workflow's top-level `name:`, or "". +func WorkflowName(text string) string { + for _, line := range yamlLines(text) { + if match := nameKey.FindStringSubmatch(line.text); line.indent == 0 && match != nil { + return unquote(match[1]) + } + } + return "" +} + +// PushesToMain is whether a workflow runs on a push to main: `on: push`, or +// a `push:` trigger whose branches (none listed: every branch) match main. +func PushesToMain(text string) bool { + lines := yamlLines(text) + for at, line := range lines { + match := onKey.FindStringSubmatch(line.text) + if line.indent != 0 || match == nil { + continue + } + if match[1] != "" { + return slices.Contains(list(nil, 0, match[1]), "push") + } + triggers := children(lines, at) + push, inline, found := child(triggers, "push") + if !found { + return false + } + filters := children(triggers, push) + if inline != "" || len(filters) == 0 { + return true + } + branches, value, listed := child(filters, "branches") + if !listed { + _, _, tagsOnly := child(filters, "tags") + return !tagsOnly + } + for _, pattern := range list(filters, branches, value) { + if matched, _ := path.Match(pattern, Main); matched || pattern == "**" { + return true + } + } + return false + } + return false +} + +// ReportedWorkflows is the `workflow_run.workflows` list of a workflow. +func ReportedWorkflows(text string) []string { + lines := yamlLines(text) + for at, line := range lines { + if line.text != "workflow_run:" { + continue + } + block := children(lines, at) + if index, inline, found := child(block, "workflows"); found { + return list(block, index, inline) + } + } + return nil +} + +// ReportGaps is the names of the workflows (file name to content) that run +// on a push to main but are missing from the post-merge report's list. +func ReportGaps(workflows map[string]string, report string) []string { + reported := ReportedWorkflows(report) + gaps := []string{} + for _, text := range workflows { + if name := WorkflowName(text); PushesToMain(text) && !slices.Contains(reported, name) { + gaps = append(gaps, name) + } + } + slices.Sort(gaps) + return gaps +} + +// TrustedTrigger is the only event ci.yml runs on. +const TrustedTrigger = "pull_request_target" + +// TrustProblems is why a pull request workflow (ci.yml) would let the pull +// request decide its own required checks, or nothing. Its jobs are the gate, +// so they run the base's code on the head as data: +// +// - only on pull_request_target: on pull_request GitHub takes the workflow +// and its checkout from the pull request, so a branch or fork could +// rewrite the job, or the code it runs (an init() that exits 0); +// - no checkout of anything but the default, the base; +// - no expression inside a run script, where a branch name or title would +// run as shell; values go through env; +// - the secret scan reads the head's objects (--head), so it scans the +// head's files with the base's scanner and .gitleaks.toml; +// - the attestation check knows the head's repository (HEAD_REPO), so a +// fork's head fails closed. +func TrustProblems(text string) []string { + lines := yamlLines(text) + problems := []string{} + if triggers := triggersOf(lines); !slices.Equal(triggers, []string{TrustedTrigger}) { + problems = append(problems, fmt.Sprintf("it runs on %v, not %s alone: on pull_request the pull request supplies the workflow and the code it runs", triggers, TrustedTrigger)) + } + for at, line := range lines { + item, isItem := strings.CutPrefix(line.text, "- ") + if !isItem { + continue + } + step := append([]yamlLine{{indent: line.indent + len("- "), text: item}}, children(lines, at)...) + problems = append(problems, stepTrustProblems(step)...) + } + return problems +} + +// triggersOf is the events under a workflow's top-level `on:`. +func triggersOf(lines []yamlLine) []string { + for at, line := range lines { + match := onKey.FindStringSubmatch(line.text) + if line.indent != 0 || match == nil { + continue + } + if match[1] != "" { + return list(nil, 0, match[1]) + } + events := []string{} + block := children(lines, at) + for _, event := range block { + if event.indent == block[0].indent { + name, _, _ := strings.Cut(event.text, ":") + events = append(events, unquote(name)) + } + } + return events + } + return nil +} + +func stepTrustProblems(step []yamlLine) []string { + problems := []string{} + text := func(prefix string) bool { + return slices.ContainsFunc(step, func(line yamlLine) bool { return strings.HasPrefix(line.text, prefix) }) + } + mentions := func(fragment string) bool { + return slices.ContainsFunc(step, func(line yamlLine) bool { return strings.Contains(line.text, fragment) }) + } + if mentions("uses: actions/checkout") && (text("ref:") || text("repository:")) { + problems = append(problems, "a checkout names a ref or repository: only the default checkout, the base, is trusted") + } + for at, line := range step { + if !strings.HasPrefix(line.text, "run:") { + continue + } + script := append([]yamlLine{line}, children(step, at)...) + if slices.ContainsFunc(script, func(line yamlLine) bool { return strings.Contains(line.text, "${{") }) { + problems = append(problems, fmt.Sprintf("%q puts an expression in its script: pass it through env", line.text)) + } + } + if mentions("scripts/cilocal secret-scan") && !mentions("--head") { + problems = append(problems, "the secret scan does not read the head as data (--head)") + } + if mentions("scripts/cilocal check") && !text("HEAD_REPO:") { + problems = append(problems, "the attestation check is not told the head's repository (HEAD_REPO), so a fork's head would not fail closed") + } + return problems +} diff --git a/scripts/cilocal/verify/workflows_test.go b/scripts/cilocal/verify/workflows_test.go new file mode 100644 index 0000000..7711620 --- /dev/null +++ b/scripts/cilocal/verify/workflows_test.go @@ -0,0 +1,167 @@ +package verify_test + +import ( + "os" + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/jayleaton/go-fast-stack/scripts/cilocal/verify" +) + +func TestPushesToMain(test *testing.T) { + test.Parallel() + for _, testCase := range []struct { + name, text string + want bool + }{ + {"push to main", "name: Deploy\non:\n push:\n branches: [main] # the deployed branch\n workflow_dispatch:\n", true}, + {"dash list", "on:\n push:\n branches:\n - 'release'\n - \"main\"\n", true}, + {"glob", "on:\n push:\n branches: ['**']\n", true}, + {"every branch", "on:\n push:\n pull_request:\n", true}, + {"inline push", "on: push\n", true}, + {"inline list", "'on': [pull_request, push]\n", true}, + {"other branches", "on:\n push:\n branches: ['bench/**']\n", false}, + {"tags only", "on:\n push:\n tags: ['v*']\n", false}, + {"pull requests only", "# push: [main]\n'on':\n pull_request:\n types: [edited]\njobs:\n push:\n runs-on: x\n", false}, + {"workflow_run", "on:\n workflow_run:\n workflows: [Deploy staging]\n", false}, + {"no triggers", "name: x\n", false}, + {"empty on", "on:\nname: x\n", false}, + } { + test.Run(testCase.name, func(test *testing.T) { + test.Parallel() + if got := verify.PushesToMain(testCase.text); got != testCase.want { + test.Fatalf("PushesToMain(%q) = %v, want %v", testCase.text, got, testCase.want) + } + }) + } +} + +func TestReportGaps(test *testing.T) { + test.Parallel() + report := "name: Post-merge report\non:\n workflow_run:\n workflows:\n - Deploy staging\n types: [completed]\n" + if got := verify.ReportedWorkflows(report); !slices.Equal(got, []string{"Deploy staging"}) { + test.Fatalf("ReportedWorkflows = %q", got) + } + if got := verify.ReportedWorkflows("on:\n workflow_run:\n workflows: [A, 'B']\n"); !slices.Equal(got, []string{"A", "B"}) { + test.Fatalf("ReportedWorkflows of an inline list = %q", got) + } + if got := verify.ReportedWorkflows("on: push\n"); got != nil { + test.Fatalf("ReportedWorkflows without workflow_run = %q", got) + } + if got := verify.ReportedWorkflows("on:\n workflow_run:\n"); got != nil { + test.Fatalf("ReportedWorkflows of an empty workflow_run = %q", got) + } + if got := verify.WorkflowName("on: push\n"); got != "" { + test.Fatalf("WorkflowName without a name = %q", got) + } + workflows := map[string]string{ + "deploy-staging.yml": "name: Deploy staging\non:\n push:\n branches: [main]\n", + "box.yml": "name: 'box-ami'\non:\n push:\n branches: [main]\n", + "ci.yml": "name: CI\non:\n pull_request:\n", + } + if got := verify.ReportGaps(workflows, report); !slices.Equal(got, []string{"box-ami"}) { + test.Fatalf("ReportGaps = %q, want [box-ami]", got) + } +} + +// Every workflow of this repository that runs on a push to main is in the +// post-merge report's list, so a failure after merge is reported on the PR. +func TestThePostMergeReportListsEveryPushToMainWorkflow(test *testing.T) { + test.Parallel() + dir := filepath.Join("..", "..", "..", ".github", "workflows") + files, err := filepath.Glob(filepath.Join(dir, "*.yml")) + if err != nil || len(files) == 0 { + test.Fatalf("no workflows in %s: %v", dir, err) + } + workflows := map[string]string{} + for _, file := range files { + text, err := os.ReadFile(file) + if err != nil { + test.Fatal(err) + } + workflows[filepath.Base(file)] = string(text) + } + report, found := workflows["post-merge-report.yml"] + if !found { + test.Fatal("no .github/workflows/post-merge-report.yml") + } + if !verify.PushesToMain(workflows["deploy-staging.yml"]) { + test.Fatal("deploy-staging.yml does not read as a push to main") + } + if gaps := verify.ReportGaps(workflows, report); len(gaps) > 0 { + test.Fatalf("post-merge-report.yml's workflow_run.workflows misses %q, which run on a push to main", gaps) + } +} + +// ci.yml as #134 had it at bf0b5d3 (Randy's review): pull_request runs the +// pull request's own workflow and code, so a fork adding +// `func init() { os.Exit(0) }` to scripts/cilocal passed both checks, and +// the secret scan used the fork's .gitleaks.toml. +const untrustedCI = `name: CI +on: + pull_request: + types: [edited] +jobs: + secrets: + name: Secret scan + runs-on: ${{ fromJSON(vars.CI_RUNNER || '"ubuntu-latest"') }} + steps: + - uses: actions/checkout@v7 + with: + ref: ${{ github.event.pull_request.head.sha }} + - name: No secrets in the changed files + env: + PR: ${{ github.event.pull_request.number }} + run: go run ./scripts/cilocal secret-scan --pr "$PR" + local: + name: Local verification + steps: + - uses: actions/checkout@v7 + with: + repository: ${{ github.event.pull_request.head.repo.full_name }} + - name: make ci-local passed on this head + run: | + echo "${{ github.event.pull_request.title }}" + go run ./scripts/cilocal check +` + +func TestTrustProblems(test *testing.T) { + test.Parallel() + problems := verify.TrustProblems(untrustedCI) + for _, want := range []string{ + "it runs on [pull_request], not pull_request_target alone", + "a checkout names a ref or repository", + "puts an expression in its script", + "does not read the head as data (--head)", + "is not told the head's repository (HEAD_REPO)", + } { + requireMention(test, problems, want) + } + if got := len(problems); got != 6 { // both checkouts are named + test.Errorf("TrustProblems = %d problems, want 6: %q", got, problems) + } + if got := verify.TrustProblems("on: [pull_request_target, push]\n"); len(got) != 1 { + test.Errorf("TrustProblems of an inline trigger list = %q", got) + } + requireNone(test, verify.TrustProblems("on: pull_request_target\n")) + requireMention(test, verify.TrustProblems("name: no triggers\n"), "it runs on [], not pull_request_target alone") +} + +// The required checks run only the base's code: a branch or fork that +// rewrites scripts/cilocal, .gitleaks.toml or ci.yml itself changes nothing +// the Secret scan and Local verification jobs do, and a fork's head fails. +func TestCIRunsOnlyTrustedCode(test *testing.T) { + test.Parallel() + text, err := os.ReadFile(filepath.Join("..", "..", "..", ".github", "workflows", "ci.yml")) + if err != nil { + test.Fatal(err) + } + requireNone(test, verify.TrustProblems(string(text))) + for _, want := range []string{"cilocal secret-scan", "cilocal check --claim", "cilocal check\n"} { + if !strings.Contains(string(text), want) { + test.Errorf("ci.yml no longer runs %q", want) + } + } +}