Repository navigation
ci: run the suite and the gate through kindkit's workflow - #78
Merged
Merged
Conversation
kindkit's reusable conformance workflow reads the runner report's `root` key, KINDKIT_REPORT_JSON and KINDKIT_GATE_REPORT, none of which exist at bf716e3 (v0.1.0). kindkit has no newer tag, so the pin moves to the commit on its main that added the workflow. No rowspec code changes. Re-measured: check, conform, conform-alt and mutants print identical output; test differs only in wall time. Refs kindspec/kindkit#4 Refs kindspec/kindkit#3
A new `kind` job calls kindkit's reusable kind.yml at 8e21b55, the commit the dev group pins. It validates the case tree, runs the suite on both implementations against conformance/cases, and judges the mutation gate by its own report. It reports as `kind / conformance`. The `conformance` job drops `just conform`, `just conform-alt` and `just mutants`, and keeps lint, `just test` without the extra, and the example recipe. Its name, and so its required check, is unchanged. tests/test_pins.py now fails if the workflow's kindkit commit and the package pin differ. Refs kindspec/kindkit#4 Refs kindspec/kindkit#3
CameronBrooks11
marked this pull request as ready for review
October 8, 2026 01:42
GitHub owner and repository names are case-insensitive, so a stale ref spelled KindSpec/KindKit called the same workflow but was never compared with the pin, and the test passed beside a correct ref.
CameronBrooks11
added a commit
that referenced
this pull request
Oct 8, 2026
## What Prepares the 0.3.0 release. No code changes. - The version is `0.3.0` in `pyproject.toml`, in `reference/rowspec/__init__.py`'s `__version__`, and in the action's default install pin (`rowspec==0.3.0`). `uv.lock` changes only rowspec's own version line. - The README's action example points at `@v0.3.0`. - The changelog's unreleased section becomes `[0.3.0] — 2026-10-08`, with compare links and an empty `[Unreleased]` above it. - The ROADMAP's "Where it is" figures are re-measured (below), and a short "Shipped in 0.3.0" section sits above "Shipped in 0.2.0". **Not tagged.** The tag is the owner's call after review. ## Why 0.3.0 Under SemVer's pre-1.0 convention a minor bump carries behaviour changes, and this release has three that a consumer can observe: - **`rowspec check` output changes on CSV input**: a lone `CR` now produces a warning, and `--strict` refuses it (#59). A pipeline running `--strict` on a file with a lone `CR` goes from green to red. - **The action has a new output**, `eval-outcome` (#72). - **`conformance/run_cases.py` exits 2, not 1**, when the implementation will not import (#62). Anyone scripting against the runner's exit code sees a different value. A patch release (0.2.1) would claim no observable behaviour changed, which is not true here. ## The changelog against `git log v0.2.0..origin/main` All 19 commits in `git log --oneline v0.2.0..origin/main` were checked against the section. The existing entries covered #52/#72, #68, #78, #77, #64, #60, #76 and #57. I added these, which consumers can see but the section did not mention: - **Added**: the action's `eval-outcome` output (#72). It came in with the post-release workflow and was missing from the changelog. - **Changed**: the runner's exit 2 on an unimportable implementation (#62); the action's `setup-python` pinned to a commit (#71); the release gate running `just test`/`just test-xlsx` and checking `__version__` (#69, #70); SPDX headers and the new `tests/` and `conformance/reserved/` `LICENSE` files (#74); the `reference/` boundary check seeing dynamic imports (#65). - **Licensing (#74), stated in the changelog entry**: `conformance/reserved/` (was MIT, from `conformance/LICENSE`) and `docs/ci/rowspec-check.yml` (was CC-BY-4.0) are now CC0-1.0. The root `LICENSE` now names Apache-2.0 OR MIT for `action.yml`, `.github/workflows/` and the `justfile`, which it did not list before. Both relicensings move to more permissive terms. `git diff 228b61e~1 228b61e -- LICENSE docs/LICENSE` shows all three. The section's opening note says what is left out. #58 and #75 are documentation-only, and the note covers them as 0.2.0's did. #66, #67 and #73 are dev and CI changes with no effect outside this repository, and the note now names them. I checked the new claims before writing them: - `cd conformance && uv run python run_cases.py no_such_impl` prints the `HARD FAILURE:` line and exits `2`. - `grep -rhn 'uses:' .github/workflows action.yml | grep -vE '@[0-9a-f]{40}'` matches only local `./` refs and comments. ## Re-measured figures Run on this branch, in the order listed. Each ROADMAP figure now has the command that produced it beside it. | figure | command | | --- | --- | | `All checks passed!` | `just check` | | 410 conformance cases | `find conformance/cases -name expect.json \| wc -l` | | `0 failure(s) across 410 case(s)` | `just conform` | | `0 failure(s) across 410 case(s)` (second impl) | `just conform-alt` | | `73 killed, 0 survived, 2 equivalent, 0 stale, 0 broken` | `just mutants` | | 75 distinct mutations | `just mutants \| grep -ciE '^ (killed\|equiv\|survived\|stale\|broken\|bogus) '` | | `73 passed, 1 skipped` | `just test` | | `9 passed` | `just test-xlsx` | Two ROADMAP rows changed: - `just test` went from 52 passed to 73 passed. The new tests are `test_pins.py`, `test_licensing.py` and `test_tooling.py`, plus additions to existing files since v0.2.0. - The "75 distinct mutations" row's right-hand column was prose ("the gate refuses two that mutate identically"). It now gives the command, with the refusal kept as a parenthetical. The `just test-xlsx` row is new. The count command matches case-insensitively and includes `bogus`, because kindkit prints the failing verdicts in uppercase (`SURVIVED`, `STALE`, `BROKEN`, `BOGUS`). It was checked against a sample of all six per-mutant line shapes and kindkit's summary lines: ``` new regex, 6 verdict lines: 6 new regex, 5 summary lines: 0 old regex, 6 verdict lines: 2 ``` It still gives `75` on this tree. ## Dry run of `release.yml`'s build job I built from `git archive HEAD` of this commit, unpacked into an empty directory, so the build saw only tracked files. The commands are the workflow's. - `uv build` produced `rowspec-0.3.0.tar.gz` and `rowspec-0.3.0-py3-none-any.whl`. - **Tag check**, with `GITHUB_REF_NAME=v0.3.0`: `tag v0.3.0 matches built version 0.3.0`, exit 0. With `GITHUB_REF_NAME=v0.2.0` as a negative control: `::error::tag v0.2.0 does not match built version 0.3.0`, exit 1. - **Action pin check**: `action.yml pins rowspec==0.3.0, matching the release`. - **Wheel smoke step**: I extracted its `run:` script from the YAML with `yaml.safe_load` and ran it with `bash -e` from the build directory. ``` wheel version: __version__ and metadata agree on 0.3.0 (…/uvcache/archive-v0/…/site-packages/rowspec/__init__.py) wheel CLI: eval refuses a broken total and annotates it ``` The printed `rowspec.__file__` is under uv's ephemeral environment, not the checkout. The version assertion therefore ran against the wheel. - **METADATA**: `Version: 0.3.0`, `License-Expression: Apache-2.0 OR MIT`, `Requires-Python: >=3.11`, `Provides-Extra: xlsx`, `Requires-Dist: openpyxl>=3.1; extra == 'xlsx'`. Besides its `.dist-info`, the wheel contains only `rowspec/` and `rowspec_xlsx/`. - **A fresh venv with the wheel installed** (`uv venv` + `uv pip install <wheel>`): - `rowspec eval` on a table whose total is `#REF!(/0)` exits 1. - `rowspec check` on the same table exits 0 (`0 refused`). That is the reason the action runs both commands. - `rowspec eval docs/example` exits 0 (`grand = 133.5`); `rowspec check docs/example` exits 0. - A CSV with a lone `CR` gives a warning and exits 0 under `check`, and is refused with exit 1 under `check --strict`. - **A second fresh venv with `<wheel>[xlsx]`**: `rowspec_xlsx` imports from the venv, with openpyxl 3.1.5. ## After merge Tag `v0.3.0` on the squash commit. `release.yml` then runs the gate, builds, publishes through the trusted publisher, and runs `action-published.yml` against the published wheel. This is the first release in which that job runs (#52); before now it has only run on branches pushed to probe it. If it goes red, the tag and the upload are already final, and the fix is a patch release.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows #77, the kindkit pin bump, which is merged. This branch has
mainmerged into it (no rebase); the diff againstmainis this changealone.
What
rowspec CI now calls kindkit's reusable workflow, following kindkit's
README section "Adopting it where
conformanceis already a requiredcheck".
New job
kindin.github/workflows/check.yml. It callskindspec/kindkit/.github/workflows/kind.yml@8e21b558c85d5612009f3a1d69a3a68bc0191637with:
The job grants
contents: readandactions: read, which the workflowneeds to find its own commit. It reports as
kind / conformance.The
conformancejob keeps its name. Its required check keepsreporting, so the existing ruleset still applies. It drops the
just conform,just mutantsandjust conform-altsteps, which are now theworkflow's gates. It keeps everything the workflow does not cover:
just check, the extra-absent assertion,just testwithout the extra,and the example recipe.
xlsx-extraandaction-selftest.ymlare unchanged.No rowspec gate code changed.
mutants.pycallskindkit.gate(),and at
8e21b55that call writesKINDKIT_GATE_REPORTitself. Its probegoes through
probe_command, which now carries the runner'srootthrough.
run_cases.pyis built onkindkit.cli, which readsKINDKIT_REPORT_JSON.just teststill runs the suite and the gate itself, throughtests/test_conformance.py, as it did before. So theconformancejobstill exercises them, but under pytest rather than as gate steps.
The workflow ref and the dev pin stay in step
kindkit's commit now appears in two places:
uses:incheck.yml, andthe
devdependency group inpyproject.toml. The workflow fetcheskindkit's
tools/at the first. The runner and the gate those tools driveare the package pinned by the second. If the two differ, the gates check a
report contract that the installed kit may not keep.
tests/test_pins.py::test_the_kindkit_workflow_runs_at_the_commit_the_package_is_pinned_tofinds every
uses:of akindspec/kindkitworkflow and requires each oneto name the dev group's commit. It also fails if it finds no such
uses:at all, so it cannot pass vacuously. Comments in
pyproject.toml,check.ymlandAGENTS.mdall say to move the two together.The owner and the repository are matched case-insensitively, because
GitHub treats
KindSpec/KindKitas the same repository. The commit muststill be lowercase hex. Review found the gap: with the original
case-sensitive pattern, a second job calling
KindSpec/KindKit/.github/workflows/kind.yml@bf716e3...beside the correctref passed (
1 passed).The test was shown to fail. Each mutation was applied to the file,
hash-checked to confirm it changed, and then reverted. This is the sweep
after the fix:
CI, green and red
Green on the current head (
3ce2477): run37715625047,
with every context green,
kind / conformanceincluded. Green on the first push: run37713862672.
Every step of every job is green,
kind / conformanceincluded. From thekind / conformancejob log (job 113105742193):The gate figures are the same as
just mutantsgives onmain. Thesourceline namesreference/rowspec/table.py. The workflow does notcheck which file was mutated (kindkit README), so a reviewer has to.
Red on a throwaway branch: run
37713891424.
tmp/red-demo-broken-casewas this branch with one expectation broken:eval/aggregate-over-an-exponent-name-is-a-broken-referencewanted#REF!(1e4)instead of#REF!(1e3). Its throwaway PR was #79, which isnow closed, and the branch is deleted. Steps of
kind / conformance(job113105834937):
with
The mutation step stays green there by design. The gate counts kills
against the baseline's failing set, and that run logged
baseline-fail:for the broken case. So a wrong expectation is the suite steps' job to
catch, not the gate's. The
conformancejob also went red, atjust test.Required checks: one ruleset edit, immediately before merge
conformance,xlsx-extra,passes-on-a-good-treeandfails-on-a-broken-totalall still report on this branch. So the PR ismergeable under today's ruleset. But
kind / conformancewould gatenothing until it is required (kindspec/.github AGENTS.md §3.2: a new
workflow is not required by default).
The owner's edit: immediately before merging this PR, and not in it,
add
kind / conformance(integration 15368, GitHub Actions) to therequired status checks of ruleset
22723542. Keep all four existingcontexts. This is an addition, not a swap: the
conformancejob stillexists and still reports. So no context goes unreported on this branch,
before or after the edit.
The
diffline was run against the live ruleset, without thePUT. Itshows exactly the one added context. After the edit, any other open branch
that does not have this change sits at Expected for
kind / conformanceuntil it takes it, and
strictmakes it do that anyway.Order: #77 is merged and this branch is up to date with
main. Theowner adds the required check immediately before merging this PR, then
merges it.
Not changed here
release.yml'sgatejob still runsjust conform,just mutantsand
just conform-altdirectly. It is not a required check, and movingit onto the workflow is a separate change.
needs a kindkit PR once this merges. Whether A reusable CI workflow that runs the suite, the gate, and the second implementation kindkit#4 and spec: the declaration production forbids trailing whitespace a fixture requires #3
close at that point is the owner's call; this PR does not close them.
STATE.mdandAGENTS.md§3.1 list rowspec'srequired checks, and will need
kind / conformanceadded once theruleset edit is made.
Refs kindspec/kindkit#4
Refs kindspec/kindkit#3