Skip to content

Test specifications as codelinks one-liners, test results through sphinx-test-reports 4 - #6

Draft
LuSilber wants to merge 4 commits into
developfrom
feat/test-specs-codelinks-test-reports
Draft

LuSilber wants to merge 4 commits into
developfrom
feat/test-specs-codelinks-test-reports

Conversation

@LuSilber

@LuSilber LuSilber commented Nov 6, 2026

Copy link
Copy Markdown

Test specifications move into the test sources as codelinks one-line comments, like the implementation needs. Test results come from sphinx-test-reports' test-report directive again, which ubc 0.39 now reads as well.

Draft: CI can't pass until the dependency stack below is pinned.

Changes, one commit each

  1. Drop get_rst. ubc 0.39 warns about the key, and False is already its default.
  2. Test specifications as one-liners.
    • The 30 specs change from @rst .. test:: blocks to // @need <GTest name>, <id>, test, [<tests>] comments in a new tests codelinks project.
    • Each component page shows them in ## Verification via src-trace.
    • codelinks follows each variant's #ifdef branches using the test-kit build's compile database.
  3. Test results through the test-report directive.
    • SPL_TEST_RESULTS_AS_NEEDS is turned OFF, and a [test_reports] table is shared by both readers.
    • The extension registers the result types and fields, so the hand-declared copies are removed.
    • results is now a sphinx-needs 9 derived link: each spec links to the test cases whose case matches.
  4. Parameterised specs link every instance.
    • A derived link only matches names exactly, so Suite/Test.Case/* can't match its instances.
    • Each spec now names its cases in test_cases, separated by \,.
    • The field is a string because ubc drops codelinks list fields that aren't link types. The derived link splits strings on commas, so two instance names containing a comma were renamed.

Need-level comparison with CI run 42134750198 (d351c44, Disco and Spa test reports)

  • Same as CI:
    • The testfile, testsuite and testcase IDs.
    • Every spec's results links, including the parameterised ones (TS_LC-003: 3, TS_MCK-001: 7, TS_FC-001: 4, TS_FC-100: 4).
  • Changed:
    • A spec's docname is now its component page.
    • Specs get a remote-url.
    • Results have is_import set to false.
  • Removed:
    • REQ_37 and REQ_58 (see open points).
    • Disco's integration-test results, only because that test can't build on macOS. CI will have them.

Verification (local, macOS)

  • test_ubproject_config.py: 29 passed.
  • Docs gate: the 38 component-report cells outside Disco pass in both readers, strict.
  • Not run locally: the Disco and whole-variant report cells (AppleClang and hammocking).

Open points before this can merge

  • spl-core pin: this needs a useblocks spl-core with sphinx-needs >=9.2,<10, sphinx-test-reports >=4,<5, and the extension loaded as sphinx_test_reports (prototyped on branch feat/sphinx-test-reports-4 from ce62088, not pushed yet).
  • SPLed's own dependencies: bump to sphinx-codelinks >=2.1,<3, then update poetry.lock. The AGENTS.md pin paragraph needs updating too.
  • REQ_37 / REQ_58: sphinx-needs 9 refuses these two untitled entries from the ubConnect ReqIF export. This fails Sphinx's strict build for variant-level docs; ubc already skipped them.
  • The gate doesn't compare test-report fields: it only compares fields declared in ubproject.toml. Because of that, errors differs between the readers unnoticed: -1 in Sphinx, 0 in ubc.
  • Follow-up: ub-test-reports should split a GTest instance into case_parameter, so test_cases can be removed. The task has been handed to the rollout session.

🤖 Generated with Claude Code

LuSilber and others added 4 commits November 6, 2026 16:06
ubc 0.39 warns config.codelinks_removed_key for it, and False is its
default in sphinx-codelinks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 30 test specifications move from @rst blocks in the test sources to
`// @need <GTest name>, <id>, test, [<tests>]` comments, read by a
`tests` codelinks project like the implementation needs. Each component
shows them in its Verification section with src-trace. Only a test-kit
build compiles the test sources, so codelinks takes their #ifdef branches
from that build's compile database and skips them in any other.

spl-core's results converter finds the specifications in the clanguru
listings, so their `results` links are gone until the next commit
derives them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ective

sphinx-test-reports 4 and ubc 0.39 both read the test-report directive,
so spl-core writes it again (SPL_TEST_RESULTS_AS_NEEDS OFF) and both
readers take its settings from the [test_reports] table. The extension
registers the testfile, testsuite and testcase types and fields, so the
hand-declared copies go, and so does conf.py's filter for `file`.

A test specification's `results` link is derived in ubproject.toml
(sphinx-needs 9): the test cases whose `case` is the specification's
title. The vendored-config check compares only the link names, since the
derive is this project's own.

Needs spl-core with sphinx-needs >=9.2 and sphinx-test-reports >=4, and
sphinx-codelinks 2.1, which this commit does not pin yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A join compares exactly, so a specification titled
`Suite/Test.Case/*` lost the results the old converter matched by
wildcard. Each specification now names the GTest cases it covers in a
`test_cases` field, separated by `\,`, and the derived `results` link
joins on it: TS_LC-003, TS_MCK-001, TS_FC-001 and TS_FC-100 link their
3, 7, 4 and 4 instances again.

The field is a string because ubc drops a codelinks list field that is
not a link type, and the join splits a string on commas, so the two
instance names with a comma are renamed. Drop the field once
ub-test-reports splits the GTest instance into case_parameter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant