Skip to content

feat(query): add relationship-derived projection columns - #368

Merged
HuiJun merged 11 commits into
developfrom
feature/traceability-matrix-columns
Sep 18, 2026
Merged

HuiJun merged 11 commits into
developfrom
feature/traceability-matrix-columns

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

What and why

Stacked on #365 (feature/query-coverage-set-operations): merge #365 into develop first; this PR then retargets to develop with only the RelatedColumn changes. RelatedColumn runs the callback-based traverseRelated #365 introduced for WhereRelated, so the two share one traversal. Because the cookbook model now holds both the coverage fixtures and the Traceability package, the cookbook's UnverifiedRequirements/UncoveredRequirements figures gain Traceability::downlinkRequirement.

The document-query engine could list the satisfiers of one requirement (RelatedElements(relationshipKind = "satisfaction", direction = "incoming") with req bound), but a Project column could only read the row's own features, so the classic traceability matrix — every requirement with its satisfiers and verifiers in one table — could not be written; the worked example emits one table per requirement.

RelatedColumn(name, relationshipKind, direction, maxDepth, aggregate = "list") is a new columns = (...) entry of Project, beside Column(name, expression):

Project(
    source = WhereType(source = Descendants(source = root, maxDepth = 1), type = "RequirementUsage"),
    properties = ("shortName", "name"),
    columns = (
        RelatedColumn(name = "satisfiedBy",   relationshipKind = "satisfaction", direction = "incoming", maxDepth = 1),
        RelatedColumn(name = "verifiedBy",    relationshipKind = "verification", direction = "incoming", maxDepth = 1),
        RelatedColumn(name = "verifications", relationshipKind = "verification", direction = "incoming", maxDepth = 1, aggregate = "count")
    )
)
  • For each row it traverses the relationship from the row's element (an object row's declaration) with the same breadth-first walk RelatedElements uses — related.go now exposes validateRelationship/traverseRelated, shared by both, so every kind supportedRelationship accepts is accepted here and the visit budget is charged the same way.
  • list (default) yields the related elements as one multi-valued cell in traversal order (an empty cell when none: (none) in the CLI report, blank in a document); count an integer; any a Boolean.
  • The cells are read downstream by name: WhereFeature (verifications = 0 for unverified requirements), OrderBy, and a document table's groupBy (docplan.columnNames now sees related columns).
  • Unknown kind / direction are the typed errors RelatedElements raises, reported with the column name; an unsupported aggregate is a typed planning error.
  • CLI, Markdown, HTML (each element keeps its data-element), gRPC run_query and the Python client carry the multi-valued cell through the existing multi-value contract — no serialisation change was needed.

Also fixed on the way: runtime.distinctAssertions/satisfactionKey deduplicated satisfaction assertions by *symbols.Symbol, so an assertion reached through two scope trees over the same document (the runtime's cached tree and a re-indexed one) was checked and reported twice in Verdicts. They now key on the declaration, the identity sameDeclaration already uses.

Also: the PDF stylesheet sized tables to their content, so a cell holding long unbreakable tokens (a list of qualified names) pushed the rightmost columns past the page edge, where they were cut off — the count column of the traceability matrix was invisible. Tables now take the text width, as the HTML backend's do, and cells wrap wherever they must.

Docs: cookbook "Traceability matrix" recipe with real console output; new docs/manual/examples/traceability.sysml + rendered traceability.md (requirement list, one matrix table, Verdicts table) wired into docs/manual/README.md and mkdocs.yml; worked-example closing note; CLI/REPL reference, introduction and authoring mentions; DocumentQueries.sysml vocabulary (+ regenerated stdlib.snapshot).

Specification basis

OpenSysML non-normative extension (DocumentQueries); no SysML v2 clause. Adds two rows to the document-query sections of docs/project/spec-compliance.md and extends the groupBy row.

How it was verified

  • queryplan/columns_test.go: TestCompileRelatedColumns, TestCompileRelatedColumnDiagnostics (argument validation, aggregates, duplicate names).
  • queryexec/related_column_test.go: matrix over testdata/trace_matrix.sysml, depth/direction, column-named errors, visit budget, WhereFeature/OrderBy over projected cells, object rows.
  • grpc/related_column_test.go: repeated values, empty list, count and any over RunDocumentQuery.
  • docplan/runs_test.go:TestCompileGroupedTableSeesRelatedColumns.
  • runtime/validate_test.go:TestValidateObjectReportsACarriedSatisfactionOnceAcrossScopeTrees (fails on develop).
  • docpdf/docpdf_test.go:TestStyleSheetKeepsTablesWithinThePage.
  • cmd/sysml/manual_examples_test.go:TestManualTraceabilityExample: -run-query, byte-for-byte Markdown against the committed traceability.md, HTML data-element cells.
  • Gates: gofmt -l . empty, go vet ./..., go build ./..., make lint, OPENSYSML_REQUIRE_TRAINING_CORPUS=1 OPENSYSML_REQUIRE_PILOT_CORPORA=1 go test -count=1 ./... (incl. TestCorpusRoundTrip, TestTrainingExamples|TestPilotCorpora), python3 scripts/changelog.py check, check-doc-links.py, check-doc-ids.py, check-doc-figures.py, go run ./cmd/doc-counts -check, python -m mkdocs build --strict.
  • By hand: the traceability example rendered as HTML and as PDF (WeasyPrint toolchain); the multi-valued verifiedBy cell renders as linked element spans / comma-separated text, no literal markup in the PDF text, and every column of the matrix and verdict tables is within the page.

Checklist

  • make test and make lint pass locally
  • Tests added or updated for the change
  • Documentation extended where it already covers the surface (see CONTRIBUTING.md)
  • Changelog entry added as changes/unreleased/<slug>.<section>.md, not as an edit to CHANGELOG.md
  • baselines regenerated and make docs-counts run if a gate count moved (compliance rows need nothing: the census is counted at docs build)
  • No internal work-item labels (waves, slices, F4, K5) in the body, docs, or changelog

Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/3f84250d945648d39a5010aebc4f829a
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/3f84250d945648d39a5010aebc4f829a?variant=devin
Requested by: @HuiJun

devin-ai-integration Bot and others added 2 commits September 17, 2026 06:22
`RelatedColumn(name, relationshipKind, direction, maxDepth, aggregate)` is a `Project` column beside `Column(...)`: for each row it traverses one relationship kind with the breadth-first walk `RelatedElements` uses and yields the related elements as a multi-valued cell, their count, or whether there are any. The cells are read downstream by `WhereFeature`, `OrderBy` and a table's `groupBy`, so one query produces a traceability matrix over every requirement.

The manual gains a "Traceability matrix" cookbook recipe and a rendered traceability report example; the verdict engine deduplicates satisfaction assertions by declaration rather than scope-tree symbol, so a document indexed twice no longer doubles a requirement's satisfaction row.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

devin-ai-integration Bot and others added 2 commits September 17, 2026 06:43
The PDF stylesheet sized tables to their content, so a cell holding long unbreakable tokens — a list of qualified names, say — pushed the rightmost columns past the page edge where they were cut off. Tables now take the text width, as the HTML backend's tables do, and cells wrap wherever they must.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review September 17, 2026 07:11

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

…ty-matrix-columns

Co-Authored-By: jason.han <hanhuijun@gmail.com>
devin-ai-integration[bot]

This comment was marked as resolved.

An event posted from outside the run has no behavior or object, so its
row's Declaration() is nil and an outgoing lineage traversal dereferenced
it. The column now returns a typed undeclared-row error naming the column
and the row before traversing.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
devin-ai-integration[bot]

This comment was marked as resolved.

WhereFeature refused an element value as an unsupported operator and
OrderBy held every element equal, so a RelatedColumn list (and a verdict's
assertion) could not be filtered or sorted. Elements now compare under the
text operators and order as the qualified name a cell prints them by.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
devin-ai-integration[bot]

This comment was marked as resolved.

Merge feature/query-coverage-set-operations so RelatedColumn shares the
callback-based relationship traversal WhereRelated introduced; the cookbook
coverage figures now include the Traceability package's downlink requirement.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration
devin-ai-integration Bot changed the base branch from develop to feature/query-coverage-set-operations September 18, 2026 06:31
@devin-ai-integration
devin-ai-integration Bot added this pull request to stack #389 September 18, 2026 06:32
devin-ai-integration[bot]

This comment was marked as resolved.

The any aggregate is an existence test, yet it walked the whole frontier
and could exhaust the visit budget after its answer was already known. It
now shares hasRelated with WhereRelated and stops at the first element.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Base automatically changed from feature/query-coverage-set-operations to develop September 18, 2026 07:03
@HuiJun
HuiJun merged commit a045e92 into develop Sep 18, 2026
12 checks passed
@HuiJun
HuiJun deleted the feature/traceability-matrix-columns branch September 18, 2026 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant