[Theauxm]

Every state-machine engine is checked by one conformance kit and one runner

Background

A Trax machine is declared in C# and exported as an IR: one canonical JSON document holding its states,
triggers, context and input schemas, and each transition's guard and reducer as data. Two engines run it. The C#
engine (Trax.Effect.StateMachine) is the source of truth and runs on the server. The TypeScript engine
(@traxsharp/state-machine, the twin) interprets the IR in the browser and predicts what the server will
answer. They must agree byte for byte on the canonical wire (a snapshot's JSON in RFC 8785 form: keys sorted
by UTF-16 code unit, numbers and strings exactly as JSON.stringify writes them) and code for code on a
refusal (GUARD_FAILED, INVALID_INPUT, ...). Messages may differ.

Until now they agreed because both test suites read shared fixtures under Trax.Api.StateMachine/machines/, each
in its own way. The plan is to add engines and bindings in other languages, so the agreement has to be something
a third engine can join without reading either test suite.

Problem

🟪⬛ 1. A missing fixture folder passed CI
FixturePaths returned null when machines/ was not found, and about ten conformance tests then called
Assert.Ignore. That was left over from when the TS engine was a separate repository. Here the folder is always
present, so any copy or path filter that left it out reported green without comparing the engines.

🟪⬛ 2. The shared vectors were copied by hand, and had drifted
The canonical-wire vectors were inline in a C# test and a TS test. Only C# checked lone surrogates and the
\b \f \r escapes. The format verdicts (uuid, date-time, uri, ...) were a TS-only table beside the C#
strict-input cases, seven of them with no C# counterpart. The refusal codes were held equal by C# parsing
results.ts with a regex.

🟪⬛ 3. The IR had no machine-readable definition
The IR format page describes it in prose. In practice the exporter and the TS interpreter define it, so an engine
in another language could only be written by reading the TS code, and a field the exporter added without the doc
was found by a failing port.

🟪⬛ 4. There was no runner another engine could join
A new engine would have needed its own reader for every fixture shape (differential corpus, advance and
rehydrate fixtures, wire hand-off samples, strict inputs, wire vectors) as well as the engine itself.

Example

The canonical-wire vectors are now one file both engines read. A string holding a lone high surrogate cannot pass
through System.Text.Json, so the file gives it as UTF-16 code units:

{"name":"string: a lone high surrogate is escaped","stringUnits":[55296],"wire":"{\"s\":\"\\ud800\"}"}

Before, only the C# test had this vector. Now the TS engine checks it too (it already escaped it correctly).

The runner turns that vector, and every other case in the kit, into one request line for the engine under test,
the testee. It waits for one answer line back. A turnstile case from the differential corpus:

→ {"id":1,"op":"load","machine":"turnstile","ir":{…}}
← {"id":1,"loaded":true}
→ {"id":2,"op":"advance","machine":"turnstile","snapshot":{"machine":"turnstile","version":1,"state":"Locked","context":{}},"trigger":"Coin","input":{"coin":"penny"}}
← {"id":2,"outcome":"rejected","reason":"GUARD_FAILED"}

If the testee answered INVALID_INPUT instead, the run fails and names the case:

FAIL turnstile/advance/03-locked-coin-penny-guard-failed
  expected {"outcome":"rejected","reason":"GUARD_FAILED"}
  got      {"outcome":"rejected","reason":"INVALID_INPUT"}
416 of 417 cases passed.

The C# engine cannot build a machine from an IR, so its testee serves its test machines by name. On load it
checks that the committed IR is the one its machine exports. On the first run this found that the committed
checkout.ir.json carries a differential block (test-only fuzzing samples) that the C# checkout does not
declare. That block is stripped from a runtime machine and is not behaviour, so the check ignores it.

How this solves it

🟪⬛ The fixtures fail instead of skipping
FixturePaths.Find throws, naming every directory it searched, and the Assert.Ignore calls are gone. The node
checks keep their existing rule: skip on a developer machine, fail under CI.

🟪⬛ One folder of engine-neutral data: Trax.Api.StateMachine/conformance/

• canonical-wire.json: the union of both old tables.
• strict-input/: the strict machine's IR and every StrictInputMachine case, each with the C# engine's answer
  (which StrictInputParityTests already holds to the CLR type's verdict). StrictInputGoldenTests writes it
  with UPDATE_CONFORMANCE=1 and checks it otherwise. vitest replays it without .NET. The seven TS-only format
  rows moved into the C# cases, where the CLR decides their verdict.
• refusal-codes.json: the closed set, in order. Both engines check themselves against the file.
• ir.schema.json: the IR as JSON Schema (draft 2020-12), every object closed, normative per the IR format page.

🟪⬛ Both engines validate their IRs against the schema
vitest validates every committed IR and checks that the schema refuses an extra field, a misspelt rule member,
an unknown rule kind, and an output on a non-done outcome. IrSchemaTests exports every declarative C# test
machine and validates it through the same validator (tools/conformance/ir-schema.mjs, over node). A field the
exporter starts writing fails until the schema names it.

🟪⬛ One runner, one protocol, any engine
tools/conformance/run.mjs builds the cases from machines/ and conformance/ (kit.mjs, which does not
import the engine) and sends them to a testee over line-delimited JSON, documented in conformance/PROTOCOL.md.
--testee <command> spawns one. --stdio lets a test in another language start the runner and answer from
inside its own process, which is how the C# side runs without a separate executable.

flowchart LR
  M["machines/ IRs, corpora, fixtures"] --> K["kit.mjs: cases"]
  C["conformance/ vectors, strict inputs"] --> K
  K --> R["run.mjs"]
  R -- "--testee: spawn" --> TS["testee-ts.mjs over the package exports"]
  R -- "--stdio: spawned by" --> CS["ConformanceTestee in NUnit"]
  S["ir.schema.json"] --> V["ir-schema.mjs"]
  V --> TS2["vitest: committed IRs"]
  V --> CS2["IrSchemaTests: every exported IR"]
  class M,C,S machines
  class K,R,V,TS2,CS2 tooling
  class TS,CS machines
  classDef machines fill:#a855f7,stroke:#7e22ce,color:#fff
  classDef tooling fill:#334155,stroke:#0f172a,color:#fff

Loading

Both engines pass all 417 cases. The TS run is checked both in process and as a spawned testee, and a test flips
one answer to check that the runner fails and names the case.

⚠️ New dev dependency: ajv 8.18.0 in Trax.Api.StateMachine, for the schema validator. Not 8.17.1, which
has a ReDoS advisory ($data only). It ships in no package.

Architectural hinges

• ⬛ Trax.Api.StateMachine/conformance/PROTOCOL.md: the contract a new engine implements. Every op, what
  the runner sends from each source, and what it leaves to each engine (the v1→v2 migration golden, whose v2
  machine has no IR).
• ⬛ Trax.Api.StateMachine/tools/conformance/kit.mjs (kitCases, wireOf): turns every fixture shape into
  a request and its expected answer. It is engine-independent, so the expectations cannot come from the engine
  under test.
• ⬛ Trax.Api.StateMachine/tools/conformance/run.mjs (runCases): the comparison (canonical JSON of the
  answer against the expectation) and the two transports.
• 🟪 Trax.Effect/tests/Trax.Effect.StateMachine.Tests/Helpers/ConformanceTestee.cs: the C# side of the
  protocol, and the load check that ties each committed IR to its C# source.
• 🟪 Trax.Api.StateMachine/conformance/ir.schema.json: the IR's normative shape, which the exporter's output
  and every committed IR are validated against.

Also on this branch: the console timeline test waits for the server

🟦⬛ The console's live-timeline browser test released its run too early
A_running_runs_steps_are_pushed_to_its_timeline_as_they_end released the held run as soon as the page sent
its OnJunctionEvent subscribe frame. graphql-ws acknowledges no single subscription, so the server could still
be registering it when the junction completed. That event reached nobody and the test timed out, three times on
2026-10-20 on branches that touch neither the console nor subscriptions, this one included. The test now
publishes a probe junction event for the run until the page receives it (PageChecks.ListeningAsync, the same
approach as the Api's JunctionEventsE2ETests.WaitUntilListeningAsync), and releases the run only then.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
trax-website	Ignored	Preview	Oct 20, 2026 3:39pm UTC

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[github-actions]

This PR is included in version 1.68.0