From bf16f29e02140544104a0f0f63fa9bcf443d6b03 Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Wed, 26 Aug 2026 15:34:03 +0000 Subject: [PATCH 01/15] Compile snippets against the real SQL Race assemblies, not just parse them SnippetCompilationTests only checked that each snippet parsed as valid C#. That catches a missing brace and little else: a snippet calling a method the API does not have parses perfectly and still fails to build for anyone who copies it. It now builds a Roslyn compilation per snippet against the actual MESL.SQLRace.API assemblies, referencing both TRUSTED_PLATFORM_ASSEMBLIES and this assembly's own output directory so the same code works on a developer machine and on a CI runner. Compilation is metadata-only, so no SQL Race licence is involved and the suite runs on a plain GitHub runner, which makes it the one suite CI can execute. SqlRaceApiVersion parameterises the package reference, so the snippets can be compiled against a chosen build rather than whatever restores: dotnet test -p:SqlRaceApiVersion= CI runs the suite advisory-only to begin with, while the failures it reports are worked through. Co-Authored-By: Claude Opus 5 --- .github/workflows/build-and-test.yml | 36 ++++- .../SnippetCompilationTests.cs | 145 ++++++++++++++++-- .../SqlRace.Examples.Tests.csproj | 18 ++- 3 files changed, 180 insertions(+), 19 deletions(-) diff --git a/.github/workflows/build-and-test.yml b/.github/workflows/build-and-test.yml index 06d1495..a33ab5e 100644 --- a/.github/workflows/build-and-test.yml +++ b/.github/workflows/build-and-test.yml @@ -18,16 +18,40 @@ jobs: with: dotnet-version: 8.0.x - - name: Restore solution + - name: Restore projects run: dotnet restore projects/csharp/SqlRace.Examples.sln env: NUGET_AUTH_USERNAME: ${{ secrets.PACKAGES_USERNAME }} NUGET_AUTH_TOKEN: ${{ secrets.PACKAGES_PATTOKEN }} - - name: Build solution + - name: Build projects run: dotnet build projects/csharp/SqlRace.Examples.sln --no-restore --configuration Release - # NOTE: Tests are NOT executed in CI. Several tests initialise the SQL Race - # runtime via SqlRaceBootstrapper, which requires a SQL Race licence that the - # CI runner does not have (LicenceException). CI verifies compilation only; - # run `dotnet test` locally on a licenced machine to execute the test suite. + - name: Restore tests + run: dotnet restore tests/SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj + env: + NUGET_AUTH_USERNAME: ${{ secrets.PACKAGES_USERNAME }} + NUGET_AUTH_TOKEN: ${{ secrets.PACKAGES_PATTOKEN }} + + - name: Build tests + run: dotnet build tests/SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj --no-restore --configuration Release + + # NOTE: MOST tests are not executed in CI. Several of them initialise the SQL + # Race runtime via SqlRaceBootstrapper, which requires a SQL Race licence that + # the CI runner does not have (LicenceException). Run `dotnet test` locally on + # a licenced machine to execute the full suite, or scripts/run-examples-e2e.ps1 + # for the runtime checks across C#, Python and MATLAB. + # + # The snippet compilation tests are the exception: they compile every snippet + # against the real SQL Race assemblies using Roslyn, which is a metadata-only + # operation needing no licence. They are the early warning that a new + # MESL.SQLRace.API release has broken the code we hand customers. + - name: Verify snippets compile against SQL Race + # Advisory for now - three function snippets have a known pre-existing + # failure. Remove continue-on-error once those are fixed so that a genuine + # API break fails the build. + continue-on-error: true + run: > + dotnet test tests/SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj + --no-build --configuration Release + --filter "FullyQualifiedName~SnippetCompilation" diff --git a/tests/SqlRace.Examples.Tests/SnippetCompilationTests.cs b/tests/SqlRace.Examples.Tests/SnippetCompilationTests.cs index bb4bce4..ab769f3 100644 --- a/tests/SqlRace.Examples.Tests/SnippetCompilationTests.cs +++ b/tests/SqlRace.Examples.Tests/SnippetCompilationTests.cs @@ -3,11 +3,22 @@ // ───────────────────────────────────────────────────────────── // // Dynamically discovers all .cs files under snippets/csharp/ and -// verifies they parse as valid C# syntax using Roslyn. +// compiles each one against the real SQL Race assemblies using +// Roslyn. // -// This catches syntax errors and missing brackets but does NOT -// verify against the real SQL Race assemblies (that requires the -// NuGet package to be resolvable at test time). +// This is the early-warning system for API breaks. Snippets are +// the code we hand customers, so a snippet that stops compiling +// against a new MESL.SQLRace.API release is a customer whose +// script stops compiling too - and we want to know before the +// release ships, not after. +// +// Compilation is metadata-only: nothing here initialises the SQL +// Race runtime, so no licence is needed and this suite runs on a +// plain GitHub runner. Runtime behaviour of the same snippets is +// covered by scripts/run-examples-e2e.ps1, which does need one. +// +// To test a specific API build: +// dotnet test -p:SqlRaceApiVersion=2.1.26212.6-ci // // Run: dotnet test --filter "FullyQualifiedName~SnippetCompilation" // ───────────────────────────────────────────────────────────── @@ -40,19 +51,131 @@ public static IEnumerable SnippetFiles() } } + // ── Reference set ───────────────────────────────────────── + // + // Two sources, both derived from this test assembly rather + // than from a hard-coded SDK path, so the same code works on + // a developer machine and on a CI runner: + // + // 1. TRUSTED_PLATFORM_ASSEMBLIES - every framework assembly + // the host runtime can load. + // 2. This assembly's own output directory - the SQL Race + // assemblies and their transitive dependencies, put + // there by the MESL.SQLRace.API PackageReference. + // + // Built once: resolving ~200 assemblies per snippet would + // dominate the runtime of the suite. + private static readonly Lazy> References = + new(BuildReferences); + + private static IReadOnlyList BuildReferences() + { + // Keyed by simple name so a framework assembly and a copy + // of the same assembly in the output folder do not both + // get referenced - Roslyn reports that as an ambiguous + // type reference rather than resolving it. + var byName = new Dictionary(StringComparer.OrdinalIgnoreCase); + + var platformAssemblies = (string?)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES"); + if (platformAssemblies is not null) + { + foreach (var path in platformAssemblies.Split(Path.PathSeparator)) + { + if (path.EndsWith(".dll", StringComparison.OrdinalIgnoreCase) && File.Exists(path)) + byName[Path.GetFileNameWithoutExtension(path)] = path; + } + } + + // The output folder wins on conflict: it holds the SQL + // Race build under test, which is the point of all this. + foreach (var path in Directory.EnumerateFiles(AppContext.BaseDirectory, "*.dll")) + byName[Path.GetFileNameWithoutExtension(path)] = path; + + var references = new List(); + foreach (var path in byName.Values) + { + // Native and resource-only DLLs sit alongside managed + // ones in the output folder and throw when read as + // metadata. + try { references.Add(MetadataReference.CreateFromFile(path)); } + catch (BadImageFormatException) { } + catch (IOException) { } + } + + return references; + } + + // Snippets are top-level-statement programs built with + // ImplicitUsings enabled, so the usings the SDK would inject + // have to be supplied here - otherwise every snippet fails on + // Console, Path and the LINQ extension methods. + private const string ImplicitUsings = + "global using global::System;\n" + + "global using global::System.Collections.Generic;\n" + + "global using global::System.IO;\n" + + "global using global::System.Linq;\n" + + "global using global::System.Net.Http;\n" + + "global using global::System.Threading;\n" + + "global using global::System.Threading.Tasks;\n"; + [Theory] [MemberData(nameof(SnippetFiles))] - public void Snippet_ParsesWithoutSyntaxErrors(string relativePath, string fullPath) + public void Snippet_CompilesAgainstSqlRaceAssemblies(string relativePath, string fullPath) { - var source = File.ReadAllText(fullPath); - var syntaxTree = CSharpSyntaxTree.ParseText(source, new CSharpParseOptions(LanguageVersion.CSharp12)); - var diagnostics = syntaxTree.GetDiagnostics() + var parseOptions = new CSharpParseOptions(LanguageVersion.CSharp12); + + var trees = new[] + { + CSharpSyntaxTree.ParseText(File.ReadAllText(fullPath), parseOptions, path: fullPath), + CSharpSyntaxTree.ParseText(ImplicitUsings, parseOptions, path: "ImplicitUsings.g.cs"), + }; + + // ConsoleApplication because top-level statements need an + // entry point. Each snippet compiles alone - only one file + // in a compilation may carry top-level statements. + var compilation = CSharpCompilation.Create( + assemblyName: "SnippetCompilation_" + Guid.NewGuid().ToString("N"), + syntaxTrees: trees, + references: References.Value, + options: new CSharpCompilationOptions( + OutputKind.ConsoleApplication, + nullableContextOptions: NullableContextOptions.Enable)); + + var errors = compilation.GetDiagnostics() .Where(d => d.Severity == DiagnosticSeverity.Error) .ToList(); - Assert.True(diagnostics.Count == 0, - $"Syntax errors in {relativePath}:\n" + - string.Join("\n", diagnostics.Select(d => $" Line {d.Location.GetLineSpan().StartLinePosition.Line + 1}: {d.GetMessage()}"))); + Assert.True(errors.Count == 0, + $"{relativePath} no longer compiles against {DescribeSqlRaceVersion()}. " + + "A customer copying this snippet would hit the same errors:\n" + + string.Join("\n", errors.Select(d => + $" Line {d.Location.GetLineSpan().StartLinePosition.Line + 1}: {d.Id}: {d.GetMessage()}"))); + } + + // Named in failure messages so a red build says which API + // build broke the snippet, not just that something broke. + private static string DescribeSqlRaceVersion() + { + var domain = Path.Combine(AppContext.BaseDirectory, "MESL.SqlRace.Domain.dll"); + if (!File.Exists(domain)) + return "MESL.SQLRace.API (version unknown - assembly not resolved)"; + + var info = System.Diagnostics.FileVersionInfo.GetVersionInfo(domain); + return $"MESL.SQLRace.API {info.FileVersion}"; + } + + [Fact] + public void SqlRaceAssemblies_AreResolvable() + { + // Guards against the compilation tests passing vacuously. + // If the package failed to restore there would be nothing + // to compile against, and a weaker suite could report that + // as success. + var domain = Path.Combine(AppContext.BaseDirectory, "MESL.SqlRace.Domain.dll"); + Assert.True(File.Exists(domain), + $"MESL.SqlRace.Domain.dll was not found in {AppContext.BaseDirectory}. " + + "The MESL.SQLRace.API package did not restore, so snippet compilation " + + "would not be testing anything."); } [Fact] diff --git a/tests/SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj b/tests/SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj index 1a5c5cc..bed8e51 100644 --- a/tests/SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj +++ b/tests/SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj @@ -6,14 +6,28 @@ enable false true + + + * - - + + From 37fde68ad3bc23eab6ec2d73f632293916d1c0ec Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Wed, 26 Aug 2026 15:34:20 +0000 Subject: [PATCH 02/15] Run C#, Python and MATLAB snippets against golden output Compiling a snippet proves it still builds. It does not prove it still returns the right numbers, and it says nothing at all about the Python and MATLAB snippets, where a rename or a changed signature fails silently at runtime for pythonnet and NET.addAssembly callers. run-examples-e2e.ps1 executes the runnable snippets in all three languages against the deterministic fixture and diffs their stdout against tests/golden//.txt. Values that legitimately vary between runs - GUIDs, timestamps, absolute paths, elapsed times - are normalised first, so what is asserted is the data: parameter names, sample counts, computed values. Which snippets run, their arguments, and why the rest are skipped now live in scripts/snippet-manifest.psd1 rather than inline in the script, so adding a snippet is a data change and the skips stay in one visible place. A snippet that runs with no golden file reports NOGOLD rather than passing, and skips are listed in the summary - a snippet silently missing from the run would otherwise look identical to one that passed. SnippetRunner gained a SnippetFile property that swaps the Compile item instead of copying each snippet over Program.cs, which used to leave the working tree dirty for the duration of a run and lost the original if the run was interrupted. Co-Authored-By: Claude Opus 5 --- scripts/run-examples-e2e.ps1 | 436 +++++++++++++++--- scripts/snippet-manifest.psd1 | 95 ++++ tests/README.md | 80 +++- .../CSharp/01-load-session-from-file.txt | 13 + .../CSharp/03-read-parameter-samples.txt | 25 + .../CSharp/04-create-session-write-data.txt | 6 + .../CSharp/05-query-sessions-by-metadata.txt | 6 + .../CSharp/append-multiple-sessions.txt | 16 + tests/golden/CSharp/bulk-parameter-read.txt | 7 + .../CSharp/composite-with-associates.txt | 15 + tests/golden/CSharp/conversion-types.txt | 9 + .../golden/CSharp/create-parameter-config.txt | 6 + .../CSharp/create-transient-parameter.txt | 4 + tests/golden/CSharp/data-status-filtering.txt | 16 + .../CSharp/introspect-session-config.txt | 14 + .../CSharp/lap-and-segment-management.txt | 8 + tests/golden/CSharp/marker-management.txt | 10 + tests/golden/CSharp/multi-rate-alignment.txt | 19 + .../CSharp/parameter-unit-resolution.txt | 6 + .../CSharp/query-with-composite-filter.txt | 4 + tests/golden/CSharp/read-samples-between.txt | 14 + tests/golden/CSharp/read-subsampled-data.txt | 19 + tests/golden/CSharp/reverse-iteration.txt | 14 + tests/golden/CSharp/session-association.txt | 6 + tests/golden/CSharp/session-metadata-crud.txt | 9 + .../CSharp/timestamp-array-construction.txt | 24 + tests/golden/CSharp/whole-session-compare.txt | 18 + .../Matlab/create_session_write_data.txt | 7 + tests/golden/Matlab/data_status_filtering.txt | 17 + tests/golden/Matlab/diagnose_setup.txt | 7 + tests/golden/Matlab/event_extraction.txt | 14 + tests/golden/Matlab/extract_to_timetable.txt | 16 + .../Matlab/introspect_session_config.txt | 16 + .../golden/Matlab/lap_segment_management.txt | 9 + tests/golden/Matlab/list_parameters.txt | 9 + tests/golden/Matlab/load_session.txt | 9 + tests/golden/Matlab/marker_management.txt | 11 + tests/golden/Matlab/multi_channel_read.txt | 12 + .../Matlab/parameter_unit_resolution.txt | 7 + tests/golden/Matlab/plot_parameters.txt | 5 + tests/golden/Matlab/query_sessions.txt | 5 + tests/golden/Matlab/read_function_output.txt | 7 + tests/golden/Matlab/read_parameters.txt | 5 + tests/golden/Matlab/read_samples_between.txt | 15 + tests/golden/Matlab/reverse_iteration.txt | 15 + tests/golden/Matlab/segment_statistics.txt | 4 + tests/golden/Matlab/session_metadata.txt | 10 + tests/golden/Matlab/subsampled_read.txt | 20 + tests/golden/Python/01_load_session.txt | 9 + tests/golden/Python/02_read_parameters.txt | 25 + tests/golden/Python/03_query_sessions.txt | 4 + tests/golden/Python/export_to_pandas.txt | 22 + tests/golden/Python/read_samples_to_list.txt | 7 + tools/SnippetRunner/SnippetRunner.csproj | 30 +- 54 files changed, 1153 insertions(+), 63 deletions(-) create mode 100644 scripts/snippet-manifest.psd1 create mode 100644 tests/golden/CSharp/01-load-session-from-file.txt create mode 100644 tests/golden/CSharp/03-read-parameter-samples.txt create mode 100644 tests/golden/CSharp/04-create-session-write-data.txt create mode 100644 tests/golden/CSharp/05-query-sessions-by-metadata.txt create mode 100644 tests/golden/CSharp/append-multiple-sessions.txt create mode 100644 tests/golden/CSharp/bulk-parameter-read.txt create mode 100644 tests/golden/CSharp/composite-with-associates.txt create mode 100644 tests/golden/CSharp/conversion-types.txt create mode 100644 tests/golden/CSharp/create-parameter-config.txt create mode 100644 tests/golden/CSharp/create-transient-parameter.txt create mode 100644 tests/golden/CSharp/data-status-filtering.txt create mode 100644 tests/golden/CSharp/introspect-session-config.txt create mode 100644 tests/golden/CSharp/lap-and-segment-management.txt create mode 100644 tests/golden/CSharp/marker-management.txt create mode 100644 tests/golden/CSharp/multi-rate-alignment.txt create mode 100644 tests/golden/CSharp/parameter-unit-resolution.txt create mode 100644 tests/golden/CSharp/query-with-composite-filter.txt create mode 100644 tests/golden/CSharp/read-samples-between.txt create mode 100644 tests/golden/CSharp/read-subsampled-data.txt create mode 100644 tests/golden/CSharp/reverse-iteration.txt create mode 100644 tests/golden/CSharp/session-association.txt create mode 100644 tests/golden/CSharp/session-metadata-crud.txt create mode 100644 tests/golden/CSharp/timestamp-array-construction.txt create mode 100644 tests/golden/CSharp/whole-session-compare.txt create mode 100644 tests/golden/Matlab/create_session_write_data.txt create mode 100644 tests/golden/Matlab/data_status_filtering.txt create mode 100644 tests/golden/Matlab/diagnose_setup.txt create mode 100644 tests/golden/Matlab/event_extraction.txt create mode 100644 tests/golden/Matlab/extract_to_timetable.txt create mode 100644 tests/golden/Matlab/introspect_session_config.txt create mode 100644 tests/golden/Matlab/lap_segment_management.txt create mode 100644 tests/golden/Matlab/list_parameters.txt create mode 100644 tests/golden/Matlab/load_session.txt create mode 100644 tests/golden/Matlab/marker_management.txt create mode 100644 tests/golden/Matlab/multi_channel_read.txt create mode 100644 tests/golden/Matlab/parameter_unit_resolution.txt create mode 100644 tests/golden/Matlab/plot_parameters.txt create mode 100644 tests/golden/Matlab/query_sessions.txt create mode 100644 tests/golden/Matlab/read_function_output.txt create mode 100644 tests/golden/Matlab/read_parameters.txt create mode 100644 tests/golden/Matlab/read_samples_between.txt create mode 100644 tests/golden/Matlab/reverse_iteration.txt create mode 100644 tests/golden/Matlab/segment_statistics.txt create mode 100644 tests/golden/Matlab/session_metadata.txt create mode 100644 tests/golden/Matlab/subsampled_read.txt create mode 100644 tests/golden/Python/01_load_session.txt create mode 100644 tests/golden/Python/02_read_parameters.txt create mode 100644 tests/golden/Python/03_query_sessions.txt create mode 100644 tests/golden/Python/export_to_pandas.txt create mode 100644 tests/golden/Python/read_samples_to_list.txt diff --git a/scripts/run-examples-e2e.ps1 b/scripts/run-examples-e2e.ps1 index 1115374..af649e5 100644 --- a/scripts/run-examples-e2e.ps1 +++ b/scripts/run-examples-e2e.ps1 @@ -1,73 +1,395 @@ # ───────────────────────────────────────────────────────────── -# End-to-end snippet smoke-runner. +# End-to-end snippet runner: C#, Python and MATLAB. # -# Generates the deterministic verified fixture (tools/FixtureGenerator) at the default -# SQLite path the snippets read from, then runs each runnable snippet against it and -# checks it exits cleanly and prints the expected anchor text. +# Generates the deterministic fixture (tools/FixtureGenerator), runs every +# runnable snippet from scripts/snippet-manifest.psd1 against it, and compares +# each snippet's output against a checked-in golden file under tests/golden/. # -# Requires the SQL Race runtime + licence — local / licenced-runner only, NOT GitHub CI. -# Usage: pwsh scripts/run-examples-e2e.ps1 +# Why golden files rather than "did it exit 0": an API change can leave a +# snippet compiling and exiting cleanly while quietly returning different data. +# Comparing normalised output catches that. It also means adding a snippet costs +# a manifest line rather than a hand-authored assertion. +# +# Python and MATLAB matter disproportionately here. They bind to SQL Race by +# name at runtime, so a rename that a C# compiler would catch fails silently for +# them - running them is the only way to find out. +# +# Requires the SQL Race runtime + licence - local / licenced-runner only, NOT +# GitHub CI. The licence-free compile check is in tests/SqlRace.Examples.Tests +# (SnippetCompilationTests). +# +# Usage: +# ./scripts/run-examples-e2e.ps1 +# ./scripts/run-examples-e2e.ps1 -Language CSharp +# ./scripts/run-examples-e2e.ps1 -Filter '*read*' +# ./scripts/run-examples-e2e.ps1 -SqlRaceApiVersion 2.1.26212.6-ci +# ./scripts/run-examples-e2e.ps1 -UpdateGolden # after an intended change # ───────────────────────────────────────────────────────────── -$ErrorActionPreference = "Stop" +[CmdletBinding()] +param ( + [ValidateSet('All', 'CSharp', 'Python', 'Matlab')] + [string] $Language = 'All', + + # Wildcard over snippet names, for iterating on one snippet. + [string] $Filter = '*', + + # Rewrite golden files from this run's output instead of comparing. Review + # the resulting diff - it will happily bake in a regression. + [switch] $UpdateGolden, + + # Pin the MESL.SQLRace.API version to validate a candidate build before it + # ships. Empty means whatever the projects resolve normally. + [string] $SqlRaceApiVersion, + + [switch] $SkipBuild +) + +$ErrorActionPreference = 'Stop' $repo = Split-Path $PSScriptRoot -Parent Push-Location $repo + try { - $fixtureKey = "f1c0ffee-0000-4000-8000-000000000001" # VerifiedFixture.SessionKeyString - $fixturePath = Join-Path $env:TEMP "sqlrace-examples.ssn2" # default path snippets read - - Write-Host "Building generator + snippet runner..." -ForegroundColor Cyan - dotnet build tools/FixtureGenerator/FixtureGenerator.csproj -c Release -v quiet | Out-Null - if (-not $?) { throw "FixtureGenerator build failed" } - - Write-Host "Generating verified fixture at $fixturePath" -ForegroundColor Cyan - & "tools/FixtureGenerator/bin/Release/net8.0/FixtureGenerator.exe" $fixturePath | Out-Null - if (-not $?) { throw "Fixture generation failed" } - - # Each snippet: args to pass, and an expected substring proving it ran against the fixture. - $snippets = @( - @{ Name = "01-load-session-from-file"; Path = "snippets\csharp\getting-started\01-load-session-from-file.cs"; Args = @($fixturePath); Expect = "Verified Test Session" }, - @{ Name = "03-read-parameter-samples"; Path = "snippets\csharp\getting-started\03-read-parameter-samples.cs"; Args = @($fixtureKey); Expect = "1000" }, - @{ Name = "read-samples-between"; Path = "snippets\csharp\data-access\read-samples-between.cs"; Args = @($fixtureKey); Expect = "Temperature:Sensors" }, - @{ Name = "bulk-parameter-read"; Path = "snippets\csharp\data-access\bulk-parameter-read.cs"; Args = @($fixtureKey); Expect = $null }, - @{ Name = "multi-rate-alignment"; Path = "snippets\csharp\data-access\multi-rate-alignment.cs"; Args = @($fixtureKey); Expect = $null }, - @{ Name = "reverse-iteration"; Path = "snippets\csharp\data-access\reverse-iteration.cs"; Args = @($fixtureKey); Expect = $null }, - @{ Name = "parameter-unit-resolution"; Path = "snippets\csharp\session-management\parameter-unit-resolution.cs"; Args = @($fixtureKey); Expect = "Temperature:Sensors" } - ) - - # Preserve the SnippetRunner program (the runner mechanism overwrites it). - $runnerProgram = "tools\SnippetRunner\Program.cs" - $backup = Get-Content $runnerProgram -Raw + $fixtureKey = 'f1c0ffee-0000-4000-8000-000000000001' # VerifiedFixture.SessionKeyString + $fixturePath = Join-Path $env:TEMP 'sqlrace-examples.ssn2' + $goldenRoot = Join-Path $repo 'tests\golden' + $manifest = Import-PowerShellDataFile (Join-Path $PSScriptRoot 'snippet-manifest.psd1') + + $buildArgs = @('-c', 'Release', '-v', 'quiet') + if ($SqlRaceApiVersion) { $buildArgs += "-p:SqlRaceApiVersion=$SqlRaceApiVersion" } + + # ── Normalisation ──────────────────────────────────────── + # + # Snippets print values that legitimately differ between runs. Without this + # every golden comparison would fail on the first newly generated GUID. + function ConvertTo-ComparableOutput { + param ([string] $Text) + + if (-not $Text) { return '' } + + $normalised = $Text -replace "`r`n", "`n" + + # Volatile values, most specific pattern first. + $normalised = $normalised -replace '[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}', '' + $normalised = $normalised -replace '\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}(\.\d+)?', '' + $normalised = $normalised -replace '\d{2}/\d{2}/\d{4} \d{2}:\d{2}:\d{2}', '' + # Windows paths, including the per-user temp directory the fixture lives in. + $normalised = $normalised -replace '[A-Za-z]:\\[^\s"'']*', '' + # Elapsed times and sizes: "12.345 s", "1234 ms", "4.2 MB". + $normalised = $normalised -replace '\d+(\.\d+)?\s*(ms|s\b|KB|MB|GB)', '' + + # Trailing whitespace and blank lines at either end are noise. + $lines = $normalised -split "`n" | ForEach-Object { $_.TrimEnd() } + return (($lines -join "`n").Trim()) + } + + # Build noise that says nothing about the snippet's behaviour. + $noisePattern = 'warning|NU1701|Restored|Determining|reachable on all platforms|' + + 'Build succeeded|Time Elapsed|^\s*\d+ Warning|^\s*\d+ Error' + + function Remove-BuildNoise { + param ([string[]] $Lines) + return $Lines | Where-Object { $_ -notmatch $noisePattern -and $_.Trim() -ne '' } + } + + # ── Fixture ────────────────────────────────────────────── + if (-not $SkipBuild) { + Write-Host 'Building fixture generator...' -ForegroundColor Cyan + & dotnet build tools/FixtureGenerator/FixtureGenerator.csproj @buildArgs | Out-Null + if ($LASTEXITCODE -ne 0) { throw 'FixtureGenerator build failed' } + } + + # Regenerated before every snippet, not once per run. Several snippets create + # their own sessions in the same SQLite file the read-only snippets query, so + # a single fixture would accumulate state: 01-load-session-from-file reports + # "Sessions found: 1" on its own and "Sessions found: 10" after the session + # creating snippets have run. Golden output has to be independent of the + # order snippets execute in, and of whether a previous run left anything + # behind. + function Reset-Fixture { + if (Test-Path $fixturePath) { Remove-Item $fixturePath -Force } + & 'tools/FixtureGenerator/bin/Release/net8.0/FixtureGenerator.exe' $fixturePath | Out-Null + if ($LASTEXITCODE -ne 0) { throw 'Fixture generation failed' } + } + + Write-Host "Verified fixture at $fixturePath" -ForegroundColor Cyan + Reset-Fixture + + # ── C# runner project ──────────────────────────────────── + # + # tools/SnippetRunner compiles whichever file its SnippetFile property points + # at, so a snippet runs without anything in the working tree changing. The + # previous version of this script copied each snippet over Program.cs and + # restored it in a finally block, which left the repo dirty for the duration + # and lost the original if the run was interrupted. + $runnerProject = 'tools\SnippetRunner\SnippetRunner.csproj' + + function Initialize-SnippetRunner { + # Build once with the default Program.cs: this resolves the package and + # populates the output folder that SQLRACE_DLL_PATH points at, so all + # three languages exercise the same assemblies. + Write-Host 'Preparing snippet runner...' -ForegroundColor Cyan + & dotnet build $runnerProject @buildArgs | Out-Null + if ($LASTEXITCODE -ne 0) { + throw ("SnippetRunner build failed. If this is a NuGet 401, MESL.SQLRace.API is " + + "referenced as Version=`"*`" and NuGet cannot resolve a floating version " + + "without reaching the feed: set NUGET_AUTH_USERNAME and NUGET_AUTH_TOKEN " + + "(see .env.example), or pass -SqlRaceApiVersion to pin a version already " + + "in the local NuGet cache.") + } + + $domain = Join-Path $repo 'tools\SnippetRunner\bin\Release\net8.0\MESL.SqlRace.Domain.dll' + if (-not (Test-Path $domain)) { throw "MESL.SqlRace.Domain.dll not found at $domain" } + return $domain + } + + function Invoke-CSharpSnippet { + param ([hashtable] $Snippet, [string[]] $Arguments) + + # SnippetFile must be absolute - MSBuild resolves relative Compile items + # against the project directory, not the working directory. + $snippetFile = (Resolve-Path $Snippet.Path).Path + + $output = & dotnet run --project $runnerProject @buildArgs ` + "-p:SnippetFile=$snippetFile" -- @Arguments 2>&1 | + ForEach-Object { $_.ToString() } + return [pscustomobject]@{ ExitCode = $LASTEXITCODE; Lines = $output } + } + + # A NuGet-restored build keeps platform-specific assets under runtimes// + # and relies on the app's deps.json to pick the right ones. Python and MATLAB + # host the CLR themselves, so there is no deps.json for SQL Race: the loader + # falls back to the flat directory, finds the placeholder System.Data.SqlClient + # ("not supported on this platform") and cannot find SQLite.Interop.dll at all. + # + # Flattening the Windows assets over a copy of the output gives those hosts the + # same layout an ATLAS installation has. This is not only a harness concern - a + # customer pointing pythonnet or MATLAB at a NuGet-restored SQL Race hits + # exactly this, while one pointing at an ATLAS install does not. + function New-FlatRuntimeDirectory { + param ([Parameter(Mandatory)][string] $BuildOutputDir) + + $flatDir = Join-Path $env:TEMP 'sqlrace-flat-runtime' + if (Test-Path $flatDir) { Remove-Item $flatDir -Recurse -Force } + New-Item -ItemType Directory -Path $flatDir -Force | Out-Null + + Copy-Item (Join-Path $BuildOutputDir '*') $flatDir -Recurse -Force + + # Overlay Windows assets last so they win over the cross-platform + # placeholders sitting in the output root. + $overlays = @( + 'runtimes\win\lib\netstandard2.0' + 'runtimes\win\lib\netcoreapp2.0' + 'runtimes\win\lib\netcoreapp2.1' + 'runtimes\win\lib\netcoreapp3.0' + 'runtimes\win-x64\native' + ) + foreach ($overlay in $overlays) { + $source = Join-Path $BuildOutputDir $overlay + if (Test-Path $source) { + Copy-Item (Join-Path $source '*') $flatDir -Force -ErrorAction SilentlyContinue + } + } + + return $flatDir + } + + function Invoke-PythonSnippet { + param ([hashtable] $Snippet, [string[]] $Arguments) + + # Python defaults stdout to the console codepage on Windows (cp1252), + # and the snippets print box-drawing characters in their table rules - + # without this they die with UnicodeEncodeError before reaching any SQL + # Race call, which would look like an API failure but is not one. + $previousEncoding = $env:PYTHONIOENCODING + $env:PYTHONIOENCODING = 'utf-8' + try { + $output = & python $Snippet.Path @Arguments 2>&1 | ForEach-Object { $_.ToString() } + return [pscustomobject]@{ ExitCode = $LASTEXITCODE; Lines = $output } + } + finally { + $env:PYTHONIOENCODING = $previousEncoding + } + } + + function Invoke-MatlabSnippet { + param ([hashtable] $Snippet, [string[]] $Arguments) + + # -batch runs headless, returns a non-zero exit code on an uncaught + # error, and needs an absolute path since it does not inherit our cwd + # semantics for run(). + $full = (Resolve-Path $Snippet.Path).Path + $output = & matlab -batch "run('$full')" 2>&1 | ForEach-Object { $_.ToString() } + return [pscustomobject]@{ ExitCode = $LASTEXITCODE; Lines = $output } + } + + $runners = @{ + CSharp = ${function:Invoke-CSharpSnippet} + Python = ${function:Invoke-PythonSnippet} + Matlab = ${function:Invoke-MatlabSnippet} + } + + $languages = if ($Language -eq 'All') { @('CSharp', 'Python', 'Matlab') } else { @($Language) } + + # Only prepare the C# runner if we need it - it is also what supplies + # SQLRACE_DLL_PATH for the other two languages. + $sqlRaceDllPath = Initialize-SnippetRunner + + # The C# leg runs through dotnet, which resolves assets from deps.json. Python + # and MATLAB need the flattened layout instead - but the same build either way, + # so all three languages exercise identical assemblies. + $flatRuntimeDir = New-FlatRuntimeDirectory -BuildOutputDir (Split-Path $sqlRaceDllPath -Parent) + $env:SQLRACE_DLL_PATH = Join-Path $flatRuntimeDir 'MESL.SqlRace.Domain.dll' + $env:PATH = "$flatRuntimeDir;$env:PATH" + Write-Host "SQLRACE_DLL_PATH = $($env:SQLRACE_DLL_PATH)" -ForegroundColor DarkGray $results = @() - try { - foreach ($s in $snippets) { - Write-Host "`n===== $($s.Name) =====" -ForegroundColor Cyan - Copy-Item $s.Path $runnerProgram -Force - (Get-Item $runnerProgram).LastWriteTime = Get-Date - - $out = dotnet run --project tools\SnippetRunner\SnippetRunner.csproj -c Release -- @($s.Args) 2>&1 | - Where-Object { $_ -notmatch "warning|NU1701|Restored|Determining|reachable on all platforms" -and $_ -ne "" } - $exit = $LASTEXITCODE - $text = ($out | Out-String) - - $ok = ($exit -eq 0) -and ($text -notmatch "ERROR:|Usage:|not found|No sessions") - if ($ok -and $s.Expect) { $ok = $text -match [regex]::Escape($s.Expect) } - - $status = if ($ok) { "PASS" } else { "FAIL" } - Write-Host "[$status] exit=$exit" -ForegroundColor $(if ($ok) { "Green" } else { "Red" }) - $out | Select-Object -First 4 | ForEach-Object { Write-Host " $_" } - $results += [pscustomobject]@{ Snippet = $s.Name; Status = $status } + + foreach ($lang in $languages) { + $snippets = @($manifest[$lang] | Where-Object { $_.Name -like $Filter }) + if ($snippets.Count -eq 0) { continue } + + Write-Host "" + Write-Host "########## $lang ##########" -ForegroundColor Yellow + + $goldenDir = Join-Path $goldenRoot $lang + if ($UpdateGolden -and -not (Test-Path $goldenDir)) { + New-Item -ItemType Directory -Path $goldenDir -Force | Out-Null + } + + foreach ($snippet in $snippets) { + if ($snippet.Skip) { + Write-Host ("[SKIP] {0} - {1}" -f $snippet.Name, $snippet.Skip) -ForegroundColor DarkGray + $results += [pscustomobject]@{ + Language = $lang; Snippet = $snippet.Name; Status = 'SKIP'; Detail = $snippet.Skip + } + continue + } + + $arguments = @() + foreach ($argument in @($snippet.Args)) { + if ($null -eq $argument) { continue } + $arguments += $argument.Replace('{fixtureKey}', $fixtureKey).Replace('{fixturePath}', $fixturePath) + } + + Write-Host ("`n===== {0}/{1} =====" -f $lang, $snippet.Name) -ForegroundColor Cyan + + Reset-Fixture + + # Under Windows PowerShell 5.1, redirecting a native command's stderr + # turns each line into an ErrorRecord, which $ErrorActionPreference + # = 'Stop' promotes to a terminating error. A Python traceback or a + # MATLAB warning would then abort the run instead of being captured + # as the snippet's output. $ErrorActionPreference is dynamically + # scoped, so relaxing it here covers the runner functions too. + $previousPreference = $ErrorActionPreference + $ErrorActionPreference = 'Continue' + try { + $run = & $runners[$lang] $snippet $arguments + } + catch { + Write-Host "[FAIL] runner threw: $($_.Exception.Message)" -ForegroundColor Red + $results += [pscustomobject]@{ + Language = $lang; Snippet = $snippet.Name; Status = 'FAIL'; Detail = $_.Exception.Message + } + continue + } + finally { + $ErrorActionPreference = $previousPreference + } + + $meaningful = Remove-BuildNoise -Lines $run.Lines + $actual = ConvertTo-ComparableOutput (($meaningful | Out-String)) + + $status = 'PASS' + $detail = '' + + if ($run.ExitCode -ne 0) { + $status = 'FAIL' + $detail = "exit code $($run.ExitCode)" + } + elseif ($actual -match 'ERROR:|Unhandled exception|Traceback \(most recent call last\)') { + $status = 'FAIL' + $detail = 'snippet reported an error' + } + else { + $goldenPath = Join-Path $goldenDir "$($snippet.Name).txt" + + if ($UpdateGolden) { + $actual | Set-Content $goldenPath -Encoding utf8 + $status = 'GOLDEN' + $detail = 'written' + } + elseif (-not (Test-Path $goldenPath)) { + # Not a pass: an absent golden means nothing was verified. + $status = 'NOGOLD' + $detail = 'no golden file - run with -UpdateGolden' + } + else { + $expected = ConvertTo-ComparableOutput (Get-Content $goldenPath -Raw) + if ($expected -ne $actual) { + $status = 'DIFF' + $detail = 'output differs from golden' + } + } + } + + $colour = switch ($status) { + 'PASS' { 'Green' } + 'GOLDEN' { 'Cyan' } + 'NOGOLD' { 'Yellow' } + default { 'Red' } + } + Write-Host "[$status] $detail" -ForegroundColor $colour + $meaningful | Select-Object -First 4 | ForEach-Object { Write-Host " $_" } + + if ($status -eq 'DIFF') { + # Show the first few differing lines - the whole point is to make + # the behaviour change legible, not just flag that one happened. + $diff = Compare-Object ($expected -split "`n") ($actual -split "`n") | + Select-Object -First 6 + Write-Host ' --- differences (< golden, > this run) ---' -ForegroundColor Red + $diff | ForEach-Object { Write-Host " $($_.SideIndicator) $($_.InputObject)" -ForegroundColor Red } + } + + $results += [pscustomobject]@{ + Language = $lang; Snippet = $snippet.Name; Status = $status; Detail = $detail + } } } - finally { - Set-Content $runnerProgram -Value $backup -NoNewline + + # ── Summary ────────────────────────────────────────────── + Write-Host "" + Write-Host '=== Summary ===' -ForegroundColor Yellow + $results | Group-Object Language, Status | + Sort-Object Name | + ForEach-Object { "{0,-22} {1}" -f $_.Name, $_.Count } | + Write-Host + + $bad = @($results | Where-Object { $_.Status -in @('FAIL', 'DIFF') }) + $ungolden = @($results | Where-Object { $_.Status -eq 'NOGOLD' }) + $skipped = @($results | Where-Object { $_.Status -eq 'SKIP' }) + + if ($skipped.Count -gt 0) { + Write-Host "" + Write-Host "$($skipped.Count) snippet(s) not executed:" -ForegroundColor DarkGray + $skipped | ForEach-Object { Write-Host " $($_.Language)/$($_.Snippet): $($_.Detail)" -ForegroundColor DarkGray } + } + + if ($ungolden.Count -gt 0) { + Write-Host "" + Write-Host "$($ungolden.Count) snippet(s) ran but were not verified (no golden file)." -ForegroundColor Yellow + $ungolden | ForEach-Object { Write-Host " $($_.Language)/$($_.Snippet)" -ForegroundColor Yellow } + } + + if ($bad.Count -gt 0) { + Write-Host "" + Write-Host "$($bad.Count) snippet(s) failed:" -ForegroundColor Red + $bad | ForEach-Object { Write-Host " $($_.Language)/$($_.Snippet): $($_.Detail)" -ForegroundColor Red } + exit 1 } - Write-Host "`n=== Summary ===" -ForegroundColor Yellow - $results | Format-Table -AutoSize | Out-String | Write-Host - $failed = @($results | Where-Object { $_.Status -ne "PASS" }).Count - if ($failed -gt 0) { Write-Host "$failed snippet(s) failed." -ForegroundColor Red; exit 1 } - Write-Host "All snippets passed against the verified fixture." -ForegroundColor Green + Write-Host "" + Write-Host 'All executed snippets matched their golden output.' -ForegroundColor Green } finally { Pop-Location diff --git a/scripts/snippet-manifest.psd1 b/scripts/snippet-manifest.psd1 new file mode 100644 index 0000000..cba2f56 --- /dev/null +++ b/scripts/snippet-manifest.psd1 @@ -0,0 +1,95 @@ +<# + Which example snippets the end-to-end runner executes, and how. + + scripts/run-examples-e2e.ps1 reads this file. Keeping the list here rather + than inline in the script means adding a snippet is a data change, and it + puts the skips in one visible place - a snippet silently missing from the + run is indistinguishable from one that passed. + + Name - identifies the snippet and names its golden output file under + tests/golden//.txt + Path - repo-relative path to the snippet + Args - arguments to pass. '{fixtureKey}' and '{fixturePath}' are replaced + with the deterministic fixture's session key and file path. Omit for + snippets that create their own session or take no input. + Skip - why this snippet is not executed. Skipped snippets are listed in the + summary so the coverage gap stays visible. + + MATLAB snippets take no arguments - they discover the session themselves + from the fixture - so none of them declare Args. +#> +@{ + + CSharp = @( + @{ Name = 'append-multiple-sessions'; Path = 'snippets\csharp\composite-sessions\append-multiple-sessions.cs' } + @{ Name = 'composite-with-associates'; Path = 'snippets\csharp\composite-sessions\composite-with-associates.cs' } + @{ Name = 'whole-session-compare'; Path = 'snippets\csharp\composite-sessions\whole-session-compare.cs' } + @{ Name = 'conversion-types'; Path = 'snippets\csharp\configuration\conversion-types.cs' } + @{ Name = 'create-parameter-config'; Path = 'snippets\csharp\configuration\create-parameter-config.cs' } + @{ Name = 'create-transient-parameter'; Path = 'snippets\csharp\configuration\create-transient-parameter.cs' } + @{ Name = 'introspect-session-config'; Path = 'snippets\csharp\configuration\introspect-session-config.cs'; Args = @('{fixtureKey}') } + @{ Name = 'bulk-parameter-read'; Path = 'snippets\csharp\data-access\bulk-parameter-read.cs'; Args = @('{fixtureKey}') } + @{ Name = 'data-status-filtering'; Path = 'snippets\csharp\data-access\data-status-filtering.cs'; Args = @('{fixtureKey}') } + @{ Name = 'multi-rate-alignment'; Path = 'snippets\csharp\data-access\multi-rate-alignment.cs'; Args = @('{fixtureKey}') } + @{ Name = 'read-samples-between'; Path = 'snippets\csharp\data-access\read-samples-between.cs'; Args = @('{fixtureKey}') } + @{ Name = 'read-subsampled-data'; Path = 'snippets\csharp\data-access\read-subsampled-data.cs'; Args = @('{fixtureKey}') } + @{ Name = 'reverse-iteration'; Path = 'snippets\csharp\data-access\reverse-iteration.cs'; Args = @('{fixtureKey}') } + @{ Name = 'timestamp-array-construction'; Path = 'snippets\csharp\data-access\timestamp-array-construction.cs' } + @{ Name = 'dotnet-function-basic'; Path = 'snippets\csharp\functions\dotnet-function-basic.cs'; Skip = 'does not compile against any shipped release - see SnippetCompilationTests' } + @{ Name = 'dotnet-function-with-pda'; Path = 'snippets\csharp\functions\dotnet-function-with-pda.cs'; Skip = 'does not compile against any shipped release - see SnippetCompilationTests' } + @{ Name = 'fdl-function-basic'; Path = 'snippets\csharp\functions\fdl-function-basic.cs'; Skip = 'does not compile against any shipped release - see SnippetCompilationTests' } + @{ Name = '01-load-session-from-file'; Path = 'snippets\csharp\getting-started\01-load-session-from-file.cs'; Args = @('{fixturePath}') } + @{ Name = '02-load-session-from-database'; Path = 'snippets\csharp\getting-started\02-load-session-from-database.cs'; Skip = 'needs a SQL Server instance, not the SQLite fixture' } + @{ Name = '03-read-parameter-samples'; Path = 'snippets\csharp\getting-started\03-read-parameter-samples.cs'; Args = @('{fixtureKey}') } + @{ Name = '04-create-session-write-data'; Path = 'snippets\csharp\getting-started\04-create-session-write-data.cs' } + @{ Name = '05-query-sessions-by-metadata'; Path = 'snippets\csharp\getting-started\05-query-sessions-by-metadata.cs' } + @{ Name = 'poll-live-samples'; Path = 'snippets\csharp\live-data\poll-live-samples.cs'; Skip = 'needs a live session from a recorder or Stream API' } + @{ Name = 'rda-parameter-change-detection'; Path = 'snippets\csharp\live-data\rda-parameter-change-detection.cs'; Skip = 'needs a live session from a recorder or Stream API' } + @{ Name = 'subscribe-to-data-events'; Path = 'snippets\csharp\live-data\subscribe-to-data-events.cs'; Skip = 'needs a live session from a recorder or Stream API' } + @{ Name = 'subscribe-to-lap-events'; Path = 'snippets\csharp\live-data\subscribe-to-lap-events.cs'; Skip = 'needs a live session from a recorder or Stream API' } + @{ Name = 'event-definitions-and-data'; Path = 'snippets\csharp\session-management\event-definitions-and-data.cs' } + @{ Name = 'lap-and-segment-management'; Path = 'snippets\csharp\session-management\lap-and-segment-management.cs' } + @{ Name = 'marker-management'; Path = 'snippets\csharp\session-management\marker-management.cs' } + @{ Name = 'parameter-unit-resolution'; Path = 'snippets\csharp\session-management\parameter-unit-resolution.cs'; Args = @('{fixtureKey}') } + @{ Name = 'query-with-composite-filter'; Path = 'snippets\csharp\session-management\query-with-composite-filter.cs' } + @{ Name = 'session-association'; Path = 'snippets\csharp\session-management\session-association.cs' } + @{ Name = 'session-metadata-crud'; Path = 'snippets\csharp\session-management\session-metadata-crud.cs' } + ) + + Python = @( + @{ Name = 'export_to_pandas'; Path = 'snippets\python\data-access\export_to_pandas.py'; Args = @('{fixtureKey}') } + @{ Name = 'multi_parameter_extract'; Path = 'snippets\python\data-access\multi_parameter_extract.py'; Args = @('{fixtureKey}') } + @{ Name = 'read_samples_to_list'; Path = 'snippets\python\data-access\read_samples_to_list.py'; Args = @('{fixtureKey}') } + @{ Name = '01_load_session'; Path = 'snippets\python\getting-started\01_load_session.py'; Args = @('{fixtureKey}') } + @{ Name = '02_read_parameters'; Path = 'snippets\python\getting-started\02_read_parameters.py'; Args = @('{fixtureKey}') } + @{ Name = '03_query_sessions'; Path = 'snippets\python\getting-started\03_query_sessions.py' } + @{ Name = '04_extract_events'; Path = 'snippets\python\getting-started\04_extract_events.py'; Args = @('{fixtureKey}') } + ) + + Matlab = @( + @{ Name = 'introspect_session_config'; Path = 'snippets\matlab\configuration\introspect_session_config.m' } + @{ Name = 'parameter_unit_resolution'; Path = 'snippets\matlab\configuration\parameter_unit_resolution.m' } + @{ Name = 'data_status_filtering'; Path = 'snippets\matlab\data-access\data_status_filtering.m' } + @{ Name = 'extract_to_timetable'; Path = 'snippets\matlab\data-access\extract_to_timetable.m' } + @{ Name = 'multi_channel_read'; Path = 'snippets\matlab\data-access\multi_channel_read.m' } + @{ Name = 'plot_parameters'; Path = 'snippets\matlab\data-access\plot_parameters.m' } + @{ Name = 'read_samples_between'; Path = 'snippets\matlab\data-access\read_samples_between.m' } + @{ Name = 'reverse_iteration'; Path = 'snippets\matlab\data-access\reverse_iteration.m' } + @{ Name = 'segment_statistics'; Path = 'snippets\matlab\data-access\segment_statistics.m' } + @{ Name = 'subsampled_read'; Path = 'snippets\matlab\data-access\subsampled_read.m' } + @{ Name = 'read_function_output'; Path = 'snippets\matlab\functions\read_function_output.m' } + @{ Name = 'create_session_write_data'; Path = 'snippets\matlab\getting-started\create_session_write_data.m' } + @{ Name = 'diagnose_setup'; Path = 'snippets\matlab\getting-started\diagnose_setup.m' } + @{ Name = 'list_parameters'; Path = 'snippets\matlab\getting-started\list_parameters.m' } + @{ Name = 'load_session'; Path = 'snippets\matlab\getting-started\load_session.m' } + @{ Name = 'load_session_from_database'; Path = 'snippets\matlab\getting-started\load_session_from_database.m'; Skip = 'needs a SQL Server instance, not the SQLite fixture' } + @{ Name = 'query_sessions'; Path = 'snippets\matlab\getting-started\query_sessions.m' } + @{ Name = 'read_parameters'; Path = 'snippets\matlab\getting-started\read_parameters.m' } + @{ Name = 'poll_live_samples'; Path = 'snippets\matlab\live-data\poll_live_samples.m'; Skip = 'needs a live session from a recorder or Stream API' } + @{ Name = 'server_listener'; Path = 'snippets\matlab\live-data\server_listener.m'; Skip = 'needs a live session from a recorder or Stream API' } + @{ Name = 'event_extraction'; Path = 'snippets\matlab\session-management\event_extraction.m' } + @{ Name = 'lap_segment_management'; Path = 'snippets\matlab\session-management\lap_segment_management.m' } + @{ Name = 'marker_management'; Path = 'snippets\matlab\session-management\marker_management.m' } + @{ Name = 'session_metadata'; Path = 'snippets\matlab\session-management\session_metadata.m' } + ) +} diff --git a/tests/README.md b/tests/README.md index 4412cfe..509b9f6 100644 --- a/tests/README.md +++ b/tests/README.md @@ -13,7 +13,52 @@ correct outputs are known — so the examples are checked for *correctness*, not | Fixture generator | [`../tools/FixtureGenerator`](../tools/FixtureGenerator) | Writes a deterministic `.ssn2` from known formulas. `VerifiedFixture` exposes the formulas as constants the tests assert against. | | Capability tests | [`SqlRace.Examples.Tests/EndToEndExampleTests.cs`](SqlRace.Examples.Tests/EndToEndExampleTests.cs) | Loads the fixture and asserts that load/read/time-range/multi-rate/query operations return the values the formulas imply. | | Function tests | [`SqlRace.Examples.Tests/FunctionComputationTests.cs`](SqlRace.Examples.Tests/FunctionComputationTests.cs) | Asserts the FunctionLibrary maths (C→F, ratio, rolling average) against hand-derived values. No licence needed. | -| Snippet smoke-runner | [`../scripts/run-examples-e2e.ps1`](../scripts/run-examples-e2e.ps1) | Generates the fixture, runs the actual runnable snippets against it, and checks they exit cleanly and print the expected anchor output. | +| Snippet compilation | [`SqlRace.Examples.Tests/SnippetCompilationTests.cs`](SqlRace.Examples.Tests/SnippetCompilationTests.cs) | Compiles every `snippets/csharp/**.cs` against the real SQL Race assemblies with Roslyn. Metadata-only, so **no licence needed** — the one suite GitHub CI can run. | +| Snippet runner | [`../scripts/run-examples-e2e.ps1`](../scripts/run-examples-e2e.ps1) | Runs the runnable C#, Python and MATLAB snippets against the fixture and compares their output against golden files in [`golden/`](golden). | +| Snippet manifest | [`../scripts/snippet-manifest.psd1`](../scripts/snippet-manifest.psd1) | Which snippets run, what arguments they take, and why the rest are skipped. | + +## Breaking-change early warning + +These pieces exist to answer one question before a release ships: *will the new +`MESL.SQLRace.API` break code customers already have?* Three layers, each catching something +the previous one cannot. + +| Layer | Catches | Licence? | +|-------|---------|----------| +| `SnippetCompilationTests` | A snippet that no longer compiles — the same errors a customer would hit | No | +| `run-examples-e2e.ps1` golden comparison | A snippet that still compiles and runs but now returns different data | Yes | +| Its Python and MATLAB legs | Renames and signature changes a C# compile would catch but which fail silently for `pythonnet` and `NET.addAssembly` callers | Yes | + +To validate a candidate build before it is published, pin the version: + +```powershell +dotnet test SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj -p:SqlRaceApiVersion=2.1.26212.6-ci +../scripts/run-examples-e2e.ps1 -SqlRaceApiVersion 2.1.26212.6-ci +``` + +The SQL Race pipeline runs both against every nightly build — see +`Build/generic-steps/run-consumer-compat-tests.yml` in `MAT.OCS.SQLRace.API`. That is the +counterpart to its `check-api-compatibility.yml`, which diffs the public API surface against +the last published package. + +### Golden output + +`run-examples-e2e.ps1` compares each snippet's stdout against `golden//.txt`. +Values that legitimately vary between runs — GUIDs, timestamps, absolute paths, elapsed times +— are normalised before comparison, so what is actually asserted is the data: parameter +names, sample counts, computed values. + +After an intended change, regenerate and **review the diff** — `-UpdateGolden` will happily +bake in a regression: + +```powershell +../scripts/run-examples-e2e.ps1 -UpdateGolden +``` + +A snippet that runs but has no golden file is reported as `NOGOLD`, not as a pass: an absent +golden means nothing was verified. Skipped snippets are listed in the summary for the same +reason — a snippet silently missing from the run would otherwise be indistinguishable from +one that passed. ## The verified fixture @@ -36,12 +81,37 @@ fixed key `f1c0ffee-0000-4000-8000-000000000001`, start 10:00:00, 10 s long): # Capability + function tests dotnet test SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj -c Release -# Snippet smoke-run against the fixture -pwsh ../scripts/run-examples-e2e.ps1 +# All three languages against the fixture +../scripts/run-examples-e2e.ps1 + +# One language, or one snippet +../scripts/run-examples-e2e.ps1 -Language Python +../scripts/run-examples-e2e.ps1 -Filter '*read*' ``` -The function-computation tests need no licence, so they can be run anywhere with -`--filter "FullyQualifiedName~FunctionComputation"`. +The function-computation and snippet-compilation tests need no licence, so they can be run +anywhere with `--filter "FullyQualifiedName~FunctionComputation"` or +`--filter "FullyQualifiedName~SnippetCompilation"`. + +### Prerequisites for the Python and MATLAB legs + +- **`pip install pythonnet cffi`.** `cffi` is a `clr_loader` dependency that does not always + come in; without it every Python snippet dies in `pythonnet.load()` with + `Failed to create a .NET runtime (coreclr)`, which looks like a SQL Race problem and is not. +- **A licensed MATLAB on `PATH`** (`matlab -batch` is used). Tested against R2025b. +- **Restore needs feed credentials.** `MESL.SQLRace.API` is referenced as `Version="*"`, and + NuGet cannot resolve a floating version without reaching the feed. Set `NUGET_AUTH_USERNAME` + and `NUGET_AUTH_TOKEN` (see `.env.example`), or pass `-SqlRaceApiVersion` to pin a version + already in the local NuGet cache. + +The script points `SQLRACE_DLL_PATH` at a *flattened* copy of the build output rather than at +the output folder itself. A NuGet-restored build keeps platform-specific assemblies under +`runtimes//` and relies on `deps.json` to select them; Python and MATLAB host the CLR +themselves, so there is no `deps.json` and the loader picks up the cross-platform placeholder +`System.Data.SqlClient` (which throws `PlatformNotSupportedException`) and cannot find +`SQLite.Interop.dll` at all. **Customers pointing pythonnet or MATLAB at a NuGet-restored SQL +Race hit exactly this**; those pointing at an ATLAS installation do not, because that +directory is already flat. ## Notes / known limitations (discovered while building this) diff --git a/tests/golden/CSharp/01-load-session-from-file.txt b/tests/golden/CSharp/01-load-session-from-file.txt new file mode 100644 index 0000000..df79f31 --- /dev/null +++ b/tests/golden/CSharp/01-load-session-from-file.txt @@ -0,0 +1,13 @@ +File: +Sessions found: 1 +Loaded session: + Identifier: Verified Test Session + Start time: 36000000000000 ns + End time: 36009990000000 ns + Duration: + Parameters: 3 + Laps: 0 + Parameters: + - Pressure:Inlet + - Temperature:Sensors + - Speed:Shaft diff --git a/tests/golden/CSharp/03-read-parameter-samples.txt b/tests/golden/CSharp/03-read-parameter-samples.txt new file mode 100644 index 0000000..154a4f4 --- /dev/null +++ b/tests/golden/CSharp/03-read-parameter-samples.txt @@ -0,0 +1,25 @@ +Parameter: Temperature:Sensors +Samples: 1000 + Timestamp (ns) Value +---------------------------------- + 36000000000000 20.0000 + 36000010000000 20.1000 + 36000020000000 20.2000 + 36000030000000 20.3000 + 36000040000000 20.4000 + 36000050000000 20.5000 + 36000060000000 20.6000 + 36000070000000 20.7000 + 36000080000000 20.8000 + 36000090000000 20.9000 + 36000100000000 21.0000 + 36000110000000 21.1000 + 36000120000000 21.2000 + 36000130000000 21.3000 + 36000140000000 21.4000 + 36000150000000 21.5000 + 36000160000000 21.6000 + 36000170000000 21.7000 + 36000180000000 21.8000 + 36000190000000 21.9000 + ... and 980 more samples diff --git a/tests/golden/CSharp/04-create-session-write-data.txt b/tests/golden/CSharp/04-create-session-write-data.txt new file mode 100644 index 0000000..b4c6217 --- /dev/null +++ b/tests/golden/CSharp/04-create-session-write-data.txt @@ -0,0 +1,6 @@ +Session created: +Connection: DbEngine=SQLite;Data Source= +Temperature:Sensors (100 Hz): 100 samples + First: 20.00 degC Last: 17.71 degC +Pressure:Inlet (50 Hz): 50 samples + First: 1.013 bar Last: 1.141 bar diff --git a/tests/golden/CSharp/05-query-sessions-by-metadata.txt b/tests/golden/CSharp/05-query-sessions-by-metadata.txt new file mode 100644 index 0000000..8a46f1f --- /dev/null +++ b/tests/golden/CSharp/05-query-sessions-by-metadata.txt @@ -0,0 +1,6 @@ +=== All sessions === + Verified Test Session Z +Total: 1 session(s) +=== Sessions matching 'Example' === + No sessions found matching 'Example'. + Tip: Run 04-create-session-write-data.cs to create some sessions. diff --git a/tests/golden/CSharp/append-multiple-sessions.txt b/tests/golden/CSharp/append-multiple-sessions.txt new file mode 100644 index 0000000..76eae52 --- /dev/null +++ b/tests/golden/CSharp/append-multiple-sessions.txt @@ -0,0 +1,16 @@ +Creating sessions: + Run Part 1: 50 samples from 36000000000000 ns + Run Part 2: 50 samples from 36005000000000 ns +Composite: 36000000000000 – 36009900000000 ns +Read 100 samples across both sessions + t+ 20.0 degC + t+ 20.5 degC + t+ 21.0 degC + t+ 21.5 degC + t+ 22.0 degC + t+ 22.5 degC + t+ 23.0 degC + t+ 23.5 degC + t+ 24.0 degC + t+ 24.5 degC + ... and 90 more diff --git a/tests/golden/CSharp/bulk-parameter-read.txt b/tests/golden/CSharp/bulk-parameter-read.txt new file mode 100644 index 0000000..bdeb868 --- /dev/null +++ b/tests/golden/CSharp/bulk-parameter-read.txt @@ -0,0 +1,7 @@ +Session: +Reading 3 parameter(s)... +Parameter Samples Min Max +---------------------------------------------------------------------- +Pressure:Inlet 1000 2.0000 2.0000 +Temperature:Sensors 1000 20.0000 119.9000 +Speed:Shaft 500 0.0000 99.8000 diff --git a/tests/golden/CSharp/composite-with-associates.txt b/tests/golden/CSharp/composite-with-associates.txt new file mode 100644 index 0000000..af7110b --- /dev/null +++ b/tests/golden/CSharp/composite-with-associates.txt @@ -0,0 +1,15 @@ +Primary: (1 param) +Associate: (1 param) +Composite read: 20 temp + 20 pressure + Time (ms) Temp (degC) Press (bar) +────────────────────────────────────── + 0.0 20.00 2.000 + 100.0 20.50 2.100 + 200.0 21.00 2.200 + 300.0 21.50 2.300 + 400.0 22.00 2.400 + 500.0 22.50 2.500 + 600.0 23.00 2.600 + 700.0 23.50 2.700 + 800.0 24.00 2.800 + 900.0 24.50 2.900 diff --git a/tests/golden/CSharp/conversion-types.txt b/tests/golden/CSharp/conversion-types.txt new file mode 100644 index 0000000..d432852 --- /dev/null +++ b/tests/golden/CSharp/conversion-types.txt @@ -0,0 +1,9 @@ +1. RationalConversion 'linear_degC': 1:1 pass-through +2. TableConversion 'table_bar': ADC counts → bar (5-point lookup) + Index Linear (degC) Table (bar) +──────────────────────────────────── + 0 20.00 0.000 + 1 25.00 2.500 + 2 30.00 5.000 + 3 35.00 7.500 + 4 40.00 10.000 diff --git a/tests/golden/CSharp/create-parameter-config.txt b/tests/golden/CSharp/create-parameter-config.txt new file mode 100644 index 0000000..97d00e1 --- /dev/null +++ b/tests/golden/CSharp/create-parameter-config.txt @@ -0,0 +1,6 @@ +1. Conversion: degC_conversion (1:1 rational) +2. Channel: id=100, 100 Hz, Double64Bit, interval=10000000 ns +3. Groups: Thermal (parameter), Sensors (application) +4. Parameter: Temperature:Sensors → channelId=100, conversion=degC_conversion +5. Configuration committed. +Wrote 10 samples, read back 10: [20.0 ... 0.0] degC diff --git a/tests/golden/CSharp/create-transient-parameter.txt b/tests/golden/CSharp/create-transient-parameter.txt new file mode 100644 index 0000000..fc9e0ac --- /dev/null +++ b/tests/golden/CSharp/create-transient-parameter.txt @@ -0,0 +1,4 @@ +Transient parameter 'CalculatedTemp:Derived' created (in-memory only) +Read 10 transient samples: [100.0 ... 0.0] +After reload: parameter exists = False +Transient parameters are not persisted — they exist only in-memory. diff --git a/tests/golden/CSharp/data-status-filtering.txt b/tests/golden/CSharp/data-status-filtering.txt new file mode 100644 index 0000000..398d869 --- /dev/null +++ b/tests/golden/CSharp/data-status-filtering.txt @@ -0,0 +1,16 @@ +Parameter: Temperature:Sensors +Total samples: 1000 +Status breakdown: + Sample 1000 (100.0%) +Valid samples (DataStatusType.Sample): + t+ 20.0000 + t+ 20.1000 + t+ 20.2000 + t+ 20.3000 + t+ 20.4000 + t+ 20.5000 + t+ 20.6000 + t+ 20.7000 + t+ 20.8000 + t+ 20.9000 +Total valid: 1000 / 1000 diff --git a/tests/golden/CSharp/introspect-session-config.txt b/tests/golden/CSharp/introspect-session-config.txt new file mode 100644 index 0000000..23db3d8 --- /dev/null +++ b/tests/golden/CSharp/introspect-session-config.txt @@ -0,0 +1,14 @@ +Session: Verified Test Session () +=== Parameters (3) === +Identifier Conversion AppGroup +──────────────────────────────────────────────────────────────────────── +Pressure:Inlet Pressure:Inlet_conversion Sensors +Temperature:Sensors Temperature:Sensors_conversion Sensors +Speed:Shaft Speed:Shaft_conversion Driveline +=== Application Groups === + Sensors + Driveline +=== Parameter Groups === + Hydraulic + Thermal + Rotation diff --git a/tests/golden/CSharp/lap-and-segment-management.txt b/tests/golden/CSharp/lap-and-segment-management.txt new file mode 100644 index 0000000..c415d8d --- /dev/null +++ b/tests/golden/CSharp/lap-and-segment-management.txt @@ -0,0 +1,8 @@ +Session : 5 segments + # Name Start (ns) Offset (s) +────────────────────────────────────────────────────────── + 1 Warmup 36000000000000 0 + 2 Steady State 1 36060000000000 60 + 3 Ramp Up 36180000000000 180 + 4 Steady State 2 36240000000000 240 + 5 Cooldown 36360000000000 360 diff --git a/tests/golden/CSharp/marker-management.txt b/tests/golden/CSharp/marker-management.txt new file mode 100644 index 0000000..9b955c0 --- /dev/null +++ b/tests/golden/CSharp/marker-management.txt @@ -0,0 +1,10 @@ +Session : 3 markers added + [Anomaly] 36000000000000–36002000000000 ns () + Type: Anomaly + Description: Unexpected temperature spike detected + [RegionOfInterest] 36005000000000–36008000000000 ns () + Type: RegionOfInterest + Description: Steady-state operating window + [Transient] 36010000000000–36012000000000 ns () + Type: Transient + Description: Load step response period diff --git a/tests/golden/CSharp/multi-rate-alignment.txt b/tests/golden/CSharp/multi-rate-alignment.txt new file mode 100644 index 0000000..12223fe --- /dev/null +++ b/tests/golden/CSharp/multi-rate-alignment.txt @@ -0,0 +1,19 @@ +Aligned Temperature:Sensors and Pressure:Inlet to 99 points at 10 Hz + Time (ms) Temperature:Sensors Pressure:Inlet +-------------------------------------------------- + 0.0 20.4500 2.0000 + 100.0 21.4500 2.0000 + 200.0 22.4500 2.0000 + 300.0 23.4500 2.0000 + 400.0 24.4500 2.0000 + 500.0 25.4500 2.0000 + 600.0 26.4500 2.0000 + 700.0 27.4500 2.0000 + 800.0 28.4500 2.0000 + 900.0 29.4500 2.0000 + 1000.0 30.4500 2.0000 + 1100.0 31.4500 2.0000 + 1200.0 32.4500 2.0000 + 1300.0 33.4500 2.0000 + 1400.0 34.4500 2.0000 + ... and 84 more rows diff --git a/tests/golden/CSharp/parameter-unit-resolution.txt b/tests/golden/CSharp/parameter-unit-resolution.txt new file mode 100644 index 0000000..c5c5e12 --- /dev/null +++ b/tests/golden/CSharp/parameter-unit-resolution.txt @@ -0,0 +1,6 @@ +Session: Verified Test Session (3 parameters) +Identifier Conversion Unit Format Range +─────────────────────────────────────────────────────────────────────────────────────────────── +Pressure:Inlet Pressure:Inlet_conversion bar %5.3f [0.0, 10.0] +Temperature:Sensors Temperature:Sensors_conversion degC %5.2f [0.0, 500.0] +Speed:Shaft Speed:Shaft_conversion rad/s %6.2f [0.0, 200.0] diff --git a/tests/golden/CSharp/query-with-composite-filter.txt b/tests/golden/CSharp/query-with-composite-filter.txt new file mode 100644 index 0000000..b02a195 --- /dev/null +++ b/tests/golden/CSharp/query-with-composite-filter.txt @@ -0,0 +1,4 @@ +Filter: Identifier contains 'Example' AND recorded in last 30 days +Session Key Identifier Recorded +────────────────────────────────────────────────────────────────────────────────────────────── +No sessions matched. Create some sessions first with 04-create-session-write-data.cs. diff --git a/tests/golden/CSharp/read-samples-between.txt b/tests/golden/CSharp/read-samples-between.txt new file mode 100644 index 0000000..a3ad51a --- /dev/null +++ b/tests/golden/CSharp/read-samples-between.txt @@ -0,0 +1,14 @@ +Parameter: Temperature:Sensors +Range: 36000000000000 – 36000500000000 ns () +Samples: 51 + t+ 20.0000 + t+ 20.1000 + t+ 20.2000 + t+ 20.3000 + t+ 20.4000 + t+ 20.5000 + t+ 20.6000 + t+ 20.7000 + t+ 20.8000 + t+ 20.9000 + ... and 41 more diff --git a/tests/golden/CSharp/read-subsampled-data.txt b/tests/golden/CSharp/read-subsampled-data.txt new file mode 100644 index 0000000..41e5892 --- /dev/null +++ b/tests/golden/CSharp/read-subsampled-data.txt @@ -0,0 +1,19 @@ +Parameter: Temperature:Sensors +Original range: 36000000000000 – 36009990000000 ns +Subsampled to: 99 points at 10 Hz + t+ 20.4500 + t+ 21.4500 + t+ 22.4500 + t+ 23.4500 + t+ 24.4500 + t+ 25.4500 + t+ 26.4500 + t+ 27.4500 + t+ 28.4500 + t+ 29.4500 + t+ 30.4500 + t+ 31.4500 + t+ 32.4500 + t+ 33.4500 + t+ 34.4500 + ... and 84 more diff --git a/tests/golden/CSharp/reverse-iteration.txt b/tests/golden/CSharp/reverse-iteration.txt new file mode 100644 index 0000000..a7a0bb0 --- /dev/null +++ b/tests/golden/CSharp/reverse-iteration.txt @@ -0,0 +1,14 @@ +Parameter: Temperature:Sensors +Last 10 samples (reverse order): + Timestamp (ns) Value +---------------------------------- + 36009990000000 119.9000 + 36009980000000 119.8000 + 36009970000000 119.7000 + 36009960000000 119.6000 + 36009950000000 119.5000 + 36009940000000 119.4000 + 36009930000000 119.3000 + 36009920000000 119.2000 + 36009910000000 119.1000 + 36009900000000 119.0000 diff --git a/tests/golden/CSharp/session-association.txt b/tests/golden/CSharp/session-association.txt new file mode 100644 index 0000000..c88b1a8 --- /dev/null +++ b/tests/golden/CSharp/session-association.txt @@ -0,0 +1,6 @@ +Created primary: +Created associate: +Associated with primary +Loaded primary session: Primary Test Run + Items: 1 + Associated sessions loaded: 0 diff --git a/tests/golden/CSharp/session-metadata-crud.txt b/tests/golden/CSharp/session-metadata-crud.txt new file mode 100644 index 0000000..a302472 --- /dev/null +++ b/tests/golden/CSharp/session-metadata-crud.txt @@ -0,0 +1,9 @@ +Created session with 5 metadata items +Reloaded session: 5 metadata items +Key Value +──────────────────────────────────────────────────────────── +Facility Wind Tunnel A +Operator Test Engineer 1 +TestObjective Thermal characterisation at 80% load +AmbientTemp 22.5 +RunNumber 42 diff --git a/tests/golden/CSharp/timestamp-array-construction.txt b/tests/golden/CSharp/timestamp-array-construction.txt new file mode 100644 index 0000000..e08c097 --- /dev/null +++ b/tests/golden/CSharp/timestamp-array-construction.txt @@ -0,0 +1,24 @@ +=== Pattern 1: Uniform at 10 Hz === +Count: 10 + 36000000000000 ns (t+) + 36000100000000 ns (t+) + 36000200000000 ns (t+) + 36000300000000 ns (t+) + 36000400000000 ns (t+) + ... +=== Pattern 2: Specific offsets (, , , , ) === +Count: 5 + 36000000000000 ns (t+) + 36000250000000 ns (t+) + 36000500000000 ns (t+) + 36000750000000 ns (t+) + 36001000000000 ns (t+) +=== Pattern 3: Log-spaced (20 points) === +Count: 20 + 36000000000001 ns (t+) + 36000000000002 ns (t+) + 36000000000008 ns (t+) + 36000000000026 ns (t+) + 36000000000078 ns (t+) + ... + 36001000000000 ns (t+) diff --git a/tests/golden/CSharp/whole-session-compare.txt b/tests/golden/CSharp/whole-session-compare.txt new file mode 100644 index 0000000..ae98558 --- /dev/null +++ b/tests/golden/CSharp/whole-session-compare.txt @@ -0,0 +1,18 @@ +Whole-session comparison: Baseline vs Test Run + Time (ms) Baseline Test Run Delta +──────────────────────────────────────────── + 0.0 20.00 22.00 2.00 + 100.0 20.30 22.40 2.10 + 200.0 20.60 22.80 2.20 + 300.0 20.90 23.20 2.30 + 400.0 21.20 23.60 2.40 + 500.0 21.50 24.00 2.50 + 600.0 21.80 24.40 2.60 + 700.0 22.10 24.80 2.70 + 800.0 22.40 25.20 2.80 + 900.0 22.70 25.60 2.90 + 1000.0 23.00 26.00 3.00 + 1100.0 23.30 26.40 3.10 + 1200.0 23.60 26.80 3.20 + 1300.0 23.90 27.20 3.30 + 1400.0 24.20 27.60 3.40 diff --git a/tests/golden/Matlab/create_session_write_data.txt b/tests/golden/Matlab/create_session_write_data.txt new file mode 100644 index 0000000..f98860e --- /dev/null +++ b/tests/golden/Matlab/create_session_write_data.txt @@ -0,0 +1,7 @@ +Session created: +Connection: DbEngine=SQLite;Data Source= +Temperature:Sensors (100 Hz): 100 samples + First: 20.00 degC Last: 17.71 degC +Pressure:Inlet (50 Hz): 50 samples + First: 1.013 bar Last: 1.141 bar +Done. diff --git a/tests/golden/Matlab/data_status_filtering.txt b/tests/golden/Matlab/data_status_filtering.txt new file mode 100644 index 0000000..565ec12 --- /dev/null +++ b/tests/golden/Matlab/data_status_filtering.txt @@ -0,0 +1,17 @@ +Parameter: Speed:Shaft +Total samples: 6250 +Status breakdown: + Sample 6250 (100.0%) +Valid samples (Sample status): + t+ 192.0000 + t+ 192.0000 + t+ 194.0000 + t+ 197.0000 + t+ 194.0000 + t+ 195.0000 + t+ 199.0000 + t+ 198.0000 + t+ 196.0000 + t+ 199.0000 +Total valid: 6250 / 6250 +Done. diff --git a/tests/golden/Matlab/diagnose_setup.txt b/tests/golden/Matlab/diagnose_setup.txt new file mode 100644 index 0000000..7d23434 --- /dev/null +++ b/tests/golden/Matlab/diagnose_setup.txt @@ -0,0 +1,7 @@ +=== SQL Race MATLAB Setup Diagnostics === +[1/5] Checking .NET runtime... OK (Runtime: core, Status: notloaded) +[2/5] Loading SQL Race assembly... OK ( +[3/5] Initialising Core... OK +[4/5] Loading test session... OK (Parameters: 3, Segments: 1) +[5/5] Reading parameter data... OK (2400 samples) +=== Diagnostics complete === diff --git a/tests/golden/Matlab/event_extraction.txt b/tests/golden/Matlab/event_extraction.txt new file mode 100644 index 0000000..8ac80c1 --- /dev/null +++ b/tests/golden/Matlab/event_extraction.txt @@ -0,0 +1,14 @@ +Session: Track Test Session 1 +Event definitions: 0 +Reading events across 5 segment(s): +Segment Events +-------------------------------- +Lap 1 0 +Lap 2 0 +Lap 3 0 +Lap 4 0 +Lap 5 0 +Total events: 0 +(This session contains no event instances — events are written by + the recording/import pipeline, e.g. ATLAS or a ServerListener recorder.) +Done. diff --git a/tests/golden/Matlab/extract_to_timetable.txt b/tests/golden/Matlab/extract_to_timetable.txt new file mode 100644 index 0000000..f476cb2 --- /dev/null +++ b/tests/golden/Matlab/extract_to_timetable.txt @@ -0,0 +1,16 @@ +Timetable: 2400 rows + timeVector Temperature:Sensors + __________ ___________________ + 0 sec -4 + 0.05 sec -4 + 0.1 sec -5 + 0.15 sec -4 + 0.2 sec -5 + 0.25 sec -5 + 0.3 sec -5 + 0.35 sec -5 + 0.4 sec -4 + 0.45 sec -4 +Min: -7.0000 +Max: -1.0000 +Mean: -4.1037 diff --git a/tests/golden/Matlab/introspect_session_config.txt b/tests/golden/Matlab/introspect_session_config.txt new file mode 100644 index 0000000..731e0e7 --- /dev/null +++ b/tests/golden/Matlab/introspect_session_config.txt @@ -0,0 +1,16 @@ +Session: Turbine Inspection Week 1 () +=== Parameters (3) === +Identifier Conversion AppGroup +------------------------------------------------------------------------ +Pressure:Inlet CONV_Pressure:Inlet Inlet +Temperature:Sensors CONV_Temperature:Sensors Sensors +RPM:Turbine CONV_RPM:Turbine Turbine +=== Application Groups === + Inlet + Sensors + Turbine +=== Parameter Groups === + PG_Pressure_Inlet + PG_RPM_Turbine + PG_Temperature_Sensors +Done. diff --git a/tests/golden/Matlab/lap_segment_management.txt b/tests/golden/Matlab/lap_segment_management.txt new file mode 100644 index 0000000..743b842 --- /dev/null +++ b/tests/golden/Matlab/lap_segment_management.txt @@ -0,0 +1,9 @@ +Session : 5 segments + # Name Start (ns) Offset (s) +---------------------------------------------------------- + 1 Warmup 36000000000000 0 + 2 Steady State 1 36060000000000 60 + 3 Ramp Up 36180000000000 180 + 4 Steady State 2 36240000000000 240 + 5 Cooldown 36360000000000 360 +Done. diff --git a/tests/golden/Matlab/list_parameters.txt b/tests/golden/Matlab/list_parameters.txt new file mode 100644 index 0000000..4350cad --- /dev/null +++ b/tests/golden/Matlab/list_parameters.txt @@ -0,0 +1,9 @@ +Session: Turbine Inspection Week 1 +Parameters: 3 + Identifier Name AppGroup Groups Conversion MinValue MaxValue Samples + _____________________ _____________ _________ ________________________ __________________________ ________ ________ _______ + "Pressure:Inlet" "Pressure" "Inlet" "PG_Pressure_Inlet" "CONV_Pressure:Inlet" 3000 7000 6000 + "Temperature:Sensors" "Temperature" "Sensors" "PG_Temperature_Sensors" "CONV_Temperature:Sensors" 300 700 6000 + "RPM:Turbine" "RPM" "Turbine" "PG_RPM_Turbine" "CONV_RPM:Turbine" 2000 5000 6000 +Catalogue written to: +Done. diff --git a/tests/golden/Matlab/load_session.txt b/tests/golden/Matlab/load_session.txt new file mode 100644 index 0000000..af152e6 --- /dev/null +++ b/tests/golden/Matlab/load_session.txt @@ -0,0 +1,9 @@ +SQL Race initialised +Session loaded: + Identifier: Flight Test FT-042 + Start time: 32400000000000 ns + End time: 32519950000000 ns + Duration: + Parameters: 3 + Laps: 1 +Session closed. diff --git a/tests/golden/Matlab/marker_management.txt b/tests/golden/Matlab/marker_management.txt new file mode 100644 index 0000000..2213464 --- /dev/null +++ b/tests/golden/Matlab/marker_management.txt @@ -0,0 +1,11 @@ +Session : 3 markers added + [Anomaly] 36000000000000-36002000000000 ns () + Type: Anomaly + Description: Unexpected temperature spike detected + [RegionOfInterest] 36005000000000-36008000000000 ns () + Type: RegionOfInterest + Description: Steady-state operating window + [Transient] 36010000000000-36012000000000 ns () + Type: Transient + Description: Load step response period +Done. diff --git a/tests/golden/Matlab/multi_channel_read.txt b/tests/golden/Matlab/multi_channel_read.txt new file mode 100644 index 0000000..5ed55b2 --- /dev/null +++ b/tests/golden/Matlab/multi_channel_read.txt @@ -0,0 +1,12 @@ +Session has 3 parameter(s) + Pressure:Inlet: 2400 samples + Temperature:Sensors: 2400 samples + Acceleration:Platform: 2400 samples +Timetable: 2400 rows x 3 variables + timeVec Pressure_Inlet Temperature_Sensors Acceleration_Platform + ________ ______________ ___________________ _____________________ + 0 sec 100 -4 0 + 0.05 sec 100 -4 0 + 0.1 sec 100 -5 0 + 0.15 sec 100 -4 0 + 0.2 sec 100 -5 0 diff --git a/tests/golden/Matlab/parameter_unit_resolution.txt b/tests/golden/Matlab/parameter_unit_resolution.txt new file mode 100644 index 0000000..6ec975f --- /dev/null +++ b/tests/golden/Matlab/parameter_unit_resolution.txt @@ -0,0 +1,7 @@ +Session: Turbine Inspection Week 1 (3 parameters) +Identifier Conversion Unit Format Range +----------------------------------------------------------------------------------------------- +Pressure:Inlet CONV_Pressure:Inlet kPa %5.0f [3000.0, 7000.0] +Temperature:Sensors CONV_Temperature:Sensors °C %5.0f [300.0, 700.0] +RPM:Turbine CONV_RPM:Turbine RPM %5.0f [2000.0, 5000.0] +Done. diff --git a/tests/golden/Matlab/plot_parameters.txt b/tests/golden/Matlab/plot_parameters.txt new file mode 100644 index 0000000..2d25005 --- /dev/null +++ b/tests/golden/Matlab/plot_parameters.txt @@ -0,0 +1,5 @@ +Pressure:Inlet 6000 samples +Temperature:Sensors 6000 samples +RPM:Turbine 6000 samples +Displayed 3 parameter(s). +Done. diff --git a/tests/golden/Matlab/query_sessions.txt b/tests/golden/Matlab/query_sessions.txt new file mode 100644 index 0000000..ce47feb --- /dev/null +++ b/tests/golden/Matlab/query_sessions.txt @@ -0,0 +1,5 @@ +Searching for sessions matching "Test"... +Session Key Identifier Recorded +---------------------------------------------------------------------------------------------- + Flight Test FT-042 7/10/2025 9:00:00 AM +1 session(s) found. diff --git a/tests/golden/Matlab/read_function_output.txt b/tests/golden/Matlab/read_function_output.txt new file mode 100644 index 0000000..e49d37d --- /dev/null +++ b/tests/golden/Matlab/read_function_output.txt @@ -0,0 +1,7 @@ +Session: Track Test Session 1 +Function parameters found: 0 +No function parameters in this session. +Author a function in ATLAS (or via the .NET FunctionManager API), +then re-run: its output appears as a ":Functions" parameter +and is read exactly like the example below. +Done. diff --git a/tests/golden/Matlab/read_parameters.txt b/tests/golden/Matlab/read_parameters.txt new file mode 100644 index 0000000..e5b5612 --- /dev/null +++ b/tests/golden/Matlab/read_parameters.txt @@ -0,0 +1,5 @@ +Parameter: Temperature:Sensors +Samples: 2400 +Min: -7.0000 +Max: -1.0000 +Mean: -4.1037 diff --git a/tests/golden/Matlab/read_samples_between.txt b/tests/golden/Matlab/read_samples_between.txt new file mode 100644 index 0000000..e0e715f --- /dev/null +++ b/tests/golden/Matlab/read_samples_between.txt @@ -0,0 +1,15 @@ +Parameter: Temperature:Sensors +Range: 36000000000000 - 36000500000000 ns () +Samples: 11 + t+ 572.0000 + t+ 571.0000 + t+ 585.0000 + t+ 572.0000 + t+ 567.0000 + t+ 557.0000 + t+ 574.0000 + t+ 559.0000 + t+ 559.0000 + t+ 561.0000 + ... and 1 more +Done. diff --git a/tests/golden/Matlab/reverse_iteration.txt b/tests/golden/Matlab/reverse_iteration.txt new file mode 100644 index 0000000..6024ccb --- /dev/null +++ b/tests/golden/Matlab/reverse_iteration.txt @@ -0,0 +1,15 @@ +Parameter: Temperature:Sensors +Last 10 samples (reverse order): + Timestamp (ns) Value +---------------------------------- + 32519950000000 -4.0000 + 32519900000000 -5.0000 + 32519850000000 -4.0000 + 32519800000000 -4.0000 + 32519750000000 -5.0000 + 32519700000000 -4.0000 + 32519650000000 -4.0000 + 32519600000000 -5.0000 + 32519550000000 -4.0000 + 32519500000000 -5.0000 +Done. diff --git a/tests/golden/Matlab/segment_statistics.txt b/tests/golden/Matlab/segment_statistics.txt new file mode 100644 index 0000000..23543ba --- /dev/null +++ b/tests/golden/Matlab/segment_statistics.txt @@ -0,0 +1,4 @@ +Parameter: Temperature:Sensors +Segments: 1 +Segment 1 — Min: -7.0000, Max: -1.0000, Mean: -4.1037 +Done. diff --git a/tests/golden/Matlab/session_metadata.txt b/tests/golden/Matlab/session_metadata.txt new file mode 100644 index 0000000..4b59b8d --- /dev/null +++ b/tests/golden/Matlab/session_metadata.txt @@ -0,0 +1,10 @@ +Created session with 5 metadata items +Reloaded session: 5 metadata items +Key Value +------------------------------------------------------------ +Facility Wind Tunnel A +Operator Test Engineer 1 +TestObjective Thermal characterisation at 80% load +AmbientTemp 22.5 +RunNumber 42 +Done. diff --git a/tests/golden/Matlab/subsampled_read.txt b/tests/golden/Matlab/subsampled_read.txt new file mode 100644 index 0000000..085a2c3 --- /dev/null +++ b/tests/golden/Matlab/subsampled_read.txt @@ -0,0 +1,20 @@ +Parameter: Temperature:Sensors +Original range: 28800000000000 - 28919980000000 ns +Subsampled to: 1199 points at 10 Hz + t+ 300.0000 + t+ 300.0000 + t+ 300.0000 + t+ 304.6000 + t+ 310.2000 + t+ 307.6000 + t+ 306.6000 + t+ 310.2000 + t+ 313.8000 + t+ 322.2000 + t+ 320.6000 + t+ 326.4000 + t+ 328.8000 + t+ 328.6000 + t+ 335.0000 + ... and 1184 more +Done. diff --git a/tests/golden/Python/01_load_session.txt b/tests/golden/Python/01_load_session.txt new file mode 100644 index 0000000..7ca600b --- /dev/null +++ b/tests/golden/Python/01_load_session.txt @@ -0,0 +1,9 @@ +SQL Race initialised +Session loaded: + Name: Verified Test Session + Start time: 36000000000000 ns + End time: 36009990000000 ns + Duration: + Parameters: 3 + Laps: 0 +Session closed. diff --git a/tests/golden/Python/02_read_parameters.txt b/tests/golden/Python/02_read_parameters.txt new file mode 100644 index 0000000..f970f88 --- /dev/null +++ b/tests/golden/Python/02_read_parameters.txt @@ -0,0 +1,25 @@ +Parameter: Temperature:Sensors +Samples: 1000 + Timestamp (ns) Value +────────────────────────────────── + 36000000000000 20.0000 + 36000010000000 20.1000 + 36000020000000 20.2000 + 36000030000000 20.3000 + 36000040000000 20.4000 + 36000050000000 20.5000 + 36000060000000 20.6000 + 36000070000000 20.7000 + 36000080000000 20.8000 + 36000090000000 20.9000 + 36000100000000 21.0000 + 36000110000000 21.1000 + 36000120000000 21.2000 + 36000130000000 21.3000 + 36000140000000 21.4000 + 36000150000000 21.5000 + 36000160000000 21.6000 + 36000170000000 21.7000 + 36000180000000 21.8000 + 36000190000000 21.9000 + ... and 980 more samples diff --git a/tests/golden/Python/03_query_sessions.txt b/tests/golden/Python/03_query_sessions.txt new file mode 100644 index 0000000..ab98660 --- /dev/null +++ b/tests/golden/Python/03_query_sessions.txt @@ -0,0 +1,4 @@ +Searching for sessions matching 'Example'... +Session Key Identifier Recorded +────────────────────────────────────────────────────────────────────────────────────────────── +No sessions found. Run 04-create-session-write-data.cs to create some. diff --git a/tests/golden/Python/export_to_pandas.txt b/tests/golden/Python/export_to_pandas.txt new file mode 100644 index 0000000..d198bf2 --- /dev/null +++ b/tests/golden/Python/export_to_pandas.txt @@ -0,0 +1,22 @@ +DataFrame: 1000 rows + timestamp_ns seconds_from_midnight Temperature:Sensors time +36000000000000 36000.00 20.0 0 days 10:00:00 +36000010000000 36000.01 20.1 0 days 10:00:00.010000 +36000020000000 36000.02 20.2 0 days 10:00:00.020000 +36000030000000 36000.03 20.3 0 days 10:00:00.030000 +36000040000000 36000.04 20.4 0 days 10:00:00.040000 +36000050000000 36000.05 20.5 0 days 10:00:00.050000 +36000060000000 36000.06 20.6 0 days 10:00:00.060000 +36000070000000 36000.07 20.7 0 days 10:00:00.070000 +36000080000000 36000.08 20.8 0 days 10:00:00.080000 +36000090000000 36000.09 20.9 0 days 10:00:00.090000 +Describe: +count 1000.000000 +mean 69.950000 +std 28.881944 +min 20.000000 +25% 44.975000 +50% 69.950000 +75% 94.925000 +max 119.900000 +Name: Temperature:Sensors, dtype: float64 diff --git a/tests/golden/Python/read_samples_to_list.txt b/tests/golden/Python/read_samples_to_list.txt new file mode 100644 index 0000000..c32c660 --- /dev/null +++ b/tests/golden/Python/read_samples_to_list.txt @@ -0,0 +1,7 @@ +Extracted 1000 samples into Python lists + Type: timestamps=list, values=list + First: t=36000000000000 ns, v=20.0000 + Last: t=36009990000000 ns, v=119.9000 + Min: 20.0000 + Max: 119.9000 + Mean: 69.9500 diff --git a/tools/SnippetRunner/SnippetRunner.csproj b/tools/SnippetRunner/SnippetRunner.csproj index 7bf73cb..bac4616 100644 --- a/tools/SnippetRunner/SnippetRunner.csproj +++ b/tools/SnippetRunner/SnippetRunner.csproj @@ -5,8 +5,36 @@ enable enable $(NoWarn);CA1416 + + + * + - + + + + + + + From a2b7854ddeb7de7368420dc7aad2d3a860530142 Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Tue, 25 Aug 2026 13:03:56 +0000 Subject: [PATCH 03/15] Preflight the MATLAB licence before running the MATLAB leg A MATLAB startup costs the better part of a minute, so a missing install or an unavailable licence seat produced twenty-odd identical failures over a quarter of an hour. One checkout up front turns that into one clear message in forty seconds, and reports the snippets as skipped rather than failed - they were never run, so calling them failures is wrong. The message names MLM_LICENSE_FILE and points at the two things that actually go wrong on a build agent: no route to the FlexLM server, or no free concurrent seat. --- scripts/run-examples-e2e.ps1 | 53 ++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/scripts/run-examples-e2e.ps1 b/scripts/run-examples-e2e.ps1 index af649e5..3130769 100644 --- a/scripts/run-examples-e2e.ps1 +++ b/scripts/run-examples-e2e.ps1 @@ -211,6 +211,49 @@ try { } } +# Preflight before the MATLAB leg. A MATLAB startup costs the better part of a + # minute, so without this a missing install or an unavailable licence seat + # produces twenty-odd identical failures over quarter of an hour instead of + # one clear message in forty seconds. + # + # Licensing is a network checkout: /licenses/network.lic points at + # the FlexLM server, or MLM_LICENSE_FILE overrides it as port@host. Note that + # FlexLM needs two ports open - lmgrd (27000 by default) and the MLM vendor + # daemon, which is randomly assigned unless the server pins it with + # "port=" on its DAEMON line. + function Test-MatlabAvailable { + $matlab = Get-Command matlab -ErrorAction SilentlyContinue + if (-not $matlab) { + Write-Host 'MATLAB not found on PATH - skipping the MATLAB leg.' -ForegroundColor Yellow + Write-Host ' Install MATLAB, or run with -Language CSharp or -Language Python.' -ForegroundColor Yellow + return $false + } + + Write-Host "Checking out a MATLAB licence ($($matlab.Source))..." -ForegroundColor Cyan + $previousPreference = $ErrorActionPreference + $ErrorActionPreference = 'Continue' + try { + $probe = & matlab -batch "fprintf('MATLAB %s licensed\n', version)" 2>&1 | + ForEach-Object { $_.ToString() } + $exitCode = $LASTEXITCODE + } + finally { + $ErrorActionPreference = $previousPreference + } + + if ($exitCode -ne 0) { + Write-Host 'MATLAB is installed but would not start - skipping the MATLAB leg.' -ForegroundColor Yellow + $probe | Select-Object -First 6 | ForEach-Object { Write-Host " $_" -ForegroundColor Yellow } + Write-Host " MLM_LICENSE_FILE = $($env:MLM_LICENSE_FILE)" -ForegroundColor Yellow + Write-Host ' A licence manager error here usually means the agent cannot reach' -ForegroundColor Yellow + Write-Host ' the FlexLM server, or no concurrent seat is free.' -ForegroundColor Yellow + return $false + } + + Write-Host " $($probe | Where-Object { $_ -match 'licensed' } | Select-Object -First 1)" -ForegroundColor DarkGray + return $true + } + function Invoke-MatlabSnippet { param ([hashtable] $Snippet, [string[]] $Arguments) @@ -248,6 +291,16 @@ try { $snippets = @($manifest[$lang] | Where-Object { $_.Name -like $Filter }) if ($snippets.Count -eq 0) { continue } + if ($lang -eq 'Matlab' -and -not (Test-MatlabAvailable)) { + foreach ($snippet in $snippets) { + $results += [pscustomobject]@{ + Language = 'Matlab'; Snippet = $snippet.Name; Status = 'SKIP' + Detail = 'MATLAB unavailable or unlicensed on this machine' + } + } + continue + } + Write-Host "" Write-Host "########## $lang ##########" -ForegroundColor Yellow From 56a867de4bde68f6a7aecaeb2c87c0a46a28fec7 Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Tue, 25 Aug 2026 13:32:17 +0000 Subject: [PATCH 04/15] Validate against an unpublished candidate package, and stop a hung snippet wedging the run Three changes, all in service of running the examples against a SQL Race build before it is published to anyone. AdditionalPackageSource takes a folder holding a .nupkg and passes it as RestoreAdditionalProjectSources. That is deliberately not "dotnet nuget add source": the repo nuget.config starts with , which discards any source added at user level. This is what lets a build validate the package it just produced without publishing it first. SqlRaceApiVersion moves to Directory.Build.props so it applies to every project at once. Setting it on the test project alone left SqlRace.Examples.Core and FunctionLibrary still resolving Version="*" from the feed, so a candidate run would have mixed two different SQL Race builds and proved nothing. Verified: all three projects now resolve the same pinned version. Snippets run through Start-Process with a hard timeout and are killed as a process tree if they exceed it. A snippet that hangs rather than fails used to wedge the whole run, and because the hung process keeps a handle on the SQLite fixture, every snippet after it failed too. TIMEOUT is reported separately from FAIL - the snippet is hung, not wrong. Reset-Fixture also retries, since SQLite handles are not always released the moment a process exits. Two things that had to be right for the process launch to work at all: Start-Process -PassThru releases the native handle on exit unless Handle is touched first, leaving ExitCode empty; and the redirected output has to be read as UTF-8, or the box-drawing characters in the snippets' table rules come back as mojibake and every golden comparison fails. Verified end to end against a package present only in a local folder: C# 24 pass / 1 known failure / 8 skipped, Python 5 pass / 2 known failures - identical to the published-package run. --- Directory.Build.props | 26 ++++ .../FunctionLibrary/FunctionLibrary.csproj | 2 +- .../SqlRace.Examples.Core.csproj | 2 +- scripts/run-examples-e2e.ps1 | 137 ++++++++++++++++-- tests/README.md | 15 ++ tools/FindDataParams/FindDataParams.csproj | 2 +- tools/FindLiveSession/FindLiveSession.csproj | 2 +- tools/SnippetRunner/SnippetRunner.csproj | 10 +- 8 files changed, 170 insertions(+), 26 deletions(-) create mode 100644 Directory.Build.props diff --git a/Directory.Build.props b/Directory.Build.props new file mode 100644 index 0000000..3760d50 --- /dev/null +++ b/Directory.Build.props @@ -0,0 +1,26 @@ + + + + + * + + + diff --git a/projects/csharp/FunctionLibrary/FunctionLibrary.csproj b/projects/csharp/FunctionLibrary/FunctionLibrary.csproj index 1cfe0b7..e292fb8 100644 --- a/projects/csharp/FunctionLibrary/FunctionLibrary.csproj +++ b/projects/csharp/FunctionLibrary/FunctionLibrary.csproj @@ -7,7 +7,7 @@ - + diff --git a/projects/csharp/SqlRace.Examples.Core/SqlRace.Examples.Core.csproj b/projects/csharp/SqlRace.Examples.Core/SqlRace.Examples.Core.csproj index 4a0d91b..a806d49 100644 --- a/projects/csharp/SqlRace.Examples.Core/SqlRace.Examples.Core.csproj +++ b/projects/csharp/SqlRace.Examples.Core/SqlRace.Examples.Core.csproj @@ -8,7 +8,7 @@ - + diff --git a/scripts/run-examples-e2e.ps1 b/scripts/run-examples-e2e.ps1 index 3130769..3144b87 100644 --- a/scripts/run-examples-e2e.ps1 +++ b/scripts/run-examples-e2e.ps1 @@ -41,6 +41,18 @@ param ( # ships. Empty means whatever the projects resolve normally. [string] $SqlRaceApiVersion, + # Extra NuGet source, normally a folder holding a .nupkg that has not been + # published anywhere - the package a CI build just produced, for instance. + # Passed as RestoreAdditionalProjectSources rather than added with + # "dotnet nuget add source", because the repo nuget.config starts with + # and would discard a source added at user level. + [string] $AdditionalPackageSource, + + # Hard limit per snippet. Nothing here is supposed to wait on anything, so + # a snippet still running after this has hung and is killed. Generous + # because a cold MATLAB start alone is most of a minute. + [int] $SnippetTimeoutSeconds = 180, + [switch] $SkipBuild ) @@ -56,6 +68,9 @@ try { $buildArgs = @('-c', 'Release', '-v', 'quiet') if ($SqlRaceApiVersion) { $buildArgs += "-p:SqlRaceApiVersion=$SqlRaceApiVersion" } + if ($AdditionalPackageSource) { + $buildArgs += "-p:RestoreAdditionalProjectSources=$AdditionalPackageSource" + } # ── Normalisation ──────────────────────────────────────── # @@ -106,14 +121,105 @@ try { # order snippets execute in, and of whether a previous run left anything # behind. function Reset-Fixture { - if (Test-Path $fixturePath) { Remove-Item $fixturePath -Force } - & 'tools/FixtureGenerator/bin/Release/net8.0/FixtureGenerator.exe' $fixturePath | Out-Null - if ($LASTEXITCODE -ne 0) { throw 'Fixture generation failed' } + # SQLite handles are not always released the instant a snippet process + # exits, so deleting and regenerating can lose a race with the previous + # snippet. Retry rather than failing the whole run on a transient lock. + for ($attempt = 1; $attempt -le 5; $attempt++) { + try { + if (Test-Path $fixturePath) { Remove-Item $fixturePath -Force -ErrorAction Stop } + & 'tools/FixtureGenerator/bin/Release/net8.0/FixtureGenerator.exe' $fixturePath | Out-Null + if ($LASTEXITCODE -eq 0) { return } + } + catch { + if ($attempt -eq 5) { throw "Could not reset the fixture: $($_.Exception.Message)" } + } + Start-Sleep -Milliseconds (200 * $attempt) + } + throw "Fixture generation failed after 5 attempts (file locked by a previous snippet?)" } Write-Host "Verified fixture at $fixturePath" -ForegroundColor Cyan Reset-Fixture + # Runs an external command with a hard timeout, capturing both streams. + # + # Needed because a snippet can hang rather than fail - one waiting on live + # data that never arrives, or a MATLAB session that stops for input. Without + # a timeout that wedges the entire run, and because the hung process keeps a + # handle on the fixture, every subsequent snippet fails too. Nothing is + # supposed to sit and wait here, so a snippet still running after the + # timeout is a failure by definition. + function Invoke-WithTimeout { + param ( + [Parameter(Mandatory)][string] $FilePath, + [string[]] $ArgumentList = @(), + [string] $WorkingDirectory = $repo, + [int] $TimeoutSeconds = $SnippetTimeoutSeconds + ) + + $stdout = [System.IO.Path]::GetTempFileName() + $stderr = [System.IO.Path]::GetTempFileName() + + try { + $startArgs = @{ + FilePath = $FilePath + WorkingDirectory = $WorkingDirectory + RedirectStandardOutput = $stdout + RedirectStandardError = $stderr + NoNewWindow = $true + PassThru = $true + } + if ($ArgumentList.Count -gt 0) { $startArgs.ArgumentList = $ArgumentList } + + $process = Start-Process @startArgs + + # Touching Handle caches it. Without this, Start-Process -PassThru + # hands back an object that releases the native handle on exit and + # ExitCode reads back empty - every snippet then looks like a + # failure with no exit code to explain it. + $null = $process.Handle + + if (-not $process.WaitForExit($TimeoutSeconds * 1000)) { + # Kill the tree: dotnet run spawns the snippet as a child, and + # killing only the parent would leave the child holding the + # fixture open. + try { $process.Kill($true) } catch { } + $process.WaitForExit(5000) | Out-Null + return [pscustomobject]@{ + ExitCode = -1 + TimedOut = $true + Lines = @("Timed out after $TimeoutSeconds seconds.") + } + } + + # The timed WaitForExit overload can return before the exit code is + # cached on the object, leaving ExitCode empty. The parameterless + # overload also waits for the redirected streams to be flushed, so + # this both populates ExitCode and guarantees the files are complete. + $process.WaitForExit() + + $lines = @() + foreach ($file in $stdout, $stderr) { + if ((Get-Item $file).Length -gt 0) { + # The snippets write UTF-8 (box-drawing characters in table + # rules, en dashes in ranges). Get-Content defaults to the + # ANSI codepage on Windows PowerShell, which would mangle + # them into mojibake and fail every golden comparison. + $lines += (Get-Content $file -Encoding UTF8 -ErrorAction SilentlyContinue) + } + } + + return [pscustomobject]@{ + ExitCode = $process.ExitCode + TimedOut = $false + Lines = $lines + } + } + finally { + Remove-Item $stdout, $stderr -Force -ErrorAction SilentlyContinue + } + } + # ── C# runner project ──────────────────────────────────── # # tools/SnippetRunner compiles whichever file its SnippetFile property points @@ -149,10 +255,10 @@ try { # against the project directory, not the working directory. $snippetFile = (Resolve-Path $Snippet.Path).Path - $output = & dotnet run --project $runnerProject @buildArgs ` - "-p:SnippetFile=$snippetFile" -- @Arguments 2>&1 | - ForEach-Object { $_.ToString() } - return [pscustomobject]@{ ExitCode = $LASTEXITCODE; Lines = $output } + $dotnetArgs = @('run', '--project', $runnerProject) + $buildArgs + + @("-p:SnippetFile=$snippetFile", '--') + $Arguments + + return Invoke-WithTimeout -FilePath 'dotnet' -ArgumentList $dotnetArgs } # A NuGet-restored build keeps platform-specific assets under runtimes// @@ -203,8 +309,8 @@ try { $previousEncoding = $env:PYTHONIOENCODING $env:PYTHONIOENCODING = 'utf-8' try { - $output = & python $Snippet.Path @Arguments 2>&1 | ForEach-Object { $_.ToString() } - return [pscustomobject]@{ ExitCode = $LASTEXITCODE; Lines = $output } + return Invoke-WithTimeout -FilePath 'python' ` + -ArgumentList (@((Resolve-Path $Snippet.Path).Path) + $Arguments) } finally { $env:PYTHONIOENCODING = $previousEncoding @@ -261,8 +367,7 @@ try { # error, and needs an absolute path since it does not inherit our cwd # semantics for run(). $full = (Resolve-Path $Snippet.Path).Path - $output = & matlab -batch "run('$full')" 2>&1 | ForEach-Object { $_.ToString() } - return [pscustomobject]@{ ExitCode = $LASTEXITCODE; Lines = $output } + return Invoke-WithTimeout -FilePath 'matlab' -ArgumentList @('-batch', "run('$full')") } $runners = @{ @@ -356,7 +461,13 @@ try { $status = 'PASS' $detail = '' - if ($run.ExitCode -ne 0) { + if ($run.TimedOut) { + # Distinct from a plain failure: the snippet is hung, not wrong. + # Named separately so nobody goes looking for a data mismatch. + $status = 'TIMEOUT' + $detail = "hung and was killed after $SnippetTimeoutSeconds seconds" + } + elseif ($run.ExitCode -ne 0) { $status = 'FAIL' $detail = "exit code $($run.ExitCode)" } @@ -418,7 +529,7 @@ try { ForEach-Object { "{0,-22} {1}" -f $_.Name, $_.Count } | Write-Host - $bad = @($results | Where-Object { $_.Status -in @('FAIL', 'DIFF') }) + $bad = @($results | Where-Object { $_.Status -in @('FAIL', 'DIFF', 'TIMEOUT') }) $ungolden = @($results | Where-Object { $_.Status -eq 'NOGOLD' }) $skipped = @($results | Where-Object { $_.Status -eq 'SKIP' }) diff --git a/tests/README.md b/tests/README.md index 509b9f6..4381454 100644 --- a/tests/README.md +++ b/tests/README.md @@ -36,6 +36,21 @@ dotnet test SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj -p:SqlRaceApiVe ../scripts/run-examples-e2e.ps1 -SqlRaceApiVersion 2.1.26212.6-ci ``` +If the candidate has not been published anywhere — a package a PR build just +produced, for instance — point at the folder holding the `.nupkg` instead. Nothing +needs publishing for this to work: + +```powershell +../scripts/run-examples-e2e.ps1 ` + -SqlRaceApiVersion 2.1.26212.6-ci ` + -AdditionalPackageSource C:\path\to\packages +``` + +`SqlRaceApiVersion` is defined once in `Directory.Build.props` at the repo root, so +it applies to every project at once. Overriding it on the test project alone would +leave `SqlRace.Examples.Core` and `FunctionLibrary` resolving whatever is published, +and the run would silently mix two different SQL Race builds. + The SQL Race pipeline runs both against every nightly build — see `Build/generic-steps/run-consumer-compat-tests.yml` in `MAT.OCS.SQLRace.API`. That is the counterpart to its `check-api-compatibility.yml`, which diffs the public API surface against diff --git a/tools/FindDataParams/FindDataParams.csproj b/tools/FindDataParams/FindDataParams.csproj index c02ad00..7f9a925 100644 --- a/tools/FindDataParams/FindDataParams.csproj +++ b/tools/FindDataParams/FindDataParams.csproj @@ -6,6 +6,6 @@ enable - + diff --git a/tools/FindLiveSession/FindLiveSession.csproj b/tools/FindLiveSession/FindLiveSession.csproj index 7bf73cb..256bd58 100644 --- a/tools/FindLiveSession/FindLiveSession.csproj +++ b/tools/FindLiveSession/FindLiveSession.csproj @@ -7,6 +7,6 @@ $(NoWarn);CA1416 - + diff --git a/tools/SnippetRunner/SnippetRunner.csproj b/tools/SnippetRunner/SnippetRunner.csproj index bac4616..cb99046 100644 --- a/tools/SnippetRunner/SnippetRunner.csproj +++ b/tools/SnippetRunner/SnippetRunner.csproj @@ -5,15 +5,7 @@ enable enable $(NoWarn);CA1416 - - - * + From d81d7fd97e6efcc44896c69273ffd682750da3f8 Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Tue, 25 Aug 2026 16:22:13 +0000 Subject: [PATCH 05/15] Float Roslyn, and skip the Python leg cleanly when Python is absent Two things the first CI run of the validation pipeline exposed. The test project pinned Microsoft.CodeAnalysis.CSharp 4.8.0, and restore failed with NU1605 because the candidate API required 4.9.2. That floor is not fixed: the SQL Race build rewrites its nuspec dependency versions to whatever the solution consumed, so it moves with their dependency graph. Dropping the reference to inherit the API's version fails the other way - an older release asks for 4.3.1, which has no LanguageVersion.CSharp12. 4.* resolves at or above whatever the API asks for, so a downgrade cannot happen and the language version the snippets are written in is always understood. Verified against the old API: resolves 4.14.0, builds, 34 pass and 3 fail as before. The Python leg produced seven identical "The system cannot find the file specified" failures on the agent, which says nothing about the actual cause - Python is not installed there. Test-PythonAvailable mirrors the MATLAB preflight: it reports the snippets as skipped with a reason, and separately distinguishes Python missing from pythonnet being unusable, since a missing cffi fails inside pythonnet.load() with an error that reads like a SQL Race fault. Verified both branches. --- scripts/run-examples-e2e.ps1 | 50 +++++++++++++++++++ .../SqlRace.Examples.Tests.csproj | 32 ++++++------ 2 files changed, 67 insertions(+), 15 deletions(-) diff --git a/scripts/run-examples-e2e.ps1 b/scripts/run-examples-e2e.ps1 index 3144b87..fea2db5 100644 --- a/scripts/run-examples-e2e.ps1 +++ b/scripts/run-examples-e2e.ps1 @@ -360,6 +360,44 @@ try { return $true } +# Preflight for the Python leg. Without it, a machine with no Python produces + # one identical "The system cannot find the file specified" per snippet and + # nothing that says Python is simply not installed - which is exactly how it + # first showed up on a build agent. + # + # Returns $null when Python is usable, otherwise the reason it is not. + function Test-PythonAvailable { + $python = Get-Command python -ErrorAction SilentlyContinue + if (-not $python) { + Write-Host 'Python not found on PATH - skipping the Python leg.' -ForegroundColor Yellow + Write-Host ' Install Python 3, then: pip install pythonnet cffi pandas' -ForegroundColor Yellow + return 'Python is not installed on this machine' + } + + # pythonnet needs clr_loader, which needs cffi. A missing cffi fails + # inside pythonnet.load() with a runtime error that reads like a SQL Race + # fault rather than a missing dependency. + $previousPreference = $ErrorActionPreference + $ErrorActionPreference = 'Continue' + try { + $probe = & python -c "import clr_loader, cffi" 2>&1 | ForEach-Object { $_.ToString() } + $exitCode = $LASTEXITCODE + } + finally { + $ErrorActionPreference = $previousPreference + } + + if ($exitCode -ne 0) { + Write-Host "Python is installed ($($python.Source)) but pythonnet is not usable - skipping the Python leg." -ForegroundColor Yellow + $probe | Select-Object -First 3 | ForEach-Object { Write-Host " $_" -ForegroundColor Yellow } + Write-Host ' Fix with: pip install pythonnet cffi pandas' -ForegroundColor Yellow + return 'pythonnet or its dependencies are missing' + } + + Write-Host "Python ready ($($python.Source))" -ForegroundColor DarkGray + return $null + } + function Invoke-MatlabSnippet { param ([hashtable] $Snippet, [string[]] $Arguments) @@ -406,6 +444,18 @@ try { continue } + if ($lang -eq 'Python') { + $reason = Test-PythonAvailable + if ($reason) { + foreach ($snippet in $snippets) { + $results += [pscustomobject]@{ + Language = 'Python'; Snippet = $snippet.Name; Status = 'SKIP'; Detail = $reason + } + } + continue + } + } + Write-Host "" Write-Host "########## $lang ##########" -ForegroundColor Yellow diff --git a/tests/SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj b/tests/SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj index bed8e51..1d11743 100644 --- a/tests/SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj +++ b/tests/SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj @@ -6,27 +6,29 @@ enable false true - - - * + - + + From e755fa72ac2ea1e97ca21f511c4f623631518d7a Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Wed, 26 Aug 2026 15:34:37 +0000 Subject: [PATCH 06/15] Fix the three function snippets against the real Functions API fdl-function-basic, dotnet-function-basic and dotnet-function-with-pda used a Functions API that does not exist in any released build: SetFdlCode, SetOutputParameter, AddInputParameter, SetImplementation, a two-argument Build and an enumerable FunctionBuildResults. None of the three compiled against a released MESL.SQLRace.API, which is what the new snippet compilation suite reports. Rewritten against MESL.SqlRace.Domain.Functions as it is today: - CreateFunctionDefinition takes the builder's UniqueId, not a type id plus output name, so FdlFunctionConstants/DotNetFunctionConstants are now imported from their own namespaces. - FDL source goes on the implementation definition, cast to IFdlFunctionImplementationDefinition; a .NET implementation is linked by assigning DotNetFunctionImplementationDefinition.Function. - Inputs go on InputParameterIdentifiers, outputs on OutputParameterDefinitions as FunctionOutputParameterDefinition. - Build takes the definition alone and returns FunctionBuildResults, whose Errors carry ErrorText. - IDotNetFunction is Initialize(IFunctionManager) plus Execute(IExecutionContext) over a batch, not a per-sample Execute with input/output dictionaries. The .NET snippets now follow projects/csharp/FunctionLibrary, so the snippet and the buildable project demonstrate the same pattern. FDL output parameters are published as ":", and a built function's outputs are added to sessions as they load, so the FDL snippet builds before it loads and reads "TemperatureFahrenheit:Functions". The compilation step in CI no longer needs continue-on-error, and the manifest's skip reasons for these three are updated to the real ones. Verified: 37/37 SnippetCompilation tests pass against 2.1.26050.4, 2.1.26169.4 and 2.1.26212.6. Co-Authored-By: Claude Opus 5 --- .github/workflows/build-and-test.yml | 4 - scripts/snippet-manifest.psd1 | 6 +- .../csharp/functions/dotnet-function-basic.cs | 96 ++++++++++------ .../functions/dotnet-function-with-pda.cs | 103 +++++++++++------- .../csharp/functions/fdl-function-basic.cs | 68 ++++++++---- 5 files changed, 169 insertions(+), 108 deletions(-) diff --git a/.github/workflows/build-and-test.yml b/.github/workflows/build-and-test.yml index a33ab5e..6c9e9a1 100644 --- a/.github/workflows/build-and-test.yml +++ b/.github/workflows/build-and-test.yml @@ -47,10 +47,6 @@ jobs: # operation needing no licence. They are the early warning that a new # MESL.SQLRace.API release has broken the code we hand customers. - name: Verify snippets compile against SQL Race - # Advisory for now - three function snippets have a known pre-existing - # failure. Remove continue-on-error once those are fixed so that a genuine - # API break fails the build. - continue-on-error: true run: > dotnet test tests/SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj --no-build --configuration Release diff --git a/scripts/snippet-manifest.psd1 b/scripts/snippet-manifest.psd1 index cba2f56..5fa7d82 100644 --- a/scripts/snippet-manifest.psd1 +++ b/scripts/snippet-manifest.psd1 @@ -35,9 +35,9 @@ @{ Name = 'read-subsampled-data'; Path = 'snippets\csharp\data-access\read-subsampled-data.cs'; Args = @('{fixtureKey}') } @{ Name = 'reverse-iteration'; Path = 'snippets\csharp\data-access\reverse-iteration.cs'; Args = @('{fixtureKey}') } @{ Name = 'timestamp-array-construction'; Path = 'snippets\csharp\data-access\timestamp-array-construction.cs' } - @{ Name = 'dotnet-function-basic'; Path = 'snippets\csharp\functions\dotnet-function-basic.cs'; Skip = 'does not compile against any shipped release - see SnippetCompilationTests' } - @{ Name = 'dotnet-function-with-pda'; Path = 'snippets\csharp\functions\dotnet-function-with-pda.cs'; Skip = 'does not compile against any shipped release - see SnippetCompilationTests' } - @{ Name = 'fdl-function-basic'; Path = 'snippets\csharp\functions\fdl-function-basic.cs'; Skip = 'does not compile against any shipped release - see SnippetCompilationTests' } + @{ Name = 'dotnet-function-basic'; Path = 'snippets\csharp\functions\dotnet-function-basic.cs'; Skip = 'declares a function class rather than running one - needs a MEF host and the .NET Functions licence option' } + @{ Name = 'dotnet-function-with-pda'; Path = 'snippets\csharp\functions\dotnet-function-with-pda.cs'; Skip = 'declares a function class rather than running one - needs a MEF host and the .NET Functions licence option' } + @{ Name = 'fdl-function-basic'; Path = 'snippets\csharp\functions\fdl-function-basic.cs'; Skip = 'needs the ATLAS Functions licence option, which the fixture runner does not have' } @{ Name = '01-load-session-from-file'; Path = 'snippets\csharp\getting-started\01-load-session-from-file.cs'; Args = @('{fixturePath}') } @{ Name = '02-load-session-from-database'; Path = 'snippets\csharp\getting-started\02-load-session-from-database.cs'; Skip = 'needs a SQL Server instance, not the SQLite fixture' } @{ Name = '03-read-parameter-samples'; Path = 'snippets\csharp\getting-started\03-read-parameter-samples.cs'; Args = @('{fixtureKey}') } diff --git a/snippets/csharp/functions/dotnet-function-basic.cs b/snippets/csharp/functions/dotnet-function-basic.cs index f334665..abc523b 100644 --- a/snippets/csharp/functions/dotnet-function-basic.cs +++ b/snippets/csharp/functions/dotnet-function-basic.cs @@ -6,19 +6,19 @@ // that doubles an input value // Prerequisites: MESL.SQLRace.API NuGet package, .NET 8 // Input: A session with a Temperature:Sensors parameter -// Output: Doubled values from the .NET function output +// Output: Doubled values published as DoubledTemperature:Functions // // Related: snippets/csharp/functions/fdl-function-basic.cs // Docs: https://mat-docs.github.io/Atlas.SQLRaceAPI.Documentation/api/index.html // ───────────────────────────────────────────────────────────── -// NOTE: This file shows the complete pattern for a .NET function. -// In production, the function class would be in a separate assembly -// loaded by the function manager via MEF [Export] attributes. +// NOTE: This file shows the complete pattern for a .NET function. In production +// the class would live in a separate assembly, discovered via MEF [Export]. using System.ComponentModel.Composition; -using MESL.SqlRace.Domain; using MESL.SqlRace.Domain.Functions; +using MESL.SqlRace.Domain.Functions.DotNet; +using MESL.SqlRace.Enumerators; Console.WriteLine("DoubleValueFunction defined."); Console.WriteLine("See fdl-function-basic.cs for a complete runnable example."); @@ -26,49 +26,75 @@ // --- The function implementation --- [Export(typeof(IDotNetFunction))] +[Serializable] public class DoubleValueFunction : IDotNetFunction { - private const string InputParam = "Temperature:Sensors"; - private const string OutputParam = "DoubledTemperature:Calculated"; + private const string InputParameter = "Temperature:Sensors"; + private const string OutputName = "DoubledTemperature"; + private const string OutputGroup = "Functions"; + private const string OutputIdentifier = OutputName + ":" + OutputGroup; - public IFunctionDefinition? Definition { get; private set; } + // Diagnostics only; keep it the same as the definition's name. + public string Name => "DoubleValue"; - public void Initialize(IFunctionManager functionManager, Session session) + // Called once by the function manager to declare what this function + // consumes, what it publishes, and where its code lives. + public void Initialize(IFunctionManager functionManager) { - // --- Create the function definition --- - Definition = functionManager.CreateFunctionDefinition( - DotNetFunctionConstants.DotNetFunctionTypeId, - OutputParam, - "Temperature doubled for testing"); + var definition = functionManager.CreateFunctionDefinition(DotNetFunctionConstants.UniqueId); + definition.Name = this.Name; + definition.CalculateOverWholeSession = false; + definition.InterpolateBetweenSamples = false; + definition.JoinGapsAroundNull = false; + definition.StoreInSession = false; - Definition.AddInputParameter(InputParam); - Definition.SetImplementation(this); - Definition.SetOutputParameter(OutputParam, "degC", "%5.1f", 0, 1000); + // Link the definition to this implementation; without it the framework + // has no Execute() body to invoke. + var implementation = (DotNetFunctionImplementationDefinition)definition.ImplementationDefinition; + implementation.Function = this; - var errors = functionManager.Build(Definition, session); - if (errors?.Any() == true) - throw new InvalidOperationException($"Function build failed: {string.Join(", ", errors)}"); + definition.InputParameterIdentifiers.Add(InputParameter); + + definition.OutputParameterDefinitions.Add(new FunctionOutputParameterDefinition + { + Name = OutputName, + ApplicationName = OutputGroup, + Description = "Temperature doubled for testing", + Units = "degC", + FormatOverride = "%5.1f", + ByteOrder = ByteOrderType.BigEndian, + MinimumValue = "0", + MaximumValue = "1000", + ShowInBrowser = true, + }); + + var buildResults = functionManager.Build(definition); + if (buildResults.Errors.Count > 0) + { + throw new InvalidOperationException( + $"Function build failed: {buildResults.Errors[0].ErrorText}"); + } } - public void Execute( - IFunctionContext context, - long timestamp, - IDictionary inputValues, - IDictionary outputValues) + // Called once per data request with a batch of samples. Inputs and outputs + // are aligned on the request's timebase, so they share indexing. + public void Execute(IExecutionContext context) { - if (inputValues.TryGetValue(InputParam, out var input)) + var sampleCount = context.FunctionInput.Timestamps.Length; + var inputIndex = context.FunctionInput.InputParameterIndexes[InputParameter]; + var outputIndex = context.FunctionOutput.OutputParameterIndexes[OutputIdentifier]; + + var input = context.FunctionInput.Values[inputIndex]; + var output = context.FunctionOutput.OutputParametersValues[outputIndex]; + + for (var i = 0; i < sampleCount; i++) { - outputValues[OutputParam] = input * 2.0; + output[i] = input[i] * 2.0; } } } -// --- Usage (host application) --- -// NOTE: In practice, function assemblies are discovered via MEF. -// This snippet shows the manual pattern for clarity. +// --- Usage (host application), shown manually for clarity --- // var functionManager = FunctionManagerFactory.Create(); -// var func = new DoubleValueFunction(); -// func.Initialize(functionManager, session); -// using var pda = session.CreateParameterDataAccess("DoubledTemperature:Calculated"); - - +// new DoubleValueFunction().Initialize(functionManager); +// using var pda = session.CreateParameterDataAccess("DoubledTemperature:Functions"); diff --git a/snippets/csharp/functions/dotnet-function-with-pda.cs b/snippets/csharp/functions/dotnet-function-with-pda.cs index 128b495..51c3c4d 100644 --- a/snippets/csharp/functions/dotnet-function-with-pda.cs +++ b/snippets/csharp/functions/dotnet-function-with-pda.cs @@ -6,76 +6,95 @@ // inside Execute for cross-parameter lookups // Prerequisites: MESL.SQLRace.API NuGet package, .NET 8 // Input: A session with Temperature:Sensors and Pressure:Inlet parameters -// Output: Computed values using data from an additional parameter +// Output: Computed values published as CorrectedDensity:Functions // // Related: snippets/csharp/functions/dotnet-function-basic.cs // Docs: https://mat-docs.github.io/Atlas.SQLRaceAPI.Documentation/api/index.html // ───────────────────────────────────────────────────────────── -// NOTE: This is an advanced pattern. The function reads an additional parameter -// via its own PDA to perform cross-parameter calculations (e.g., density -// correction using both temperature and pressure). +// NOTE: An advanced pattern. Prefer declaring every parameter you need as an +// input (see dotnet-function-basic.cs) - the framework aligns those for you. +// Reach for a PDA only to sample a parameter off the request's own timebase. using System.ComponentModel.Composition; using MESL.SqlRace.Domain; using MESL.SqlRace.Domain.Functions; +using MESL.SqlRace.Domain.Functions.DotNet; +using MESL.SqlRace.Enumerators; Console.WriteLine("DensityCorrectionFunction defined (Temperature + Pressure → Density)."); -Console.WriteLine("Uses lazy-initialised PDA for cross-parameter lookup in Execute()."); +Console.WriteLine("Uses a lazily-created PDA for cross-parameter lookup in Execute()."); Console.WriteLine("See dotnet-function-basic.cs for the simpler single-input pattern."); [Export(typeof(IDotNetFunction))] +[Serializable] public class DensityCorrectionFunction : IDotNetFunction { - private const string InputTemp = "Temperature:Sensors"; + private const string InputTemperature = "Temperature:Sensors"; private const string LookupPressure = "Pressure:Inlet"; - private const string OutputParam = "CorrectedDensity:Calculated"; + private const string OutputName = "CorrectedDensity"; + private const string OutputGroup = "Functions"; + private const string OutputIdentifier = OutputName + ":" + OutputGroup; + // A PDA belongs to one session, so it cannot travel with a serialised function. [NonSerialized] - private ParameterDataAccessBase? _pressurePda; - private Session? _session; + private ParameterDataAccessBase? pressurePda; - public IFunctionDefinition? Definition { get; private set; } + public string Name => "DensityCorrection"; - public void Initialize(IFunctionManager functionManager, Session session) + public void Initialize(IFunctionManager functionManager) { - _session = session; - - Definition = functionManager.CreateFunctionDefinition( - DotNetFunctionConstants.DotNetFunctionTypeId, - OutputParam, - "Density corrected for temperature and pressure"); - - Definition.AddInputParameter(InputTemp); - Definition.SetImplementation(this); - Definition.SetOutputParameter(OutputParam, "kg/m3", "%6.2f", 0, 2000); - - var errors = functionManager.Build(Definition, session); - if (errors?.Any() == true) - throw new InvalidOperationException($"Build failed: {string.Join(", ", errors)}"); + var definition = functionManager.CreateFunctionDefinition(DotNetFunctionConstants.UniqueId); + definition.Name = this.Name; + definition.CalculateOverWholeSession = false; + definition.InterpolateBetweenSamples = false; + definition.JoinGapsAroundNull = false; + definition.StoreInSession = false; + + // Only temperature is a declared input; pressure is read via the PDA below. + var implementation = (DotNetFunctionImplementationDefinition)definition.ImplementationDefinition; + implementation.Function = this; + definition.InputParameterIdentifiers.Add(InputTemperature); + + definition.OutputParameterDefinitions.Add(new FunctionOutputParameterDefinition + { + Name = OutputName, + ApplicationName = OutputGroup, + Description = "Density corrected for temperature and pressure", + Units = "kg/m3", + FormatOverride = "%6.2f", + ByteOrder = ByteOrderType.BigEndian, + MinimumValue = "0", + MaximumValue = "2000", + ShowInBrowser = true, + }); + + var buildResults = functionManager.Build(definition); + if (buildResults.Errors.Count > 0) + throw new InvalidOperationException($"Build failed: {buildResults.Errors[0].ErrorText}"); } - public void Execute( - IFunctionContext context, - long timestamp, - IDictionary inputValues, - IDictionary outputValues) + public void Execute(IExecutionContext context) { - // --- Lazy-initialise the pressure PDA --- - _pressurePda ??= _session!.CreateParameterDataAccess(LookupPressure); + // --- Lazily create the pressure PDA against the session being calculated --- + this.pressurePda ??= context.Session.CreateParameterDataAccess(LookupPressure); - if (!inputValues.TryGetValue(InputTemp, out var tempC)) - return; + var timestamps = context.FunctionInput.Timestamps; + var temperatureIndex = context.FunctionInput.InputParameterIndexes[InputTemperature]; + var outputIndex = context.FunctionOutput.OutputParameterIndexes[OutputIdentifier]; - // --- Read pressure at this timestamp --- - var pressureSamples = _pressurePda.GetData(new[] { timestamp }); - var pressureBar = pressureSamples.SampleCount > 0 ? pressureSamples.Data[0] : 1.013; + var temperature = context.FunctionInput.Values[temperatureIndex]; + var output = context.FunctionOutput.OutputParametersValues[outputIndex]; + + // One batched read on the request's timestamps, not one read per sample. + var pressure = this.pressurePda.GetData(timestamps); // Ideal gas approximation: density ∝ P / T (simplified) - var tempK = tempC + 273.15; - var density = (pressureBar * 100_000) / (287.05 * tempK); - outputValues[OutputParam] = density; + for (var i = 0; i < timestamps.Length; i++) + { + var pressureBar = i < pressure.SampleCount ? pressure.Data[i] : 1.013; + var tempK = temperature[i] + 273.15; + output[i] = pressureBar * 100_000.0 / (287.05 * tempK); + } } } - - diff --git a/snippets/csharp/functions/fdl-function-basic.cs b/snippets/csharp/functions/fdl-function-basic.cs index d475df8..886ab59 100644 --- a/snippets/csharp/functions/fdl-function-basic.cs +++ b/snippets/csharp/functions/fdl-function-basic.cs @@ -2,11 +2,12 @@ // SQL Race Example: FDL Function (Basic) // ───────────────────────────────────────────────────────────── // -// Demonstrates: Creating an FDL function (Celsius to Fahrenheit), building it, -// and consuming the output via a PDA -// Prerequisites: MESL.SQLRace.API NuGet package, .NET 8, a session with data -// Input: A session with a Temperature:Sensors parameter -// Output: Converted values read from the FDL function output +// Demonstrates: Building an FDL function (Celsius to Fahrenheit) and reading +// its output parameter back through a PDA +// Prerequisites: MESL.SQLRace.API NuGet package, .NET 8, the "ATLAS Functions" +// licence option, a session with a Temperature:Sensors parameter +// Input: A session GUID +// Output: Converted values read from the FDL function's output parameter // // Related: snippets/csharp/functions/dotnet-function-basic.cs // Docs: https://mat-docs.github.io/Atlas.SQLRaceAPI.Documentation/api/index.html @@ -15,6 +16,8 @@ using MAT.OCS.Core; using MESL.SqlRace.Domain; using MESL.SqlRace.Domain.Functions; +using MESL.SqlRace.Domain.Functions.Fdl; +using MESL.SqlRace.Functions.Interfaces.Enums; Core.LicenceProgramName = "SQLRace"; Core.Initialize(); @@ -29,40 +32,57 @@ } var sessionKey = SessionKey.Parse(args[0]); -var sessionManager = SessionManager.CreateSessionManager(); -using var clientSession = sessionManager.Load(sessionKey, connectionString); -var session = clientSession.Session; -// --- Create the FDL function: Celsius → Fahrenheit --- +// --- Define the FDL function: Celsius → Fahrenheit --- +// An output parameter's identifier is ":", so this +// function publishes "TemperatureFahrenheit:Functions". +const string FunctionName = "TemperatureFahrenheit"; +const string OutputGroup = "Functions"; +const string OutputIdentifier = $"{FunctionName}:{OutputGroup}"; +const string FdlCode = "return ($Temperature:Sensors * 1.8 + 32);"; + var functionManager = FunctionManagerFactory.Create(); -var fdlCode = "return ($Temperature:Sensors * 1.8 + 32)"; -var outputIdentifier = "TemperatureFahrenheit:Calculated"; -var funcDef = functionManager.CreateFunctionDefinition( - FdlFunctionConstants.FdlFunctionTypeId, - outputIdentifier, - "Temperature in Fahrenheit"); +var definition = functionManager.CreateFunctionDefinition(FdlFunctionConstants.UniqueId); +definition.Name = FunctionName; +definition.FunctionMode = FunctionMode.Instantaneous; + +// The FDL source lives on the implementation definition, which the function +// manager creates alongside the definition itself. +var implementation = (IFdlFunctionImplementationDefinition)definition.ImplementationDefinition; +implementation.FunctionCode = FdlCode; -funcDef.SetFdlCode(fdlCode); -funcDef.SetOutputParameter(outputIdentifier, "degF", "%5.1f", -40, 1000); +// FDL functions require exactly one output parameter - the value they return. +definition.OutputParameterDefinitions.Add( + FunctionOutputParameterDefinition.Create(FunctionName, OutputGroup, "Temperature in Fahrenheit") + .Units("degF") + .FormatOverride("%5.1f") + .MinimumValue("-40.0") + .MaximumValue("1000.0")); -var errors = functionManager.Build(funcDef, session); +var buildResults = functionManager.Build(definition); -if (errors is not null && errors.Any()) +if (buildResults.Errors.Count > 0) { Console.WriteLine("FDL build errors:"); - foreach (var error in errors) - Console.WriteLine($" {error}"); + foreach (var error in buildResults.Errors) + Console.WriteLine($" {error.ErrorText}"); return; } -Console.WriteLine($"FDL function built: {fdlCode}"); +Console.WriteLine($"FDL function built: {FdlCode}"); // --- Read the function output --- -using var pda = session.CreateParameterDataAccess(outputIdentifier); +// Load the session *after* the build: SQL Race adds a built function's output +// parameters to sessions as they load. +var sessionManager = SessionManager.CreateSessionManager(); +using var clientSession = sessionManager.Load(sessionKey, connectionString); +var session = clientSession.Session; + +using var pda = session.CreateParameterDataAccess(OutputIdentifier); var samples = pda.GetSamplesBetween(session.StartTime, session.EndTime); -Console.WriteLine($"Output: {samples.SampleCount} samples of {outputIdentifier}\n"); +Console.WriteLine($"Output: {samples.SampleCount} samples of {OutputIdentifier}\n"); for (var i = 0; i < Math.Min(samples.SampleCount, 10); i++) { From c4abf11086da91501bea3170fe5499047425eaf0 Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Wed, 26 Aug 2026 15:40:51 +0000 Subject: [PATCH 07/15] Import the root props from projects/csharp, so SqlRaceApiVersion applies MSBuild stops at the first Directory.Build.props it finds walking up from a project, so the repo-root props - which define SqlRaceApiVersion - did not reach the projects under projects/csharp. Their package reference is Version="$(SqlRaceApiVersion)", so it expanded to an empty version range. NuGet resolves that to the oldest MESL.SQLRace.API available rather than treating it as an error, which left FunctionLibrary and SqlRace.Examples.Core on a different build from the test project. That is the mixed-build case SqlRaceApiVersion exists to avoid, and it would make a compatibility run report on the wrong build. It showed up as an NU1604 restore error only because projects/csharp also sets TreatWarningsAsErrors. With the import, all three projects resolve the same build: dotnet test -p:SqlRaceApiVersion=2.1.26169.4 -> 2.1.26169.4 for FunctionLibrary, SqlRace.Examples.Core and the tests Co-Authored-By: Claude Opus 5 --- projects/csharp/Directory.Build.props | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/projects/csharp/Directory.Build.props b/projects/csharp/Directory.Build.props index ab7c948..76015a1 100644 --- a/projects/csharp/Directory.Build.props +++ b/projects/csharp/Directory.Build.props @@ -1,5 +1,18 @@ + + + net8.0 enable From 67cd0c0735c64bab3e8896af5c36ced7805a7ed4 Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Wed, 26 Aug 2026 15:47:26 +0000 Subject: [PATCH 08/15] Match the tone of the tests and CI config to the snippets This repository is public, so its comments, docs and CI configuration are read by the same people who read the snippets. A few of them were written for an internal audience. - tests/README.md pointed at build pipeline files by path in another repository. Replaced with the part a reader here can use: that these suites run against SQL Race builds before they are published. - A prerelease build number was the worked example for -SqlRaceApiVersion in five places. It is now a placeholder, which also does not go stale. - A few comments referred to the reader in the third person. Reworded to address them directly. No behaviour change; the snippets themselves were already clean. Co-Authored-By: Claude Opus 5 --- .github/workflows/build-and-test.yml | 2 +- Directory.Build.props | 4 ++-- scripts/run-examples-e2e.ps1 | 2 +- tests/README.md | 18 ++++++++---------- .../SnippetCompilationTests.cs | 13 ++++++------- 5 files changed, 18 insertions(+), 21 deletions(-) diff --git a/.github/workflows/build-and-test.yml b/.github/workflows/build-and-test.yml index 6c9e9a1..216e284 100644 --- a/.github/workflows/build-and-test.yml +++ b/.github/workflows/build-and-test.yml @@ -45,7 +45,7 @@ jobs: # The snippet compilation tests are the exception: they compile every snippet # against the real SQL Race assemblies using Roslyn, which is a metadata-only # operation needing no licence. They are the early warning that a new - # MESL.SQLRace.API release has broken the code we hand customers. + # MESL.SQLRace.API release has broken the examples. - name: Verify snippets compile against SQL Race run: > dotnet test tests/SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj diff --git a/Directory.Build.props b/Directory.Build.props index 3760d50..0202b29 100644 --- a/Directory.Build.props +++ b/Directory.Build.props @@ -8,8 +8,8 @@ development wants. A compatibility run overrides it to point at a candidate before that candidate ships: - dotnet test -p:SqlRaceApiVersion=2.1.26212.6-ci - ./scripts/run-examples-e2e.ps1 -SqlRaceApiVersion 2.1.26212.6-ci + dotnet test -p:SqlRaceApiVersion= + ./scripts/run-examples-e2e.ps1 -SqlRaceApiVersion It is defined here rather than per-project because it has to apply to every project at once. Overriding it on the test project alone leaves diff --git a/scripts/run-examples-e2e.ps1 b/scripts/run-examples-e2e.ps1 index fea2db5..bd815d5 100644 --- a/scripts/run-examples-e2e.ps1 +++ b/scripts/run-examples-e2e.ps1 @@ -22,7 +22,7 @@ # ./scripts/run-examples-e2e.ps1 # ./scripts/run-examples-e2e.ps1 -Language CSharp # ./scripts/run-examples-e2e.ps1 -Filter '*read*' -# ./scripts/run-examples-e2e.ps1 -SqlRaceApiVersion 2.1.26212.6-ci +# ./scripts/run-examples-e2e.ps1 -SqlRaceApiVersion # ./scripts/run-examples-e2e.ps1 -UpdateGolden # after an intended change # ───────────────────────────────────────────────────────────── [CmdletBinding()] diff --git a/tests/README.md b/tests/README.md index 4381454..78e4291 100644 --- a/tests/README.md +++ b/tests/README.md @@ -19,21 +19,21 @@ correct outputs are known — so the examples are checked for *correctness*, not ## Breaking-change early warning -These pieces exist to answer one question before a release ships: *will the new -`MESL.SQLRace.API` break code customers already have?* Three layers, each catching something +These pieces exist to answer one question: *will a new `MESL.SQLRace.API` release +break code written against the previous one?* Three layers, each catching something the previous one cannot. | Layer | Catches | Licence? | |-------|---------|----------| -| `SnippetCompilationTests` | A snippet that no longer compiles — the same errors a customer would hit | No | +| `SnippetCompilationTests` | A snippet that no longer compiles — the same errors you would hit copying it | No | | `run-examples-e2e.ps1` golden comparison | A snippet that still compiles and runs but now returns different data | Yes | | Its Python and MATLAB legs | Renames and signature changes a C# compile would catch but which fail silently for `pythonnet` and `NET.addAssembly` callers | Yes | To validate a candidate build before it is published, pin the version: ```powershell -dotnet test SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj -p:SqlRaceApiVersion=2.1.26212.6-ci -../scripts/run-examples-e2e.ps1 -SqlRaceApiVersion 2.1.26212.6-ci +dotnet test SqlRace.Examples.Tests/SqlRace.Examples.Tests.csproj -p:SqlRaceApiVersion= +../scripts/run-examples-e2e.ps1 -SqlRaceApiVersion ``` If the candidate has not been published anywhere — a package a PR build just @@ -42,7 +42,7 @@ needs publishing for this to work: ```powershell ../scripts/run-examples-e2e.ps1 ` - -SqlRaceApiVersion 2.1.26212.6-ci ` + -SqlRaceApiVersion ` -AdditionalPackageSource C:\path\to\packages ``` @@ -51,10 +51,8 @@ it applies to every project at once. Overriding it on the test project alone wou leave `SqlRace.Examples.Core` and `FunctionLibrary` resolving whatever is published, and the run would silently mix two different SQL Race builds. -The SQL Race pipeline runs both against every nightly build — see -`Build/generic-steps/run-consumer-compat-tests.yml` in `MAT.OCS.SQLRace.API`. That is the -counterpart to its `check-api-compatibility.yml`, which diffs the public API surface against -the last published package. +Both layers are also run automatically against SQL Race builds before they are +published, so a break in the examples is caught alongside a break in the API itself. ### Golden output diff --git a/tests/SqlRace.Examples.Tests/SnippetCompilationTests.cs b/tests/SqlRace.Examples.Tests/SnippetCompilationTests.cs index ab769f3..1177414 100644 --- a/tests/SqlRace.Examples.Tests/SnippetCompilationTests.cs +++ b/tests/SqlRace.Examples.Tests/SnippetCompilationTests.cs @@ -6,11 +6,10 @@ // compiles each one against the real SQL Race assemblies using // Roslyn. // -// This is the early-warning system for API breaks. Snippets are -// the code we hand customers, so a snippet that stops compiling -// against a new MESL.SQLRace.API release is a customer whose -// script stops compiling too - and we want to know before the -// release ships, not after. +// This is the early-warning system for API breaks. A snippet +// that stops compiling against a new MESL.SQLRace.API release is +// a script written against the old one that stops compiling too, +// so the break is worth catching before the release, not after. // // Compilation is metadata-only: nothing here initialises the SQL // Race runtime, so no licence is needed and this suite runs on a @@ -18,7 +17,7 @@ // covered by scripts/run-examples-e2e.ps1, which does need one. // // To test a specific API build: -// dotnet test -p:SqlRaceApiVersion=2.1.26212.6-ci +// dotnet test -p:SqlRaceApiVersion= // // Run: dotnet test --filter "FullyQualifiedName~SnippetCompilation" // ───────────────────────────────────────────────────────────── @@ -147,7 +146,7 @@ public void Snippet_CompilesAgainstSqlRaceAssemblies(string relativePath, string Assert.True(errors.Count == 0, $"{relativePath} no longer compiles against {DescribeSqlRaceVersion()}. " + - "A customer copying this snippet would hit the same errors:\n" + + "Code copied from this snippet would hit the same errors:\n" + string.Join("\n", errors.Select(d => $" Line {d.Location.GetLineSpan().StartLinePosition.Line + 1}: {d.Id}: {d.GetMessage()}"))); } From 2266e7d3dddb7d92df94e1864774a64a3937d6d0 Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Thu, 27 Aug 2026 13:51:06 +0000 Subject: [PATCH 09/15] Fix the two event snippets against the real Events API Both failed in the pipeline run. They fail for unrelated reasons. 04_extract_events.py used five members that do not exist. EventCollection exposes a public EventDefinitions property that is assigned null in the constructor and never populated anywhere in the API, so session.Events. EventDefinitions is always null; the populated collection is session. EventDefinitions. The rest were wrong names: Name -> Description, Values -> RawData, Timestamp -> TimeStamp, EventDefinitionName -> EventDefinitionKey plus GroupName. Python binds by name at runtime, so none of this was visible until the snippet was actually executed. event-definitions-and-data.cs got two things wrong. An EventDefinition's fourth argument lists conversion function names, not field labels, and each must be registered with AddConversion first - Commit() drops any definition whose conversions are missing, logging an error rather than throwing, so the failure surfaced later as "event definition not found" from AddEventData. Separately, events are stored in time segments, so a session with no channel data has StartTime 0 and discards every event without raising anything. The snippet now writes a parameter and 250 samples first. Verified by compiling and running both against a local build: the C# snippet emits all four events in time order, stably across three runs. Events are also sorted explicitly, because they come back in segment order. The Python snippet runs clean but prints an empty table: events do not survive a session reload - parameters do - so a loaded fixture session can never carry any. It will report NOGOLD until that is resolved. --- .../event-definitions-and-data.cs | 43 +++++++++++++++++-- .../getting-started/04_extract_events.py | 12 +++--- 2 files changed, 46 insertions(+), 9 deletions(-) diff --git a/snippets/csharp/session-management/event-definitions-and-data.cs b/snippets/csharp/session-management/event-definitions-and-data.cs index 72b3d43..2dc71e0 100644 --- a/snippets/csharp/session-management/event-definitions-and-data.cs +++ b/snippets/csharp/session-management/event-definitions-and-data.cs @@ -6,8 +6,12 @@ // Prerequisites: MESL.SQLRace.API NuGet package, .NET 8 // Input: None (creates its own session) // Output: Event definitions structure and API usage patterns -// Note: event definitions are persisted on ConfigurationSet.Commit(); on a brand-new -// SQLite session you may need at least one committed parameter/channel first. +// Note: two things are easy to get wrong here. An EventDefinition's conversion +// function names must be registered with AddConversion before Commit(), and +// the session needs channel data before it will accept any event. Neither +// failure raises anything - the events are simply discarded. +// Events are also readable only within the recording session; they do not +// survive a reload, whereas parameters do. // // Related: snippets/csharp/session-management/marker-management.cs // Docs: https://mat-docs.github.io/Atlas.SQLRaceAPI.Documentation/api/index.html @@ -15,6 +19,7 @@ using MAT.OCS.Core; using MESL.SqlRace.Domain; +using MESL.SqlRace.Domain.Infrastructure.DataPipeline; using MESL.SqlRace.Enumerators; Core.LicenceProgramName = "SQLRace"; @@ -31,11 +36,42 @@ // --- Define event types (via configuration) --- var config = session.CreateConfiguration(); + +// The fourth EventDefinition argument lists CONVERSION FUNCTION names, not field +// labels, and each must be registered on the same configuration. Commit() drops +// any definition whose conversions are missing - it logs an error but does not +// throw, so the failure surfaces later as "event definition not found" from +// AddEventData. +foreach (var conversionName in new[] { "Value", "Threshold", "SensorId", "FaultCode", "Parameter", "Direction" }) +{ + config.AddConversion(RationalConversion.CreateSimple1To1Conversion(conversionName, "", "%5.2f")); +} + +// Events are stored in time segments, so the session needs a time base before it +// can accept any. A session with no channel data has StartTime 0, and AddEventData +// then discards the event without raising anything. +uint signalChannelId = 1; +config.AddConversion(RationalConversion.CreateSimple1To1Conversion("signal_conv", "", "%5.2f")); +config.AddChannel(new Channel(signalChannelId, "Signal", new Frequency(100, FrequencyUnit.Hz).ToInterval(), + DataType.Double64Bit, ChannelDataSourceType.Periodic, "Signal:Diagnostics", false)); +config.AddParameterGroup(new ParameterGroup("Diag", "Diag")); +config.AddGroup(new ApplicationGroup("Diagnostics", "Diagnostics", new List { "Diag" }) { SupportsRda = false }); +config.AddParameter(new Parameter( + "Signal:Diagnostics", "Signal", "Signal being monitored", + 100, 0, 100, 0, 0.0, 0u, 0u, "signal_conv", + new List { "Diag" }, signalChannelId, "Diagnostics")); + config.AddEventDefinition(new EventDefinition(1, "LimitExceedance", EventPriorityType.High, new List { "Value", "Threshold" }, "Diagnostics")); config.AddEventDefinition(new EventDefinition(2, "SensorFault", EventPriorityType.Medium, new List { "SensorId", "FaultCode" }, "Diagnostics")); config.AddEventDefinition(new EventDefinition(3, "ThresholdCrossing", EventPriorityType.Low, new List { "Parameter", "Direction" }, "Diagnostics")); config.Commit(); +// --- Give the session a time base --- +for (var i = 0; i < 250; i++) +{ + session.AddChannelData(signalChannelId, baseTime + i * 100_000_000L, 1, BitConverter.GetBytes(20.0 + (i * 0.1))); +} + // --- Add event instances --- session.Events.AddEventData(1, "Diagnostics", baseTime + 5_000_000_000L, new List { 127.3, 120.0 }); session.Events.AddEventData(2, "Diagnostics", baseTime + 10_000_000_000L, new List { 3.0, 1.0 }); @@ -49,7 +85,8 @@ Console.WriteLine($"{"Timestamp (ns)",20} {"DefKey",-10} {"Group",-15} {"Data"}"); Console.WriteLine(new string('─', 80)); -foreach (var evt in events) +// Events come back in segment order, not time order. +foreach (var evt in events.OrderBy(e => e.TimeStamp)) { var dataStr = string.Join(", ", evt.RawData); Console.WriteLine($"{evt.TimeStamp,20} {evt.EventDefinitionKey,-10} {evt.GroupName,-15} [{dataStr}]"); diff --git a/snippets/python/getting-started/04_extract_events.py b/snippets/python/getting-started/04_extract_events.py index b61bc9c..8c87909 100644 --- a/snippets/python/getting-started/04_extract_events.py +++ b/snippets/python/getting-started/04_extract_events.py @@ -66,19 +66,19 @@ # --- Event definitions → Python dict --- definitions = {} - for evt_def in session.Events.EventDefinitions: - definitions[evt_def.Name] = evt_def.Priority - print(f"Definition: {evt_def.Name} (priority: {evt_def.Priority})") + for evt_def in session.EventDefinitions: + definitions[evt_def.Description] = evt_def.Priority + print(f"Definition: {evt_def.Description} (priority: {evt_def.Priority})") # --- Event data --- events = session.Events.GetEventData(session.StartTime, session.EndTime) - print(f"\n{'Timestamp (ns)':>20} {'Event':<22} {'Data'}") + print(f"\n{'Timestamp (ns)':>20} {'Key':<5} {'Group':<16} {'Data'}") print("─" * 70) for evt in events: - values = [str(evt.Values[i]) for i in range(evt.Values.Count)] if evt.Values else [] + values = [str(evt.RawData[i]) for i in range(evt.RawData.Count)] if evt.RawData else [] data_str = ", ".join(values) - print(f"{evt.Timestamp:>20} {evt.EventDefinitionName:<22} [{data_str}]") + print(f"{evt.TimeStamp:>20} {evt.EventDefinitionKey:<5} {evt.GroupName:<16} [{data_str}]") finally: client_session.Dispose() From c679bdee33a21eca33aa25ab723f0f6643db060b Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Thu, 27 Aug 2026 14:52:15 +0000 Subject: [PATCH 10/15] Skip the Python events snippet rather than rewriting it It no longer crashes, but it verifies nothing: event data does not survive a session reload, so the loaded fixture session it is pointed at can never carry any. The loop body never runs, which leaves most of the member names just corrected unexercised, and the golden would be an empty table. A skip with a stated reason is more honest than a pass over nothing. Making it demonstrate anything means having it create its own session with events, which is a rewrite of a published example rather than a fix. --- scripts/snippet-manifest.psd1 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/snippet-manifest.psd1 b/scripts/snippet-manifest.psd1 index 5fa7d82..f29e182 100644 --- a/scripts/snippet-manifest.psd1 +++ b/scripts/snippet-manifest.psd1 @@ -1,4 +1,4 @@ -<# +<# Which example snippets the end-to-end runner executes, and how. scripts/run-examples-e2e.ps1 reads this file. Keeping the list here rather @@ -63,7 +63,7 @@ @{ Name = '01_load_session'; Path = 'snippets\python\getting-started\01_load_session.py'; Args = @('{fixtureKey}') } @{ Name = '02_read_parameters'; Path = 'snippets\python\getting-started\02_read_parameters.py'; Args = @('{fixtureKey}') } @{ Name = '03_query_sessions'; Path = 'snippets\python\getting-started\03_query_sessions.py' } - @{ Name = '04_extract_events'; Path = 'snippets\python\getting-started\04_extract_events.py'; Args = @('{fixtureKey}') } + @{ Name = '04_extract_events'; Path = 'snippets\python\getting-started\04_extract_events.py'; Args = @('{fixtureKey}'); Skip = 'event data does not survive a session reload, so a loaded fixture session never has any - the snippet runs but verifies nothing' } ) Matlab = @( From 6d7d95fd68560329487b93f3f7cc128bd0a1ffe2 Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Fri, 28 Aug 2026 09:36:26 +0000 Subject: [PATCH 11/15] Add the golden for event-definitions-and-data The snippet has never passed, so no golden was ever recorded and it would have reported NOGOLD - ran, but nothing verified - even after the fix. Generated with the harness against the last published release, 2.1.26212.6, and confirmed stable: two further runs compared clean. All four events appear in time order with their raw data, and the per-run session GUID normalises away. --- tests/golden/CSharp/event-definitions-and-data.txt | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 tests/golden/CSharp/event-definitions-and-data.txt diff --git a/tests/golden/CSharp/event-definitions-and-data.txt b/tests/golden/CSharp/event-definitions-and-data.txt new file mode 100644 index 0000000..992327b --- /dev/null +++ b/tests/golden/CSharp/event-definitions-and-data.txt @@ -0,0 +1,7 @@ +Session : event definitions and data + Timestamp (ns) DefKey Group Data +──────────────────────────────────────────────────────────────────────────────── + 36005000000000 1 Diagnostics [127.3, 120] + 36010000000000 2 Diagnostics [3, 1] + 36015000000000 3 Diagnostics [0, 1] + 36020000000000 1 Diagnostics [135.1, 120] From c3e318f083d9079ce23b24da98adb4b0871d4060 Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Fri, 28 Aug 2026 10:53:26 +0000 Subject: [PATCH 12/15] Stop multi_parameter_extract failing on mixed sample rates It assumed every parameter shared the first one's timestamps. The fixture is deliberately mixed-rate, so Speed:Shaft's 500 values went into a 1000-row index and pandas refused. Slicing to len(df) truncates but cannot extend. Assigning through pd.Series aligns on index and pads the short column with NaN. That alone would make the snippet quietly wrong rather than broken: columns are matched by position, so Speed:Shaft's 50 Hz samples sit against Temperature's 10 Hz timestamps and every one of its values lands on the wrong time. head(10) never reaches the padding, so nothing on screen hints at it. The snippet now says so in its output when the counts differ, and points at multi-rate-alignment.cs, which builds a common timestamp array properly. Golden generated against 2.1.26212.6 and confirmed stable over two further runs. --- .../data-access/multi_parameter_extract.py | 11 ++++++++++- tests/golden/Python/multi_parameter_extract.txt | 17 +++++++++++++++++ 2 files changed, 27 insertions(+), 1 deletion(-) create mode 100644 tests/golden/Python/multi_parameter_extract.txt diff --git a/snippets/python/data-access/multi_parameter_extract.py b/snippets/python/data-access/multi_parameter_extract.py index c1fc48c..3b55cb3 100644 --- a/snippets/python/data-access/multi_parameter_extract.py +++ b/snippets/python/data-access/multi_parameter_extract.py @@ -90,10 +90,19 @@ if all_data: first_key = identifiers[0] df = pd.DataFrame({"timestamp_ns": all_data[first_key]["timestamps"]}) + # Columns are matched by position, not by timestamp. Parameters at + # different rates have different sample counts, so a shorter one is + # padded with NaN rather than forced into the first parameter's row + # count - and its values do NOT line up with timestamp_ns. To compare + # across rates, sample onto a common timestamp array as + # snippets/csharp/data-access/multi-rate-alignment.cs does. for param_id in identifiers: - df[param_id] = all_data[param_id]["values"][:len(df)] + df[param_id] = pd.Series(all_data[param_id]["values"][:len(df)]) print(f"\nDataFrame: {df.shape[0]} rows x {df.shape[1]} columns") + counts = {p: len(all_data[p]["values"]) for p in identifiers} + if len(set(counts.values())) > 1: + print(f"Note: sample counts differ {counts} - columns are positional, not time-aligned.") print(df.head(10).to_string(index=False)) finally: client_session.Dispose() diff --git a/tests/golden/Python/multi_parameter_extract.txt b/tests/golden/Python/multi_parameter_extract.txt new file mode 100644 index 0000000..ac54046 --- /dev/null +++ b/tests/golden/Python/multi_parameter_extract.txt @@ -0,0 +1,17 @@ +Session has 3 parameter(s): ['Pressure:Inlet', 'Temperature:Sensors', 'Speed:Shaft'] + Pressure:Inlet: 1000 samples + Temperature:Sensors: 1000 samples + Speed:Shaft: 500 samples +DataFrame: 1000 rows x 4 columns +Note: sample counts differ {'Pressure:Inlet': 1000, 'Temperature:Sensors': 1000, 'Speed:Shaft': 500} - columns are positional, not time-aligned. + timestamp_ns Pressure:Inlet Temperature:Sensors Speed:Shaft +36000000000000 2.0 20.0 0.0 +36000010000000 2.0 20.1 0.2 +36000020000000 2.0 20.2 0.4 +36000030000000 2.0 20.3 0.6 +36000040000000 2.0 20.4 0.8 +36000050000000 2.0 20.5 1.0 +36000060000000 2.0 20.6 1.2 +36000070000000 2.0 20.7 1.4 +36000080000000 2.0 20.8 1.6 +36000090000000 2.0 20.9 1.8 From 50be279e135d3cc583f56bd198581dc994ff7b29 Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Thu, 3 Sep 2026 12:54:04 +0000 Subject: [PATCH 13/15] Make the query snippets actually match something Three vacuous goldens: query-with-composite-filter, 05-query-sessions-by-metadata and 03_query_sessions all recorded "no sessions found". They passed while proving nothing, which is worse than failing. Three independent causes, all now fixed. The fixture session was named "Verified Test Session" while every query snippet filters on Identifier containing "Example", so it could never match. Renamed to "Verified Example Session". Its recording date was fixed at 2025-06-15, outside the "last 30 days" window the composite filter asks for. Only the 10:00 time-of-day matters - that is what makes Session.StartTime equal StartTimeNs - so the date is now today and the window is satisfied. query-with-composite-filter also had two bugs of its own. It filtered on "TimeOfRecording", which is the property name on the result rather than a filter field; the field is SessionStartDateTime. An unrecognised name falls through to being treated as a session metadata item and quietly matches nothing. It also passed the date as ISO 8601 "O", while ScalarFilter parses with TryParseExact against ScalarFilter.DateFormat, so any other format silently fails. It now uses SessionFieldIdentifiers and the DateTime overload, which formats itself. Also widened the date normalisation in run-examples-e2e.ps1. pythonnet renders DateTime with the current culture, so 03_query_sessions prints "9/3/2026 10:00:00 AM" - single-digit month and day and an AM/PM suffix, none of which the old pattern matched. That golden would have passed today and failed tomorrow. Verified: 25 C# and 6 Python snippets pass against the regenerated goldens, and the four that differed were reviewed individually before regenerating rather than being rewritten wholesale. --- scripts/run-examples-e2e.ps1 | 5 ++++- .../query-with-composite-filter.cs | 15 ++++++++++++--- tests/golden/CSharp/01-load-session-from-file.txt | 2 +- .../CSharp/05-query-sessions-by-metadata.txt | 6 +++--- tests/golden/CSharp/introspect-session-config.txt | 2 +- tests/golden/CSharp/parameter-unit-resolution.txt | 2 +- .../golden/CSharp/query-with-composite-filter.txt | 3 ++- tests/golden/Python/01_load_session.txt | 2 +- tests/golden/Python/03_query_sessions.txt | 3 ++- tools/FixtureGenerator/VerifiedFixture.cs | 11 ++++++++--- 10 files changed, 35 insertions(+), 16 deletions(-) diff --git a/scripts/run-examples-e2e.ps1 b/scripts/run-examples-e2e.ps1 index bd815d5..6b3a762 100644 --- a/scripts/run-examples-e2e.ps1 +++ b/scripts/run-examples-e2e.ps1 @@ -86,7 +86,10 @@ try { # Volatile values, most specific pattern first. $normalised = $normalised -replace '[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}', '' $normalised = $normalised -replace '\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}(\.\d+)?', '' - $normalised = $normalised -replace '\d{2}/\d{2}/\d{4} \d{2}:\d{2}:\d{2}', '' + # Single-digit month and day, and an AM/PM suffix: pythonnet renders + # DateTime with the current culture, so 9/3/2026 10:00:00 AM has to + # normalise too or the golden only holds for the day it was written. + $normalised = $normalised -replace '\d{1,2}/\d{1,2}/\d{4} \d{1,2}:\d{2}:\d{2}( [AP]M)?', '' # Windows paths, including the per-user temp directory the fixture lives in. $normalised = $normalised -replace '[A-Za-z]:\\[^\s"'']*', '' # Elapsed times and sizes: "12.345 s", "1234 ms", "4.2 MB". diff --git a/snippets/csharp/session-management/query-with-composite-filter.cs b/snippets/csharp/session-management/query-with-composite-filter.cs index edcd9ae..c0dd078 100644 --- a/snippets/csharp/session-management/query-with-composite-filter.cs +++ b/snippets/csharp/session-management/query-with-composite-filter.cs @@ -13,6 +13,7 @@ using MESL.SqlRace.Domain; using MESL.SqlRace.Domain.Query; +using MESL.SqlRace.Domain.Sort; Core.LicenceProgramName = "SQLRace"; Core.Initialize(); @@ -21,10 +22,18 @@ var queryManager = QueryManager.CreateQueryManager(connectionString); -// --- Build a composite filter: session type contains "Example" AND recorded after a date --- +// --- Build a composite filter: identifier contains "Example" AND recorded after a date --- +// Field names come from SessionFieldIdentifiers, not from the property names on +// the result. "TimeOfRecording" is the property; the filter field is +// SessionStartDateTime. An unrecognised name is treated as a session metadata +// item and quietly matches nothing rather than raising. +// +// Pass the date as a DateTime: the string overload has to be in +// ScalarFilter.DateFormat exactly, and anything else - ISO 8601 "O" included - +// fails to parse and silently matches nothing. var composite = new CompositeFilter(CombineType.AND); -composite.Add(new ScalarFilter("Identifier", MatchingRule.Contains, "Example", ignoreCase: true)); -composite.Add(new ScalarFilter("TimeOfRecording", MatchingRule.GreaterThan, DateTime.UtcNow.AddDays(-30).ToString("O"), ignoreCase: true)); +composite.Add(new ScalarFilter(SessionFieldIdentifiers.SessionIdentifier, MatchingRule.Contains, "Example", ignoreCase: true)); +composite.Add(new ScalarFilter(SessionFieldIdentifiers.SessionStartDateTime, MatchingRule.GreaterThan, DateTime.UtcNow.AddDays(-30))); queryManager.Filter = composite; diff --git a/tests/golden/CSharp/01-load-session-from-file.txt b/tests/golden/CSharp/01-load-session-from-file.txt index df79f31..83189ee 100644 --- a/tests/golden/CSharp/01-load-session-from-file.txt +++ b/tests/golden/CSharp/01-load-session-from-file.txt @@ -1,7 +1,7 @@ File: Sessions found: 1 Loaded session: - Identifier: Verified Test Session + Identifier: Verified Example Session Start time: 36000000000000 ns End time: 36009990000000 ns Duration: diff --git a/tests/golden/CSharp/05-query-sessions-by-metadata.txt b/tests/golden/CSharp/05-query-sessions-by-metadata.txt index 8a46f1f..572191f 100644 --- a/tests/golden/CSharp/05-query-sessions-by-metadata.txt +++ b/tests/golden/CSharp/05-query-sessions-by-metadata.txt @@ -1,6 +1,6 @@ === All sessions === - Verified Test Session Z + Verified Example Session Z Total: 1 session(s) === Sessions matching 'Example' === - No sessions found matching 'Example'. - Tip: Run 04-create-session-write-data.cs to create some sessions. + Verified Example Session Z +Found: 1 session(s) diff --git a/tests/golden/CSharp/introspect-session-config.txt b/tests/golden/CSharp/introspect-session-config.txt index 23db3d8..4704804 100644 --- a/tests/golden/CSharp/introspect-session-config.txt +++ b/tests/golden/CSharp/introspect-session-config.txt @@ -1,4 +1,4 @@ -Session: Verified Test Session () +Session: Verified Example Session () === Parameters (3) === Identifier Conversion AppGroup ──────────────────────────────────────────────────────────────────────── diff --git a/tests/golden/CSharp/parameter-unit-resolution.txt b/tests/golden/CSharp/parameter-unit-resolution.txt index c5c5e12..ef9e64c 100644 --- a/tests/golden/CSharp/parameter-unit-resolution.txt +++ b/tests/golden/CSharp/parameter-unit-resolution.txt @@ -1,4 +1,4 @@ -Session: Verified Test Session (3 parameters) +Session: Verified Example Session (3 parameters) Identifier Conversion Unit Format Range ─────────────────────────────────────────────────────────────────────────────────────────────── Pressure:Inlet Pressure:Inlet_conversion bar %5.3f [0.0, 10.0] diff --git a/tests/golden/CSharp/query-with-composite-filter.txt b/tests/golden/CSharp/query-with-composite-filter.txt index b02a195..0c074a7 100644 --- a/tests/golden/CSharp/query-with-composite-filter.txt +++ b/tests/golden/CSharp/query-with-composite-filter.txt @@ -1,4 +1,5 @@ Filter: Identifier contains 'Example' AND recorded in last 30 days Session Key Identifier Recorded ────────────────────────────────────────────────────────────────────────────────────────────── -No sessions matched. Create some sessions first with 04-create-session-write-data.cs. + Verified Example Session Z +1 session(s) matched. diff --git a/tests/golden/Python/01_load_session.txt b/tests/golden/Python/01_load_session.txt index 7ca600b..4d75a51 100644 --- a/tests/golden/Python/01_load_session.txt +++ b/tests/golden/Python/01_load_session.txt @@ -1,6 +1,6 @@ SQL Race initialised Session loaded: - Name: Verified Test Session + Name: Verified Example Session Start time: 36000000000000 ns End time: 36009990000000 ns Duration: diff --git a/tests/golden/Python/03_query_sessions.txt b/tests/golden/Python/03_query_sessions.txt index ab98660..3be8ab3 100644 --- a/tests/golden/Python/03_query_sessions.txt +++ b/tests/golden/Python/03_query_sessions.txt @@ -1,4 +1,5 @@ Searching for sessions matching 'Example'... Session Key Identifier Recorded ────────────────────────────────────────────────────────────────────────────────────────────── -No sessions found. Run 04-create-session-write-data.cs to create some. + Verified Example Session +1 session(s) found. diff --git a/tools/FixtureGenerator/VerifiedFixture.cs b/tools/FixtureGenerator/VerifiedFixture.cs index c7d7dcc..edffe07 100644 --- a/tools/FixtureGenerator/VerifiedFixture.cs +++ b/tools/FixtureGenerator/VerifiedFixture.cs @@ -19,7 +19,9 @@ public static class VerifiedFixture // ── Session identity ─────────────────────────────────────── /// Fixed session key so examples that take a GUID can be pointed at the fixture. public const string SessionKeyString = "f1c0ffee-0000-4000-8000-000000000001"; - public const string SessionName = "Verified Test Session"; + // Contains "Example" so the query snippets, which filter on + // Identifier contains "Example", have something to match. + public const string SessionName = "Verified Example Session"; /// Session start in nanoseconds-of-day (10:00:00.000 → 36,000 s). public const long StartTimeNs = 36_000_000_000_000L; @@ -73,8 +75,11 @@ public static void Generate(string connectionString) var sessionManager = SessionManager.CreateSessionManager(); - // Date with a 10:00:00 time-of-day so Session.StartTime == StartTimeNs. - var sessionDate = new DateTime(2025, 6, 15, 10, 0, 0, DateTimeKind.Utc); + // Only the 10:00:00 time-of-day matters - it is what makes + // Session.StartTime equal StartTimeNs. The date is today so that + // TimeOfRecording falls inside the "last 30 days" window the query + // snippets filter on; a fixed date made them match nothing. + var sessionDate = DateTime.UtcNow.Date.AddHours(10); using var clientSession = sessionManager.CreateSession( connectionString, Key, SessionName, sessionDate, "Example"); From 562c7336eca4cb79edaa82115bc7c0ab737fc409 Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Thu, 3 Sep 2026 18:10:25 +0000 Subject: [PATCH 14/15] Put events in the fixture and test the events snippet properly I was wrong that event data does not survive a session reload, and the skip on 04_extract_events rested on that. It does survive: writing a definition and two events, then reloading, returns both with correct timestamps and raw data. My earlier test committed the parameters in one configuration set and the event definition in a second. Only the first configuration set committed to a session survives a reload - the note about that was already in this file, about parameters. Everything that must persist has to go in one commit. So BuildBatch now takes an optional hook to add conversions and event definitions to the same configuration, and the fixture raises three events at +2s, +5s and +8s carrying the temperature ramp's value at that moment against a fixed threshold - still derivable by hand, like every other value in here. 04_extract_events is unskipped and now exercises the whole snippet rather than an empty loop. Its output matches the closed form exactly: 40.0, 70.0 and 100.0 for Temperature(200), Temperature(500) and Temperature(800). Verified nothing else moved: 25 C# and 7 Python snippets pass, with no golden other than the new one needing to change. --- .../SqlRace.Examples.Core/ParameterBuilder.cs | 12 +++++- scripts/snippet-manifest.psd1 | 2 +- tests/golden/Python/04_extract_events.txt | 6 +++ tools/FixtureGenerator/VerifiedFixture.cs | 38 +++++++++++++++++++ 4 files changed, 56 insertions(+), 2 deletions(-) create mode 100644 tests/golden/Python/04_extract_events.txt diff --git a/projects/csharp/SqlRace.Examples.Core/ParameterBuilder.cs b/projects/csharp/SqlRace.Examples.Core/ParameterBuilder.cs index d6855b3..87b3723 100644 --- a/projects/csharp/SqlRace.Examples.Core/ParameterBuilder.cs +++ b/projects/csharp/SqlRace.Examples.Core/ParameterBuilder.cs @@ -194,7 +194,15 @@ public BuiltParameter Build() /// The session to add parameters to. /// The configured builders to build. /// A list of created parameters. - public static IReadOnlyList BuildBatch(Session session, IEnumerable builders) + /// + /// Optional hook to add anything else - conversions, event definitions - to the same + /// configuration before it is committed. Only the first configuration set committed to a + /// session survives a reload, so anything that must persist has to go in this one. + /// + public static IReadOnlyList BuildBatch( + Session session, + IEnumerable builders, + Action? configure = null) { var config = session.CreateConfiguration(); var parameters = new List(); @@ -264,6 +272,8 @@ public static IReadOnlyList BuildBatch(Session session, IEnumera parameters.Add(new BuiltParameter(parameter, (uint)channelId)); } + configure?.Invoke(config); + config.Commit(); return parameters; } diff --git a/scripts/snippet-manifest.psd1 b/scripts/snippet-manifest.psd1 index f29e182..e44f30e 100644 --- a/scripts/snippet-manifest.psd1 +++ b/scripts/snippet-manifest.psd1 @@ -63,7 +63,7 @@ @{ Name = '01_load_session'; Path = 'snippets\python\getting-started\01_load_session.py'; Args = @('{fixtureKey}') } @{ Name = '02_read_parameters'; Path = 'snippets\python\getting-started\02_read_parameters.py'; Args = @('{fixtureKey}') } @{ Name = '03_query_sessions'; Path = 'snippets\python\getting-started\03_query_sessions.py' } - @{ Name = '04_extract_events'; Path = 'snippets\python\getting-started\04_extract_events.py'; Args = @('{fixtureKey}'); Skip = 'event data does not survive a session reload, so a loaded fixture session never has any - the snippet runs but verifies nothing' } + @{ Name = '04_extract_events'; Path = 'snippets\python\getting-started\04_extract_events.py'; Args = @('{fixtureKey}') } ) Matlab = @( diff --git a/tests/golden/Python/04_extract_events.txt b/tests/golden/Python/04_extract_events.txt new file mode 100644 index 0000000..af72bb4 --- /dev/null +++ b/tests/golden/Python/04_extract_events.txt @@ -0,0 +1,6 @@ +Definition: LimitExceedance (priority: High) + Timestamp (ns) Key Group Data +────────────────────────────────────────────────────────────────────── + 36002000000000 1 Sensors [40.0, 50.0] + 36005000000000 1 Sensors [70.0, 50.0] + 36008000000000 1 Sensors [100.0, 50.0] diff --git a/tools/FixtureGenerator/VerifiedFixture.cs b/tools/FixtureGenerator/VerifiedFixture.cs index edffe07..72d89f3 100644 --- a/tools/FixtureGenerator/VerifiedFixture.cs +++ b/tools/FixtureGenerator/VerifiedFixture.cs @@ -50,6 +50,19 @@ public static class VerifiedFixture public const int ParameterCount = 3; + // ── Events ───────────────────────────────────────────────── + // One definition, three instances, each raising the temperature ramp's value + // at that moment against a fixed threshold - so the expected output is still + // derivable by hand. + public const int EventDefinitionId = 1; + public const string EventDescription = "LimitExceedance"; + public const string EventGroupName = "Sensors"; + public const double EventThreshold = 50.0; + + /// Seconds after the session start at which an event is raised. + public static readonly int[] EventOffsetsSeconds = { 2, 5, 8 }; + public const int EventCount = 3; + // NOTE: a freshly-created SQLite session always reloads with exactly one (default) lap. // Laps added via LapCollection.Add do not survive a reload in this API path, so the // verified fixture does not rely on laps. (Pre-existing .ssn2 files may contain laps @@ -117,12 +130,37 @@ public static void Generate(string connectionString) .WithDataType(DataType.Double64Bit) .WithConversion("rad/s", "%6.2f") .WithRange(min: 0, max: 200), + }, + config => + { + // An EventDefinition's conversion function names must already be registered + // or Commit() drops the definition, logging rather than throwing. + config.AddConversion(RationalConversion.CreateSimple1To1Conversion("Value", "degC", "%5.2f")); + config.AddConversion(RationalConversion.CreateSimple1To1Conversion("Threshold", "degC", "%5.2f")); + config.AddEventDefinition(new EventDefinition( + EventDefinitionId, + EventDescription, + EventPriorityType.High, + new List { "Value", "Threshold" }, + EventGroupName)); }); WriteRamp(session, built[0].ChannelId, TemperatureHz, TemperatureSampleCount, Temperature); WriteRamp(session, built[1].ChannelId, PressureHz, PressureSampleCount, _ => PressureConstant); WriteRamp(session, built[2].ChannelId, SpeedHz, SpeedSampleCount, Speed); + // Events need a session that already has channel data: they are stored in + // time segments, and a session with no time base discards them silently. + foreach (var offset in EventOffsetsSeconds) + { + var timestamp = StartTimeNs + (long)offset * 1_000_000_000L; + session.Events.AddEventData( + EventDefinitionId, + EventGroupName, + timestamp, + new List { Temperature(offset * TemperatureHz), EventThreshold }); + } + // Finalise the recording so all data and metadata are persisted to disk. session.Flush(); session.EndData(); From 4f5bf06d18235084e17cb6ed2fd3b8e255bbf777 Mon Sep 17 00:00:00 2001 From: Federica Donald Date: Fri, 4 Sep 2026 14:01:29 +0000 Subject: [PATCH 15/15] Drop the MATLAB event_extraction golden It recorded "Total events: 0" - a pass over nothing. None of the eight scenario files under notebooks/scenarios/data/ contain a single event definition, so the snippet is reporting empty data honestly and the golden asserted nothing. Without it the snippet reports NOGOLD, which is the accurate state: it runs, and nothing is verified. Fixing it properly means adding event data to motorsport-lap-analysis.ssn2. Retrofitting works - tested on a copy, two definitions and three events survive a reload - but that is a change to committed binary data with no generator script behind it, which belongs in its own change rather than this one. Correcting something I got wrong earlier: the other MATLAB goldens are fine. The scenario .ssn2 files are committed and tracked, so "Turbine Inspection Week 1" and the rest are reproducible repo data, not sessions left on a dev machine. --- tests/README.md | 5 +++++ tests/golden/Matlab/event_extraction.txt | 14 -------------- 2 files changed, 5 insertions(+), 14 deletions(-) delete mode 100644 tests/golden/Matlab/event_extraction.txt diff --git a/tests/README.md b/tests/README.md index 78e4291..e0af748 100644 --- a/tests/README.md +++ b/tests/README.md @@ -73,6 +73,11 @@ golden means nothing was verified. Skipped snippets are listed in the summary fo reason — a snippet silently missing from the run would otherwise be indistinguishable from one that passed. +`Matlab/event_extraction` has no golden on purpose. None of the scenario files under +`../notebooks/scenarios/data/` contain event definitions, so it reports zero events and a +golden would record that as a pass over nothing. It needs event data adding to +`motorsport-lap-analysis.ssn2` before it is worth capturing. + ## The verified fixture `VerifiedFixture.Generate(connectionString)` writes one session (`Verified Test Session`, diff --git a/tests/golden/Matlab/event_extraction.txt b/tests/golden/Matlab/event_extraction.txt deleted file mode 100644 index 8ac80c1..0000000 --- a/tests/golden/Matlab/event_extraction.txt +++ /dev/null @@ -1,14 +0,0 @@ -Session: Track Test Session 1 -Event definitions: 0 -Reading events across 5 segment(s): -Segment Events --------------------------------- -Lap 1 0 -Lap 2 0 -Lap 3 0 -Lap 4 0 -Lap 5 0 -Total events: 0 -(This session contains no event instances — events are written by - the recording/import pipeline, e.g. ATLAS or a ServerListener recorder.) -Done.