Skip to content

Fix live reading examples returning data minutes behind live (ADO 91065) - #66

Merged
federicadonald merged 6 commits into
masterfrom
fix-live-reading-server-listener
Oct 20, 2026
Merged

federicadonald merged 6 commits into
masterfrom
fix-live-reading-server-listener

Conversation

@federicadonald

Copy link
Copy Markdown
Contributor

Fixes ADO 91065: snippets/csharp/live-data/poll-live-samples.cs returns data about 5 minutes behind the session end time. This PR also folds in #63, keeping its two commits by @salman-mclarenapplied, and fixes the issues found while testing it.

Root cause

All the live-reading examples load a session with SessionSummary.GetConnectionString(). That returns the Server Listener connection string (SqlRaceServer:...) only when the recorder registered a listener for the session. For a LiveNotInServer session it silently falls back to the database, and two things go wrong:

  • The database copy trails the leading edge by the recorder's flush delay. Slots are flushed once MaximumTimeInMemoryBeforeFlushingInSeconds (default 300 s) has passed since their last write, and that is checked every 60 s. The session end time, by contrast, is written every 0.5 s.
  • EndTime is read once at load and never updates.

The SessionState.LiveNotInServer docs already say such a session "should not be loaded until its state has changed to historic".

Changes

  • poll-live-samples.cs: require State == Live and a SqlRaceServer connection string, otherwise exit with a pointer to docs/server-listener.md. Drop the extra database load that only validated the parameter. Print how far the latest sample is behind the session end.
  • docs/server-listener.md: the troubleshooting row told users to reload the session every poll and cited this snippet. It now names the real cause and fix.
  • LiveDashboard (from Fix live reading scenarios as they are all wrong #63): load through the Server Listener and stop reloading every cycle. On top of Fix live reading scenarios as they are all wrong #63: refuse a configured LiveNotInServer session and skip it when scanning, because otherwise the dashboard freezes with no explanation.
  • Notebooks (from Fix live reading scenarios as they are all wrong #63, plus fixes):
    • load_session used SessionSummary.key, but pythonnet is case-sensitive, so this raises AttributeError. It now uses .Key, raises when the session isn't found, and warns on LiveNotInServer.
    • The polling loop now reuses the PDAs Fix live reading scenarios as they are all wrong #63 creates; previously it still made a new one per parameter per poll.
    • samples.Data(i) → samples.Data[i]. .NET arrays aren't callable from pythonnet. This bug was already on master.
    • Reverted the local .venv (3.13.9) kernel metadata.

Testing

Tested against a local SQL Server, with a recorder writing 100 Hz data stamped with wall-clock time. One recording had the Server Listener on (Live), one had it off (LiveNotInServer). The client side used SQL Race built from current source.

Reader Live (listener) LiveNotInServer
Original snippet (on master), first poll 6 min into session – 341 s behind EndTime, then EndTime frozen
Reload from database each poll (old LiveDashboard pattern) 309–311 s behind wall clock –
Fixed snippet 0 ms behind session end (14 polls) Exits with message
LiveDashboard (this PR), configured and scan 0.0–0.1 s behind wall clock Refused / skipped with message
load_session (Python) .key → AttributeError confirmed; Live path not run end-to-end from Python* Warning printed

* Loading a source build of SQL Race into pythonnet without a deps.json hit native SQL client loading problems, so the Python Live path through the listener wasn't exercised end-to-end. The ATLAS-installed SQL Race couldn't talk to the source-built recorder (session summary wire-format mismatch).

dotnet test passes (65/65) against MESL.SQLRace.API 2.1.26272.6; the solution builds with warnings as errors.

Supersedes #63.

salman-mclarenapplied and others added 5 commits October 16, 2026 14:45
The snippet loaded with SessionSummary.GetConnectionString(), which only
returns the Server Listener connection string when the recorder registered
a listener for the session. For a LiveNotInServer session it silently falls
back to the database, whose copy trails the leading edge by the recorder's
flush delay (MaximumTimeInMemoryBeforeFlushingInSeconds, 300 s by default),
and whose EndTime is fixed at load. Reproduced locally: the first poll
returned data 341 s behind EndTime, then EndTime never moved again.

Require State == Live and a SqlRaceServer connection string, and fail with
a pointer to docs/server-listener.md otherwise; this matches the API's own
guidance that LiveNotInServer sessions should not be loaded. Drop the extra
database load used only to validate the parameter, and print how far the
latest sample is behind the session end.

Correct the server-listener.md troubleshooting row, which told users to
reload the session every poll and cited this snippet for it.
- load_session used SessionSummary.key; pythonnet is case-sensitive, so it
  raised AttributeError. Use .Key, raise when the session is not found,
  warn on LiveNotInServer, and stop shadowing the session_summary() helper.
- The polling loop created its own PDA per parameter per poll, leaving the
  new pdas list unused. Reuse the pre-created PDAs.
- samples.Data(i) is not callable from pythonnet; index the array instead.
- Revert the kernelspec/language_info metadata to the repo's defaults.
GetConnectionString() falls back to the database for a session whose
recorder has no Server Listener, where EndTime is fixed at load and data
trails the leading edge by the flush delay, so the dashboard froze with no
explanation. Refuse a configured LiveNotInServer session and skip it when
scanning, pointing at docs/server-listener.md.
Comment thread notebooks/06-live-data-monitoring.ipynb Outdated
CreateParameterDataAccess can return None (composite sessions), so skip
those parameters and drop them from monitor_params, keeping the plotted
history columns aligned.
@federicadonald
federicadonald merged commit 339c6e4 into master Oct 20, 2026
2 checks passed
@federicadonald
federicadonald deleted the fix-live-reading-server-listener branch October 20, 2026 10:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants