Skip to content

feat(cli): hgs watch --json event stream (event data plane M3) - #368

Merged
ufna merged 3 commits into
mainfrom
work-ba0ed9fd-watch
Oct 27, 2026
Merged

ufna merged 3 commits into
mainfrom
work-ba0ed9fd-watch

Conversation

@ufna

@ufna ufna commented Oct 27, 2026

Copy link
Copy Markdown
Owner

Event data plane M3 (epic zerus-604d, task zerus-604d.3). Stacked on #367 (M1). Until that merges, this PR also shows its two commits; the M3 change is the last commit.

What changes

hgs [@host] watch --json: a long-lived NDJSON stream of one computer's live sessions, plus the Activity of the sessions a client subscribes to. It is additive; no existing command output changes.

  • Frames: hello, snapshot (pages ≤ 512 KiB), delta (whole-row upserts, removals with ended|archived|forgotten|replaced), events (cursor = the events.sqlite3 seq that inspect --after takes), state (inspect summary without events/processes), heartbeat (10 s), gap, error, pong. Frames are at most 1 MiB; a 4 MiB writer queue coalesces per key and falls back to gap + snapshot on overflow. EOF on stdin ends the stream.
  • Compatibility: hgs -h prints Event stream ABI: watch-v1. @host runs the stream over one SSH connection with ServerAliveInterval=10/ServerAliveCountMax=3, guarded by that marker, so an older peer answers with an error and never launches an agent.
  • Change detection, no new dependency: a 250 ms pulse stats the listing's inputs and polls PRAGMA data_version on a held read-only journal connection. A 5 s safety listing catches changes no file shows. goal_observed_at and process_work.sampled_at are masked, so deltas carry real changes only.
  • crates/watch-protocol: serde frame types without Loro or Qt, for the desktop (M4), the Tauri client and later the connector. It is wired into checks.yml, dependabot and mac-install.
  • The inspection's journal query is extracted (timeline_rows/timeline_event), not forked. inspect output is unchanged.
  • Documented in docs/reference.md → JSON and external clients → Event stream.

Verification (arch)

  • cargo +1.99.1 test --locked: 1243 passed, 3 ignored. Watch subset rerun by the reviewer: 14 passed. crates/watch-protocol: 7 passed; fmt and clippy -D warnings clean.
  • PYTHONPATH=tests python3 -m unittest test_watch: 9/9 (reviewer rerun). The subagent also ran 178 focused integration tests (1 skipped), the Windows target check and check-source (0 privacy findings).
  • Real arch state, catalog only, 60 s: 12.8 % of a core, 36 frames, 631 KB (snapshot 297 KB, then 28 real deltas).

Known limits, tracked for M4/M5

  • The watcher re-runs the full listing (about 0.26 s CPU) on each change, which is the CPU above; an incremental listing is follow-up work.
  • event (open-event replace) frames are defined but not sent yet. Text between tools arrives through 2 s state re-reads while an agent is busy.
  • hgs.exe @host watch from Windows over piped ssh is untested.

🤖 Generated with Claude Code

ufna and others added 3 commits October 27, 2026 16:41
Owner direction 2026-10-27: data parsing and similar processing run outside
the GUI thread. First milestone of the event data plane (zerus-604d.1).

- ProcessRunner passes every result through one serial preparation thread,
  hgs-prepare, keeping completion order. run(request, owner, prepare, apply)
  and post(owner, prepare, apply) shape values there; the owner's thread only
  applies them.
- Every HgsClient reply, the connector panels, the speech client, the update
  feed and startup fleet discovery are parsed in prepare steps. The task
  parse pool and the 8 MiB worktree snapshot re-parse per lookup are gone.
- JsonParse is the tray's only JSON parser and reports a parse on the GUI
  thread; every Qt suite runs with ZERUS_GUI_THREAD_CHECKS=fatal.
  check-source.py rejects QJsonDocument::fromJson elsewhere and limits
  JsonParse::local() to a listed, shrink-only set of small local values
  (owner: "согласен, оставляем исключения до M2").
- SettingsJson parses QSettings JSON values once per change, warms the large
  keys at startup and records what writers stored, so per-poll read-mark
  reads no longer re-parse settings.
- IdentityHash remembers per-row SHA-256 identities, 3.9 % of the Mac GUI
  thread in a 2026-10-27 sample.

Design: docs/superpowers/specs/2026-10-27-event-data-plane-design.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Event data plane M3 (zerus-604d.3). `hgs [@host] watch --json` is a
long-lived NDJSON stream: hello, a bounded catalog snapshot of the live
listing (rows exactly as ls --json --local --no-archives, keyed run:<run_id>),
deltas with removal reasons, catalog.meta, and Activity subscriptions over
stdin whose cursor is the events.sqlite3 seq that inspect --after takes.
Heartbeats, gap/snapshot resync, a 4 MiB coalescing writer queue and 1 MiB
frames bound it; EOF on stdin ends it. `hgs -h` prints the marker
`Event stream ABI: watch-v1`; @host runs it over one SSH connection with
keepalives, guarded so an older peer never reads it as a launch.

Change detection uses no new dependency: a 250 ms pulse stats the listing's
inputs and polls PRAGMA data_version on a held journal connection, with a 5 s
safety listing for changes no file shows. Volatile observation fields are
masked, so deltas carry real changes only. The serde frame types live in
crates/watch-protocol (no Loro or Qt) for the desktop, the Tauri client and
later the connector. The inspection's journal query is shared, not forked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ufna

ufna commented Oct 27, 2026

Copy link
Copy Markdown
Owner Author

Superseded by #369, which carries this commit (34f9585) plus a macOS test-timing fix; this PR closes as merged when #369 lands.

@ufna
ufna merged commit 34f9585 into main Oct 27, 2026
25 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant