Skip to content

feat(improve): start a cycle on the observe daemon's score regression - #1000

Merged
LinuxDevil merged 3 commits into
mainfrom
s26-score-trigger
Nov 8, 2026
Merged

LinuxDevil merged 3 commits into
mainfrom
s26-score-trigger

Conversation

@LinuxDevil

Copy link
Copy Markdown
Owner

Why

S26 of the M4 plan (m3-notes/m4/parity.md). Paper B detects within the session, while a 15-minute timer waiting for minNewFailures reacts slowly. When the observe daemon's sweep raises agent.score_regression (S9), the watcher should start a cycle on its next tick with trigger: 'score'. The daemon stays read-only towards agent directories and spawns nothing.

Changes

  • src/observe/schema.ts: a forward-only migration adding a small score_regressions table (agent_id, revision, at, details). A table rather than rows in scores, so markers never show up as scores or low-score failures.
  • src/observe/sweep.ts:
    • every agent.score_regression the sweep raises is recorded there, once per agent and revision, also across daemon restarts (the in-memory edge memory used to reset);
    • startStatusSweep keeps the score pass on even without an alert sink, so markers are written whether or not webhooks are configured;
    • a failing insert is logged and never costs the pass its other alerts;
    • retention (pruneBefore) prunes old markers;
    • scoreRegressionsSince(db, agent, since) matches the agent id, else its registered name.
  • src/improve/watch.ts: each tick polls the markers newer than the last one it acted on, for the current revision (or unstamped runs). One starts a cycle with trigger: 'score', skipping the dedupe. healOnce still enforces the lock, the kill switch, the cooldown and every daily cap, and the marker is consumed only by a cycle that did work, so a blocked or paused tick retries it.
  • src/improve/watchers.ts: the watcher registers { agent, dir, owner, pid } in watchers.json beside the observe db while it runs (the CLI passes it). There is one entry per agent directory, and entries of dead processes are dropped. The file is informational for now.
  • Docs: docs/observability.md.

Testing

  • src/improve/watch.test.ts: a regression marker triggers a cycle with trigger score (where failures would be idle), and the next tick is back to failures; a marker for another revision is ignored; a score cycle still hits the automatic-apply cap and the cooldown after an audited rollback; a paused tick keeps the marker for the next tick; the registry holds one entry per directory and drops a dead watcher; watchAgent registers and unregisters.
  • src/observe/scoreRegressionAlert.test.ts: the daemon records a marker without an alert sink and writes nothing outside the db (the agent directory and its parent are unchanged); lookup by id and by name; a restarted sweep doesn't record the same revision again; a steady series records nothing.
  • Independent review: no blockers; the should-fix items (insert outside the sweep's error handling, re-firing after a daemon restart, an unused registry with stale entries, docs on what a score cycle learns from) and the nits (pruning, id-before-name, a stale JSDoc, missing tests) are fixed in the second commit. A pure score regression with no failed or low-scored runs still ends idle, as documented.
  • typecheck (src, tests, examples), lint, build, api:update + api:check, docs:llms + docs:llms:check, docs:verify-snippets -- --skip-build, docs:ratchet (1724), vitest src/improve src/cli src/observe, npx fallow audit --base origin/main after scoped coverage.

🤖 Generated with Claude Code

LinuxDevil and others added 3 commits November 9, 2026 02:34
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record a regression once per agent and revision across daemon restarts, keep
a failed marker insert from costing the sweep its alerts, prune old markers,
match an agent id before a name, and keep one live watcher entry per agent
directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@LinuxDevil
LinuxDevil merged commit 5dca091 into main Nov 8, 2026
6 checks passed
@LinuxDevil
LinuxDevil deleted the s26-score-trigger branch November 8, 2026 23:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant