Skip to content

feat(improve): lousho improve --watch with budgets, rate limits, lease and kill switch - #997

Merged
LinuxDevil merged 2 commits into
mainfrom
s24-watch
Nov 8, 2026
Merged

LinuxDevil merged 2 commits into
mainfrom
s24-watch

Conversation

@LinuxDevil

Copy link
Copy Markdown
Owner

Why

S24 of the M4 plan (m3-notes/m4/parity.md): E3, E4 and E5. Self-healing means the cycle runs without anyone starting it, so it needs hard limits: a kill switch, one watcher per directory, daily budgets, a cooldown after a rollback, dedupe, rate limits, and a clean stop.

Changes

  • src/improve/watch.ts:
    • watchTick(options, context), in order:
      1. Kill switch: .lousho/improve.stop, LOUSHO_IMPROVE=off or improve.json enabled: false (re-read every tick) stops the tick, audited once as pause (remembered in state.json); turning it off audits resume.
      2. Lease: acquireLease('improve-<sha of dir>', '<pid>:watch:<uuid>', 2 × interval) on a file store in .lousho/watch (a store without leases is refused); a second watcher gets leased. The lease is renewed every third of its TTL while the tick runs; losing it aborts the tick at the next step. healOnce's cycle.lock still blocks concurrent cycles.
      3. checkApplied (S10); then the harvest below.
      4. The corpus guard every guard.everyHours (rollback: true, each run of one revision gets its own firstRepeat range from 200).
      5. A cycle, unless the monitor's cooldownUntil is in the future, and unless the failure-group signatures, the revision, the failed-run count and the newest failed trace equal those of the last cycle that did work (dedupe; an outage, busy, budget, cooldown, paused or interrupted cycle doesn't set the key, so the next tick retries). healOnce gets the day's budget left after the watcher's own guard and corpus runs as maxTokens; healOnce enforces cycles/day, applies/day, tokens/day and the audited-rollback cooldown under its lock.
      6. Three outage cycles in a row write the stop file and audit pause.
    • The kill switch and stop() are re-checked between every step (monitor, harvest, guard, cycle); settings are re-read every tick (reloadSettings; the CLI keeps its --approval).
    • watchAgent(options) ticks every intervalMs (default 15 min); a tick that throws is reported via onError and the watcher keeps going. stop() aborts between steps (an apply and its guard always finish), waits for the tick, and releases the lease.
    • improveSchedule(options): a defineSchedule() schedule whose run is one tick.
    • The editor provider is wrapped with withRateLimit at rateLimit.rpm (new setting, default 20).
  • src/improve/watchSteps.ts: watcher state, the harvest (once per apply: when the monitor says kept, voteOnOutcome('kept') and admit the training cases the edit fixed to the corpus as origin: 'fixed' after 6/6 fresh runs; on rolled-back, voteOnOutcome('rolled-back')), and the scheduled guard. Their probes are metered against the day's budget left and their spend is recorded even when they stop early; usageToday now includes it, so healOnce (and a one-shot lousho improve) see it too. The harvest claim is a check-and-set under the state lock.
  • src/improve/state.ts: state.json read/update under .lousho/state.lock with an atomic write, now shared by monitor.ts and the watcher.
  • src/improve/pause.ts + lousho improve pause <dir> [--reason] [--by] / resume <dir> [--by]: write / remove the stop file, audited.
  • CLI: lousho improve <dir> --check <m> --watch [--interval 15m] [--agent X]; Ctrl-C/SIGTERM stops after the step. --watch refuses a dry run, --max-tokens and --out; --interval needs --watch.
  • Telegram scaffold (src/cli/init/telegramTemplate.ts) wrote .lousho/state.json without the lock and would clobber the loop's state; it now keeps its own .lousho/telegram.json (atomic rename) and still reads the owner from an old state.json. Bridges generated before this change still write state.json without the lock; the S27 release note should tell Telegram users to regenerate bridge.ts.
  • R10 writers added: state.ts (.lousho/state.json), pause.ts (.lousho/improve.stop), the watcher's lease files under .lousho/watch/ (via fileStore).

Review (independent): one blocker (the dedupe key was saved after an outage or a blocked cycle, so the next tick never retried) and should-fix items (lease renewal, per-dir lease id and refusing lease-less stores, metering the watcher's own spend, checking stop/pause between steps, re-reading settings, Telegram {} fallback, missing budget tests) are fixed in this PR. Not done: fake timers in tests (the tests drive watchTick directly and watchAgent with real short intervals); a paused watcher also skips the monitor's auto-rollback, which S27 should document.

Testing

  • src/improve/watch.test.ts (mock editor model, temp agent directory, real observe db): the stop file prevents any cycle and is audited once (and resume is audited); LOUSHO_IMPROVE=off / enabled: false; the second watcher on the same dir does nothing; maxAppliesPerDay 1 blocks a second apply; the monitor's cooldown blocks cycles; the token budget passed to healOnce shrinks as spent; same signatures twice skip the second cycle; three outages in a row (no new failures between ticks) pause with one audit entry; maxCyclesPerDay, the daily token cap and an audited rollback stop the cycle; the watcher's own guard tokens count toward the day; the lease is renewed during a long tick so a second watcher stays out; the scheduled guard runs once everyHours passed; a kept edit's fixed cases join the corpus once; improveSchedule; watchAgent.stop() during a cycle exits after it with nothing applied. Cycles/day and tokens/day caps are covered by heal.test.ts.
  • src/cli/improve.test.ts: --watch runs until stopped, refuses --dry-run, --interval parsing; pause/resume audited and the paused CLI cycle writes nothing.
  • typecheck (src, tests, examples), lint, build, api:update + api:check, docs:llms + docs:llms:check, docs:verify-snippets -- --skip-build, docs:ratchet (1724), vitest src/improve src/cli src/observe, npx fallow audit --base origin/main after scoped coverage.

🤖 Generated with Claude Code

LinuxDevil and others added 2 commits November 9, 2026 01:54
…e and kill switch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Retry after an outage or a blocked cycle, renew the lease during a tick, a
per-directory lease id, meter the watcher's own spend into the day's usage,
check stop and pause between steps, re-read settings each tick, and keep the
Telegram owner when telegram.json is empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@LinuxDevil
LinuxDevil merged commit 6f4d4e1 into main Nov 8, 2026
6 checks passed
@LinuxDevil
LinuxDevil deleted the s24-watch branch November 8, 2026 23:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant