Skip to content

feat(improve): regression corpus store and growth - #987

Merged
LinuxDevil merged 1 commit into
mainfrom
s13-corpus
Nov 8, 2026
Merged

LinuxDevil merged 1 commit into
mainfrom
s13-corpus

Conversation

@LinuxDevil

Copy link
Copy Markdown
Owner

Why

M4 S13 (part 1 of parity B9). The corpus guard (S14) needs a corpus: protected passes, plus each training case a kept edit fixed, so a later edit can't silently break it again.

Changes

  • New src/improve/corpus.ts, exported from lousho/improve:
    • CorpusCase { id, traceId, input, addedAt, revision, origin: 'protected' | 'fixed', baseline }.
    • readCorpus(dir) reads .lousho/corpus.json. It returns an empty list when the file is missing. It throws LOUSHO_CONFIG_INVALID when the file is broken, a symlink or not a regular file, or holds an id or trace twice. A broken file is never treated as empty, so it is never overwritten.
    • addToCorpus(dir, cases, maxCases) holds a file lock and does the following:
      • dedupes by traceId;
      • refuses an id that another trace already uses, since the guard keys by id;
      • drops the oldest fixed cases over maxCases, ordered by actual time;
      • never evicts protected cases;
      • writes a temp file with mode 0600, fsyncs and renames it, and removes the temp file if that fails;
      • refuses a symlinked .lousho.
    • It is async, unlike the ticket's sketch, because it takes a lock. That matches S4/S5.
    • Cases are validated: non-empty id, trace and revision; an ISO addedAt; and a baseline with 0 <= passed <= scored and scored >= 1.
  • Inputs are stored as captured, as our own local traces (R8). Nothing in the editor path reads the corpus (evidence.ts, run.ts and improve.ts don't reference it). S14 replays it only through the probe.
  • The admission rule is left to S23, the caller: a fixed case joins only after the monitor (S10) returns kept and a fresh 6/6 under the new revision through admitProtectedCases.

Testing

  • src/improve/corpus.test.ts (13 tests) covers:
    • adding and deduping by traceId, across calls and within one batch;
    • evicting the oldest fixed case first, ordered by time across timezone offsets;
    • never evicting protected cases;
    • the round trip through disk;
    • missing and broken files;
    • malformed cases and a bad maxCases;
    • concurrent adds;
    • a symlinked file or .lousho, which is refused and leaves the target untouched;
    • id collisions;
    • impossible baselines and timestamps;
    • a broken file left untouched, with no temp file left behind.
  • An independent review found 3 high items (a symlinked or non-regular corpus file wiping the corpus, a symlinked .lousho, and id collisions) and 3 medium items (addedAt ordering and validation, baseline bounds, empty fields). All are fixed and tested.
  • Standard checks: typecheck, lint, build, api:update/api:check, docs:llms:check, docs:verify-snippets -- --skip-build, docs:ratchet (1724) and fallow audit --base origin/main.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@LinuxDevil
LinuxDevil merged commit 49d245f into main Nov 8, 2026
6 checks passed
@LinuxDevil
LinuxDevil deleted the s13-corpus branch November 8, 2026 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant