auto-distill `strategy` excerpts mix success-summary prose with fix narration and truncate mid-fix

[stwhwing]

Summary

In controlled closed-loop runs, auto-distilled gene strategy text is excerpted from the session's own turns. Two quality problems observed on 2.0.30:

1. Fix narration and success-summary prose are mixed in one excerpt. Current on-disk example (single strategy step, as stored):

   "The file is GBK-encoded (contains Chinese characters),…"
   

   …and the neighboring step continues: "Let me fix the script: All correct. analyze.py outputs: - number of events: 8 …" — i.e. the fix turn and the closing success summary are stitched together.

2. Truncation cuts at arbitrary points, sometimes right at an actionable keyword. Stored steps end with a literal ellipsis (…), e.g. "The file is GBK-encoded (contains Chinese characters),…" — the actual fix (use encoding="gbk") lands after the cut.

Impact

Excerpt quality varies run-to-run with R1 error density (observed 0/3 vs 5/5 runs containing the fix across two experiment batches). A noise strategy, once injected, is worse than no injection (measured: R2 token +36pp vs baseline), so this variance directly breaks inheritance effectiveness.

Suggestion

1. Prefer turns matching error/repair keywords when excerpting strategy (or at least exclude the closing success-summary turn);
2. Keyword-aware truncation (never cut before the first actionable keyword), or a higher floor length;
3. Consider an LLM-refine fallback that also triggers when a strong signal WAS found but the excerpted strategy looks like success prose — the current autoDistillTranscript gate only fires when no strong signal exists ("do not double-spend an LLM"), which misses exactly this failure mode. (Notably the extractor already classifies SUCCESS_PROSE signals — that classification could gate the fallback.)

Full report: https://github.com/stwhwing/pi-evox-lab/blob/main/docs/experiment-report.md (§20.3)