Repository navigation
Description
What happened
On the person's fourth message in a conversation, the scripted test model (test/remote/stub) answers codeaf's own helpers note instead of the person. A scripted PROBE-PWD on that turn never calls bash. The same thing happens on the first message after a long chat is opened in a new process, because a new process counts the person's earlier messages and lands the note on that turn.
Seen on app/v3 a687c92 and on the idle hand-off branch (#2287), on Linux and macOS, with codeaf chat --no-host --model stub/scripted --one-model:
› PROBE-ECHO d
STUB-REPLY: Instructions from codeaf, not from the person: when to suggest helpers. If …
› PROBE-PWD
<the working folder, from a real bash call>
The note lands once per process, at the step boundary, as a message after the person's latest words (landHelpersNoteLocked, at helpersNoteAfter = 4 person messages). The stub's script takes the last user-role message as the person's ask, so on that one turn it reads the note.
Replication
Deterministic (no model).
make build, thengo build -o modelstub ./test/remote/stub, then start it:./modelstub -addr <stub address>.- Use a throwaway home: set
HOME,CODEAF_HOMEandCODEAF_PROFILE_DIRto an empty folder, plusCODEAF_SYNC_URL=off,CODEAF_TELEMETRY=off,CODEAF_BASE_URL=<stub address>/api/v1andOPENROUTER_API_KEY=stub. Put{"model.talk":"stub/scripted","tools.approvalMode":"allow"}in$CODEAF_HOME/config.json. - Run
bin/codeaf chat --no-host --model stub/scripted --one-model, and press esc at setup. - Send
PROBE-ECHO a,PROBE-ECHO b,PROBE-ECHO c, thenPROBE-PWD.- Today: the fourth reply is
STUB-REPLY: Instructions from codeaf, not from the person: when to suggest helpers. …and nobashcall is made. - Expected: a
bashcall withpwd, quoted back.
- Today: the fourth reply is
- A fifth
PROBE-PWDworks. To see the new-process variant,/quit, reopen with--session <that chat's .cell/transcript.jsonl>, and sendPROBE-PWDfirst.
Field (real models). Not shown to misbehave. A real model reads the note as codeaf's, because of its opening words. This defect is about scripted runs, where it makes any e2e or hand test that scripts a tool call on the fourth message (or right after a reopen) flaky.
Where
scriptintest/remote/stub/main.go(searchSCANNING BACKWARDS FOR THE LAST USER MESSAGE).landHelpersNoteLockedandhelpersNoteOpeningininternal/session/helpersnote.go.- The engine's own list of note openings, in
internal/session/agent.go(searchstrings.HasPrefix(text, helpersNoteOpening)).
The fix
The stub looks for the person's ask the way the engine tells a note from a person: it skips user-role messages that open with Instructions from codeaf, not from the person. Keep the product's placement of the note, because tail notes are where volatile instructions land without disturbing the cached prefix. The opening string should come from one shared place, so the stub cannot drift from the engine.
Acceptance
- e2e: through the real door, with
bin/codeaf chatdriven in tmux against the stub (keyless). Four messages, the fourthPROBE-PWD. Assert that the fourth reply contains the working folder from abashcall and does not containInstructions from codeaf. - e2e: the control. The same run, reopened with
--sessionandPROBE-PWDsent first, gets abashcall. - Unit:
scriptreturns the person's last message when a later user-role message opens withInstructions from codeaf, not from the person.
Activity