Repository navigation
appendOnly: every request repeats what was sent and only appends — keeps Claude thinking signatures valid (core 4.3) - #57
Open
MyPrototypeWhat wants to merge 145 commits into
Conversation
The E10 cooldown skipped the whole budget check on the compile right after a compression. A host that passes its full history to setHistory every turn (the documented in-flight mode, where the summary is recomputed per call) therefore sent the uncompressed history on every other turn, and in a long conversation that payload grew until the provider rejected it. The compile after a compression now drops only a reported usage, which may describe the payload from before the compression, and still checks the window by its own count. A host that kept the compressed history stays under budget; one that passes its full history again gets it compressed again. JanitorSnapshot.suppressNextCompression keeps its name so stored snapshots restore. Tests that pinned the alternating behavior now assert the new one.
…odels Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5 sign every thinking block over everything before it; on accounts created on or after 2026-08-31 the API rejects a request whose earlier turns differ from what the model saw. - fromAnthropic records each parsed message's origin (Message._anthropic_origin) and the Anthropic target replays the original message while nothing has modified it: every thinking block, separate text blocks, parallel tool results with the user text after them, unmodeled blocks and compaction signatures. Modified messages are rebuilt from IR fields as before. The heuristic token estimate skips the stored original. - A history that opens with an on-demand compaction block keeps the block first on Anthropic: the target drops ensureValidHistory's opening placeholder there (isOpeningPlaceholder). Other targets are unchanged. - withGuardrails takes prefillMode: 'native' | 'instruction'. Sonnet 4.6 / 5 / 5.5 and Haiku 5.5 reject a prefilled last assistant turn; 'instruction' sends the prefill as an instruction wherever placement puts it. - announce() docs note two documented, not yet verified behaviors of the newest models. Docs, READMEs and TODO follow-ups updated.
…bility Compares two consecutive payloads (any target shape, including OpenAI Responses) and reports the first place the later one stops repeating the earlier one, ignoring key order and cache_control. The test suite uses it to record today's behavior: no injections and a static-system guardrail keep the prefix; tail injections (dynamic state, announcements on both channels, guardrail placement 'last_user') break it. Phase P0 of the append-only spec.
New guide page (en/zh) on the prefix binding of Fable 5.1 / Opus 5.5 / Sonnet 5.5 / Haiku 5.5 thinking blocks: who is checked, what ContextChef already keeps byte-stable, what still changes earlier turns (tail injections, client-side overflow, compact() clearing), and how a request can opt into thinking.block_binding drop_block as a stopgap, with its model constraints and token cost. Linked from the adapters page and the READMEs.
Phase P1 of the append-only spec. A pure class, not wired yet and not exported from the package root, so P2 can build the pipeline on it and the integration packages can reuse it without ContextChef. The window is rebuilt every compile from the host history and the ledger: the compaction head's replacement, then host messages from head.cut on, with each appended event placed before host[anchor]. The ledger never copies host messages. Anchors are positional and validated by per-message digests (key order, cache_control, _cache_breakpoint, pinned and _anthropic_origin are ignored). reconcile() applies the first mismatch as a checkpoint and reports it (history, system, tools, or ledger, including a positional system event a user turn now follows) instead of realigning old entries. Entries record one source's text each, so the last thing a source appended is read off the entries; one append() call renders as at most one message per channel (joined), and a withdrawal is a tombstone entry (withdrawn). A checkpoint drops what it invalidates, keeps earlier entries in place without letting them speak for any source, and strips thinking from the affected host range unless the compaction is signed; stripped ranges are never given back for a message that still exists. Snapshots are plain data and round-trip through JSON.
…digest Review fixes and author rulings C-F for the P1 ledger. - Unanswered events are provisional (Ruling E): an entry renders before the first assistant message at or after its anchor, or at the tail while no reply exists. Each place renders user events, then system events, then turn-scoped ones, one message each, so a same-length retry never puts a user event after a system one, and a user turn the host appends after an unanswered request is not a break. Once answered, the group renders the same bytes it was sent with. The stranded-system ledger break and the stored `joined` flag are gone; finality is read off the history. - Last appended per source is read from the entries still in the window (Ruling D): entries kept before a break keep their sources, so a source stated before the break and withdrawn after it gets a fresh tombstone. - PrefixBreak.at counts rendered messages, not entries. Drops and compaction absorption follow where an entry renders, not its raw anchor. - The digest is a whitelist of wire fields (Ruling F) and includes the Anthropic origin's identity (Ruling C): host metadata is no edit, dropping or replacing an origin is. - checkpoint(reason, hostIndex, head?) validates its input; commit keeps `sent` as a JSON copy, so functions no longer throw. The ledger keeps the history being compiled, and window() renders over it.
The Chat Completions adapter strips IR-internal fields with a destructuring list and spreads the rest onto the wire. _anthropic_origin, added with the Anthropic verbatim replay, was missing from that list, so history parsed with fromAnthropic and compiled for OpenAI sent the field to the API. Add it to the strip list and cover it with a test that parses an Anthropic history and checks no emitted message carries the key. The Responses and Gemini adapters build their wire objects field by field and were checked not to leak it.
Phase P2 of the append-only spec: the detection half of concept three.
With ChefConfig.appendOnly on, the chef owns a PrefixLedger, and every
compile rebuilds the window from (host history, ledger) in a new `prefix`
phase after transform-tool-results. A rewrite of what was already sent is
applied as a checkpoint and reported through the `prefix:break` event and
CompileMeta.prefix ({ stable, break? }); nothing is appended yet (P3/P4).
Off, the phase does nothing and the output is 4.x, golden unchanged.
- history / ledger checkpoints come from reconcile() in the prefix phase.
clearHistory() resets the ledger, and the next compile reports a `ledger`
checkpoint at 0.
- system / tools checkpoints (Ruling G) come from reconcileSent() once
adapt built the payload: they keep the entries and strip every thinking
block after the head. The sent head is the target's system field
(system / systemInstruction / instructions) and tools; Chat Completions
has no such field, so there it is the system-layer messages the window
follows. Compared as the JSON the ledger keeps, so a toJSON schema is no
change.
- stable compares the windows message by message, never past a checkpoint
of the compile: `at` alone can sit one group late after a rollback.
`seen` digests lead with the role's initial so a restored ledger can
rebuild where its entries rendered.
- The commit happens in `done`. A compile that fails is rolled back, so the
next one meets the same checkpoint and reports it; a compile a hook runs
inside another one leaves the ledger alone.
- Snapshot carries prefixLedger and restore() brings it back with the
history; a fresh ledger meeting a long history warns once.
- PrefixLedger and the prefix types are exported from the package root.
…ver saw P2 review fixes and Ruling I. - What is sent ahead of the window is copied once, where adapt reads it, and the same copy is compared and committed (plainSent). The commit used to copy the payload after the after-adapt handlers ran, so a handler that edited payload.tools in place read as a `tools` checkpoint every compile. - Ruling I: a ledger that has seen nothing, meeting a history that already holds a reply, applies a `ledger` checkpoint at 0 and strips the whole window. Those replies answered requests sent with additions this ledger never recorded. A conversation nobody has answered yet is no break. This covers a chef per request that does not keep prefixLedger, setHistory on a fresh chef, and a history restored without its ledger. The one-time warning now says the thinking was invalidated. - A clearHistory() the next compile has not reported yet survives a snapshot (PrefixLedgerSnapshot.restarted), so a fresh conversation after it is still reported as a `ledger` checkpoint. - restore() reads the ledger snapshot before replacing anything, so a snapshot it cannot read leaves the chef as it was. - ChefEvents['prefix:break'] is derived from PrefixMeta['break'].
Until now every push source was still rendered again at the moving tail on each compile, so under appendOnly the prefix broke every turn: the turn an injection sat in last time changed, and the thinking signed over it could not be replayed. The tail phase now compares each source with the last entry the window holds from it (dynamic state, implicit context, each announcement, the last_user guardrail). Only what changed is appended through the ledger, at the end of the host history or before the assistant prefill it ends with: a changed state as the next revision (<dynamic_state rev="2">), a source gone quiet as a tombstone, an unchanged one not at all. The handoff notice is appended once per window. One compile's events render as one user message and one positional system message, in the stitch's order, and never merge into a host message. Revisions are read back from the last entry, not stored, so they keep counting after a checkpoint. Dynamic state at 'system' becomes a positional system event instead of a system message after the history that Anthropic hoisted. Implicit context is a user event of its own. A 'system' guardrail becomes the last system layer: the same bytes on Anthropic, no longer moving down on Chat Completions, and new options are a `system` checkpoint. The reusable compare-and-append step lives in src/prefix/events.ts for the memory and skill sources. The ledger gains sources() and renderedAt(). With appendOnly off nothing changes: the 4.x stitch moved into its own function verbatim and the golden suite is untouched.
…ed system event as user eventAnchor walked back over every trailing assistant message, so a compile made after a signed reply arrived (to meter it, say) anchored a changed state before that reply. The ledger treats any assistant as the answer to the group before it, so the state joined a group whose bytes the reply's thinking was signed over: the next turn sent them changed, with no checkpoint. It now steps back over at most one trailing assistant, and only over a prefill, never one that carries thinking, tool calls or a provider's own fields. A system-channel event whose place has no user turn before it (an empty window, an assistant greeting, a reply right before) was a leading or misplaced positional system message, which Anthropic hoists into the top-level system prompt; the next compile then reported a `system` checkpoint, or the window kept it hoisted. The ledger now decides this when it renders, as 4.x did: such entries are said as a user message and keep their channel, so the per-source comparison does not re-append them. The committed-window rebuild in stable() applies the same rule from the role initials it keeps. Also: - the splice that replaces the event group now counts what follows it (the history's prefill, the guardrail's), rather than skipping every trailing assistant, which skipped the reply too once events can follow one; - a restated last_user guardrail says it replaces the earlier one, and the user-channel guardrail is withdrawn when it is cleared or moves into the system layers (pointing at the system prompt then); - the fallback for events a handler moved adds only this compile's events, and its warning says what happened; - the 'auto' channel rule and the announcement block are defined once and used by both the 4.x stitch and the event path; assemble tells tail that the guardrail went into the system layers.
…Only Under appendOnly the tail still merged skill instructions and memory data into the last user message on every compile, often a host message, so the turn before changed each time and the prefix broke with no checkpoint. Both are events now, and the 4.x stitch is gone from the appendOnly path. Memory is pulled first. The window starts with one snapshot of what the selector picks (the vocabulary's memoryBlock), as a user event. After that a delta is appended only for what the model was not told: the host's own writes and TTL expiry, as changed entries and <entry key="…" removed="true"/>. What the model was told is read back from the memory entries still in the window (Ruling D), so a checkpoint that dropped the snapshot or a delta gets it stated again, and a compaction in P6 re-snapshots by the same rule. Writes the model made through handleTool are not pushed back: handleTool runs the dispatch in an AsyncLocalStorage context, and the memory bridge records each such write as the model left it. A host write running at the same time is still the host's. An event that states the key again, a history or ledger checkpoint, and restore() all forget the record. A selector that picks nothing pushes nothing. An entry the selector stops picking is not reported as gone: compileArtifacts now also returns the live entries, from the same store read, and the selector gets a copy so it cannot trim them. Skill instructions are a system event: appended once when a skill is activated or switched (a switch says it replaces the earlier one), with a tombstone when none is active. appendOnly changes defaults, never explicit choices (Ruling T3): an unset skillPlacement is 'tail' and an unset memoryPlacement 'before_history_tail', both events. An explicit 'after_system' keeps the system message, where each change is a `system` checkpoint. CompileMeta.injectedMemoryKeys reports the keys this compile's snapshot or delta speaks for. With appendOnly off nothing changes; the golden suite is untouched.
…ep the record in snapshots handleTool ran the dispatch in an AsyncLocalStorage context and counted every memory write in it as the model's. That context reaches everything the call sets off: memory:changed listeners, the onMemoryChanged and onMemoryUpdate hooks, and timers or promises they start, even after the call returned. Writes the host made from any of those were never pushed, and re-reading the entry afterwards could pick up a concurrent host write. The dispatcher now reports each write it makes itself, through an optional ContextToolHost.memoryWritten(key, entry | null) called from the memory-write branches with the entry the write returned (null for a delete). Memory's own methods carry no marker, since host code calls them too. The chef wires the callback only under the appendOnly event model, so an explicit 'after_system' records nothing. node:async_hooks is gone. The record is now an optional ChefSnapshot field, modelMemoryWrites, present when appendOnly is on. A chef created per request restores it with the history and ledger, so it no longer states the model's previous writes again. A snapshot without the field restores with none recorded. Also: - Memory renders one entry with an exported renderMemoryEntry, which _renderXml composes, instead of the memory phase slicing the <memory> wrapper off toXml([entry]); the phase is synchronous now. - Docs: a host change to an entry the selector hides is not pushed, but its removal is once the model saw or wrote it, even under a selector that picks nothing; any checkpoint forgets the record, so a model write may be stated once more after one; ContextChef, not Memory, supplies the appendOnly default placement. - Tests: host writes from a listener, from both hooks and from a timer during a tool call are pushed; per-request snapshot round trip; a skill switched away and back within one compile; re-activation after a tombstone; ms-based TTL; continuity on openai-responses.
…ped reminders under clearAt
appendOnly recorded which thinking an unsigned checkpoint invalidated but
still sent it. The adapt phase now renders the messages without the thinking
in the ledger's stripped ranges (blocks known by signature, copies only, the
host's objects untouched), and renders again when a system or tools
checkpoint strips the whole window in the same compile. A message parsed by
fromAnthropic falls back to its IR rebuild once its thinking is gone. OpenAI
and Gemini strip nothing.
appendOnly: { clearAt: true } on Anthropic makes the system-channel sources
(skill instructions, dynamic state at 'system', system-channel announcements)
a turn-scoped reminder in every request: clear_at on the wire, the beta in
payload.betas, kept in place once cleared, stopped rather than tombstoned when
the source goes quiet. A reminder is only appended where a user turn comes
right before it: said as a user message it could not be turn-scoped, so a
request without one goes without.
Anthropic primitives:
- a thinking block without a signature is not sent (was signature: '');
- fromAnthropic(messages, system, { keepPositionalSystem }) keeps the host's
own mid-conversation system messages, clear_at included;
- anthropicCompactionMessage(block) replays a signed on-demand compaction
block; a signed head adds compact-2026-09-04 to payload.betas;
- a breakpoint on a turn-scoped message moves to the message before it.
Events now go in front of host system messages the history ends with, since
a user message right after a mid-conversation system message is a 400. Under
appendOnly a guardrail prefill with no prefillMode is an instruction on
OpenAI and Gemini, whose adapters would fold it into a different user message
every turn; an explicit 'native' is kept and warned about. The cache audit
reads chef events in messages as appended once. appendOnly off compiles what
4.x compiles; the golden snapshots are unchanged.
…ve a hoisted system run alone A stripped turn fromAnthropic parsed was rebuilt from IR: a tool turn went out with an empty text block in front of its tool_use, and a server-tool turn lost its search results and citations and had its text blocks joined. Removing thinking is the only edit the signature check allows, so the strip step now uses withoutThinking(): when the message still matches its original, the original loses only its thinking blocks (with a new hash) and every other block is replayed as sent. A message its original no longer matches, or one that held nothing but thinking, is rebuilt from IR as before. Events went in front of every system run the history ends with, including one after a reply, which the adapter hoists. In front of the event it became positional, and the next turn hoisted it again: two `system` checkpoints, each stripping every thinking block. eventAnchor and the ledger's placement now step back only over a run that follows a user turn, judged at the cut by what the head ends with. Whether a target has a native prefill is now the adapter's own `nativePrefill` (false on the built-in OpenAI, Responses and Gemini adapters) instead of instanceof checks, so subclasses inherit it and wrappers can say so. reconcilePrefixSent no longer reports whether to render again; adapt compares the stripped ranges itself.
Under appendOnly, overflow still ran as in 4.x: a landed summary rewrote the start of the window without a checkpoint, and since 991a669 the in-flight mode summarized again on every compile, so every turn was an unreported rewrite of what was already sent. Now a landed result becomes ledger.checkpoint('compaction', cut, head). The head is everything the result puts before the first host message it kept (the summary, pinned messages put back, whatever of the previous head it kept); later windows replay it, and nothing is summarized again until the window, head included, is over budget. The window is rendered from the ledger afterwards, so an unlanded result, or an onBeforeCompress rewrite, never reaches the payload. An unsigned compaction waits for the end of a tool turn (D3). It reads the budget without consuming the reported usage, so the boundary compile still sees it. When it cannot wait (the window is past the raw contextWindow, or new_context / requestNewContext() was called, T2) it collapses: the strategy gets OverflowInput.collapse and summarizes the whole window, the turn in progress included. Pinned messages the head carries over are stripped of their thinking on Anthropic along with the kept tail. OverflowInput.wire (Anthropic, once a payload was committed) renders the window as it was sent, for the signed strategy P8 adds. A prefix:break event carries the window it applied to: a history/ledger checkpoint the window the compile started in, a compaction the one it opens.
…apsed window, strip all of an unsigned head Review of 39da2a5, all three probe-confirmed: - A compile that waited for a tool turn to end never ran the runner's budget check, so the E10 flag a landing set survived the wait and dropped the accurate usage on the boundary compile. The wait reading now applies E10 itself (Janitor.dropStaleUsage) and also counts the window being sent (Janitor.estimateTokens), since reported usage describes the previous request. A collapse skips the runner's budget verdict: the phase already found the window cannot wait. - summarize() with no compression model (the 4.x default) collapsed a window into nothing: messages: [] on Anthropic. It now leaves the reset() notice in its place. - An unsigned head carried valid-looking thinking whenever its messages were not the window's own objects (an onBeforeCompress hook or a strategy that copies), since stripping went by host ranges and identity. Nothing in an unsigned head was signed over what now precedes it, so all of its thinking is left out and counted, by signature, which replaces the identity-based carried ranges. Also (Ruling N): a history ending in tool calls not answered yet is mid tool turn. And the prefix:break and after-overflow docs say what a compaction does not roll back and what result.history means under appendOnly.
…ver orphan a tool result Ruling O replaces the max(reported, local) collapse check from 9a1a743. With no tokenizer the local count JSON-stringified attachments, so a screenshot loop collapsed the whole window every couple of images at a few percent of real usage, and the max overrode an explicit 'feedFirst'. The check now reads Janitor.readRequestBudget: the last reported usage plus a local count of only what was appended since the last committed payload (the window without a report), each attachment at a flat ATTACHMENT_TOKENS, with usagePreference applied as in readBudget. Ruling P: a collapse summarized an assistant message whose tool calls had no results yet, so the result the host appended next followed no tool_use, which Anthropic rejects on every later request. The call now stays as the first kept turn (its thinking stripped, the head being unsigned), also when summarize() has no compression model and leaves the reset notice.
Ruling P covers every strategy a collapse can run, not only summarize(). reset(), on its own or as a chain's fallback, evicted a tool call still waiting for its result, so the result the host appended next followed no tool_use. Under collapse it now keeps that turn after the pinned messages, like summarize() and anchored() do. The rule moves to turns.ts (waitingCallStart) so both read it from one place; without collapse reset() is unchanged.
…count attachments flat in every appendOnly reading Ruling Q: the collapse check counted on top of a report only what the window gained since the last committed payload. While compiles wait, the report is not consumed, so it can describe a payload several commits old, and what came between was counted nowhere: with one report and no more, a window at 148% of the raw context window kept waiting. reportTokenUsage() now marks the window it covers in the ledger (PrefixLedger.markReported(), kept in its snapshot, cleared by reset()), and the check counts everything after that. Ruling R: under appendOnly the runner's trigger reading still counted attachments by their base64, so a screenshot loop that waited correctly through its tool turn compacted at the boundary on a large overestimate. The Janitor now takes its no-tokenizer count from an internal `estimate` option, which the chef sets to estimateMessages under appendOnly for every reading; without appendOnly it stays the 4.x estimateObject.
…vered by it Ruling S: the report marker recorded how many leading window messages the reported request held, but a host that replaced, regenerated or dropped one of them in a tool turn kept the marker past the change, so the new content was taken as already counted. A placeholder result filled in with 40,000 characters was read at the old report and sent at 145% of the raw window, and every retry without a report read the same. The collapse check now covers at most the messages the window still shares with the last commit (min of the marker and stable()), and commit() cuts the marker back to that shared prefix, so an edit in a compile that fit is still counted once a later compile grows past the limit. A rollback leaves the marker alone: nothing the failed compile built was sent.
…ackground jobs on a window that only grew Under appendOnly the window overflow sees holds the chef's events among the host's turns. groupIntoTurns made every event a turn of its own, so a strategy could keep turns that start with an event: the event was summarized and restated after the kept turns, and keepLast(1) on a retried request kept only the event and summarized the question it belonged to. A provisional event after a pending tool call also hid that call from waitingCallStart, so a collapse summarized it and orphaned the result that came next. The ledger now marks every message it renders `_chef_event` (provenance, not a guess from content; the OpenAI Chat adapter strips it, the other adapters build their wire field by field). groupIntoTurns puts an event in the turn it follows, tool results after it included; pinning a turn does not pin its events, which the ledger states again. Without appendOnly no message carries the mark, so 4.x grouping is unchanged. background() learns two appendOnly rules, read off a new OverflowInput.appendOnly the pipeline sets: - a job is fresh while the window it ran on is still the start of the current window, message for message as checkPrefixContinuity compares them; any checkpoint that rewrote it, kept turns included, makes it stale. Turns that arrived meanwhile are kept. 4.x keeps comparing only the ends of the span. - a collapse cannot wait for a job: the inner strategy collapses the window on the spot and any job is dropped (a finished one is credited to the breaker). Jobs otherwise start and land only between turns, since the pipeline does not call the runner mid tool turn.
… request past the raw window at a turn boundary A background job's freshness compared the window it ran on with the payload comparator over IR, which reads host-only fields: a host that marks its newest message for caching on a copy every compile moved `_cache_breakpoint`, so every job went stale with no checkpoint reported and the window grew past the raw context window. Freshness now compares messages by the ledger's wire-field digest (`wireDigest`, now exported from prefix/ledger), plus `pinned`: a message pinned since the job started was summarized instead of kept. The overflow → pipeline import is gone. At a turn boundary background() still returned the window unchanged when it started a job, waited on one in flight, or restarted a stale one, so a window past the raw context window went out as it was (a host compiling again only at q7 sent 13 messages against a window of 10). The overflow phase now reads the request about to be sent at a boundary too, with the reading the collapse check uses (the last report plus what it does not cover); past the raw window it passes `pastLimit`. The runner then skips its own verdict, which may read only an earlier report, and background() runs its inner strategy on this compile instead of starting, awaiting or restarting a job; a finished job that still applies lands as before. The `_chef_event` JSDoc says where the mark reaches host-facing data.
…ded background job still past the raw window A usage report held across a checkpoint was stacked on top of the window from the changed message: the report still counted the old messages and the local count added the new ones. An early edit with a report of 50 read 120 against a raw window of 100 and compacted a 70-token window. Once a message of the request a report came back for changes, the report describes no window that exists: `PrefixLedger.reportCovers()` returns 0 then, commit() keeps it at 0 until the next report, and the request reading counts the whole window locally (`readRequestBudget` with `appended` null). Both the turn-boundary and the collapse check read it. Under `pastLimit` a fresh finished background job landed whatever the window it left: a job from q4 with the host back at q12 sent 18 messages against a window of 10. When the landed window still counts past the raw window, background() now runs its inner strategy on it in the same compile; both are one overflow, and commit reaches both inner results. A message the job kept that was pinned since no longer discards it: the pinned comparison covers only the span it summarized. And with no finite contextWindow (the default) the request reading is skipped, so such a chef counts no more tokens than 4.x.
…ough its refit, and count a refit that fails The refit of a landed job still past the raw window picked its pinned messages by filtering input.pinned against the landed window by object identity. The messages the job put back as pinned are its own copies, so a host that builds its messages anew per request (fromAnthropic does, and pins compaction messages) lost them: the second run summarized them away. The refit now reads the pinned set off the landed window with the runner's own predicate (extractPinnedMessages), for summarize() and reset() alike. A refit that changed nothing still landed the job as a success, so the runner cleared the breaker and the inner failure was forgotten while the window went out past the raw limit. Such a result is now `meta.partial`, and the Janitor does not clear the breaker for it; 4.x never sets it. Freshness compared `pinned` only when the object differed, so pinning the very object the job read went unnoticed and the job summarized it. A job now records each message's pinned flag when it starts, and the summarized span must still carry the same flags when it lands.
…ground job by what it read when it started A background() refit that failed inside a chain() was composed into a plain changed result, so the Janitor cleared the breaker after all. chain() now marks its result `partial` when any step was, whatever the steps after it did. Freshness still let a message through when it was the very object the job read, so a host that edited that object in place after the job started got a summary of the old content. Under appendOnly a job now records, when it starts, each message's wire digest (the ledger's) along with its pinned flag, and at landing every message must still have that digest, same object or not; pinned is still compared over the summarized span only. 4.x records nothing and keeps its own check.
…d chain()'s pinned set off the window A landed job whose refit failed came back `partial`, so the Janitor kept the breaker as it was, but nothing had credited the job that succeeded. Failures counted before it stayed counted on top of the refit's own: one failed job, then a job that landed with a failing refit, left the breaker at 2; with two failures already counted it opened at 3, and the next request went out past the raw window. fitLanded now calls runner.succeed() before the refit, as now() and the stale path do for their finished job, so a failing refit leaves exactly its own failure. chain() passed pinned messages on to the next step by filtering input.pinned against the new window by object identity. A landed background() job puts back its own copies, so with a host that builds its messages anew per request, chain(background(summarize), reset()) reset the pinned message away, with or without appendOnly. The next step's pinned set is now read off the window with the runner's own predicate (extractPinnedMessages). This is a 4.x behavior fix; with the same objects the set is unchanged, and the golden output is too.
…hat the window flags a814d62 read the next step's pinned set off the window alone, so a direct caller's own pinned list, messages no `pinned` flag marks, was lost after the first step that changed the window: chain(dropOne, reset()) with q3 handed in as pinned reset q3 away. A message now stays pinned for later steps when the caller handed it in or the window flags it. In-tree the runner hands in exactly what the window flags, so nothing changes there, and a landed background() job's own copies stay protected. The Janitor comment on a partial result no longer says the count is the failing step's alone: in a chain a later failing step counts too.
…turns kept after it keep their thinking
Every client-side compaction so far is an unsigned checkpoint: the turns
it keeps lose their thinking, so it waits for a tool turn to end and
collapses the window when it cannot. Anthropic's on-demand compaction
(beta compact-2026-09-04) returns a block the provider signs, which
legally replaces the messages it summarizes. onDemandCompaction({
compact, keepRecentTurns, instructions, fallback }) uses it:
- It cuts where the request the model already answered ends: the
summarized messages end in a user turn whose tool calls all have their
results, the kept ones start with the reply, keepRecentTurns of them
(0 summarizes everything, as does a collapse). The old block is always
inside the range, pinned or not.
- The request is wire.renderPrefix(cut): the system prompt and tools last
sent, the messages byte for byte as sent, compaction: { type:
'summarize', instructions? } (instructions validated at 16,384 chars),
the payload's betas plus compact-2026-09-04, never context_management.
It is rendered before anything is awaited, and the host gets a copy.
- A response that is not stop_reason 'compaction' with one signed block
is a failure the breaker counts; the window is left alone. Without a
wire (another target, nothing sent yet) the fallback runs.
- The result carries meta.signed (the request it was signed over), so
the overflow phase lands it as a signed head that strips nothing. A
signed strategy handed a wire is exempt from the wait/collapse timing:
the turn in progress keeps its thinking too.
- Pinned messages it summarized are not put back between the block and
the kept turns: the tail states them again as a `pinned` event, every
time a compaction absorbs that event, and withdraws it once unpinned.
- background(onDemandCompaction()) lands a job only while the wire still
renders the request it was signed over (a system or tools checkpoint
changes no message), and its Ruling W refit asks for the window the
compile was handed, since the job's own block was never sent.
- A signed head vouches for its block alone: what it keeps of an
unsigned head before it stays without the thinking it lost there.
server() and onDemandCompaction() never both run, so configuring them
together warns once at construction (the tree is walked through the new
internal OverflowStrategy.parts). signed propagates through background(),
server()'s fallback and a chain whose every step is signed; a chain
passes on a signed result no other step changed. onCompress gets the
compaction message for a signed result instead of the placeholder.
OverflowWire is exported, with the window it renders (history).
appendOnly off and the strategy unused: output unchanged, golden too.
…the window summarize(), anchored() and reset() put the pinned messages of the span they summarize back after their summary or notice. A compaction block is pinned (fromAnthropic pins it, and so does anthropicCompactionMessage), so a history that opens with one came back as [summary, block, ...]. The API reads nothing before the block and rejects a message ahead of it (compaction_block_misplaced): a 4.x host resuming an Anthropic session from fromAnthropic got a request it could not send the first time the window was compacted, and under appendOnly so did an unsigned compaction after a signed one, such as chain(onDemandCompaction(), summarize()) whose later signed attempt fails. The three now build their head through one helper, summaryHead(): a compaction block among the pinned messages goes first, the summary after it, the other pinned messages after that. The runner finds the summary message after the leading blocks (summaryIndex()) when it re-renders it with the archive citation and when it hands it to onCompress, instead of reading history[0]. A history without a compaction block is rebuilt exactly as before; golden output is unchanged.
Calls on one integration session run one at a time, and a call aborted while an earlier one was still compiling kept its place: it waited for that call to finish before its own compile threw. It now rejects with its abort reason as soon as the signal fires, and the call after it still waits for the calls ahead. A summarizer that calls back through the same middleware on the same session can never get its turn, since the call ahead of it is the one waiting for its summary. The session JSDoc, the AI SDK's compress.model JSDoc and README rows, the append-only guide and both integration changesets now say to pass the model itself.
…essage' moves The appendOnly construction warning named compact.reasoning: 'before-last-message' among the settings that rewrite messages an earlier call sent. It keeps reasoning on the last message of a request only, which is the new question or a tool result, so every reply loses its reasoning before any call sends it, exactly as with 'all'. The warning no longer names it, the warned-options type drops the field nothing reads, and the docs name the moving compact setting as compact.toolCalls.
…trip' The durable helpers' 'strip' removed every reasoning part (AI SDK) or every thinking block (TanStack) from the kept messages, though only a signature signed over the messages a summary replaced stops verifying. OpenAI's or Gemini's reasoning was thrown away for nothing. 'strip' now removes what a checkpoint removes inside appendOnly: reasoning signed through @ai-sdk/anthropic or @ai-sdk/amazon-bedrock, and TanStack thinking blocks that carry a signature. A TanStack reply keeps its unsigned blocks. The AI SDK's isReasoningPart had no other user and is gone.
… under appendOnly The transformContext docs suggest gating the hook on ctx.phase === 'init' to keep it idempotent. With appendOnly the init firing passes the config through untouched, so such a hook is never called. The JSDoc, the README, the package page and the append-only guide now say so.
On TanStack AI the usual way a conversation loses its session is a chat() run without a threadId: the engine makes up a new one each run, so every run starts empty. The warning only talked about middleware instances created per request. createPrefixSessionPool takes an optional sessionKeyAdvice sentence that opens the warning's advice, and the TanStack middleware passes one telling the host to pass the conversation's own threadId to every run.
When a chat() run was aborted, its onConfig work throws, and the catch logged "onConfig transform failed; passing the request through unchanged" although nothing of the run goes out. The catch still passes the config through, but no longer warns when the run's signal is aborted. With appendOnly off this only shows when a hook of the host's throws after the run was aborted, so it gets a patch changeset.
…gned is always false PrefixSession.compile said a call that fails keeps nothing. Only the ledger is rolled back: a compression that landed before the failure has fired onCompress, and the runner keeps the state it moved to. Both onPrefixBreak JSDocs and the append-only guide now say what PrefixBreakEvent.signed means and that it is always false in the middlewares, which cannot request a compaction the provider signs.
…xt store, end to end The write-back tests fed the middleware hand-built provider prompts. This one runs seven generateText calls against a model whose replies carry @ai-sdk/anthropic signatures, with a host that stores ModelMessages (response.messages included) and writes details.replacement back from onCompress. The summary lands once, the store that took it in is read as continuous, and the reasoning after it is replayed with its signatures.
The skill and its API quick reference still named 4.2 as the version they target, while they already document the 4.3 appendOnly API. The per-feature "4.2" markers say when a feature arrived and stay.
…ad grow The warning for compress firing without onCompress said the payload grows without bound and eventually overflows the context window. Since the call after a compression checks the budget again and a usage report only counts while the history holds the summary, what is sent stays compressed. The real cost is a summarizer call on every call over budget (every chat() call with TanStack AI), over a history that keeps growing, and both warnings now say so. The pool's JSDoc and the AI SDK's Chinese README, which still said compression fires every other call, are corrected too. Patch changeset for the 4.x log line.
The overflow changeset named the new optional field without saying why it is there: a chef restored from the snapshot still counts a usage report of the compressed history the host kept, and a snapshot saved without it restores with that report ignored.
…o the abort The session JSDoc put "so the summarizer must never call back through the same session" after the sentence about aborted calls, though it follows from calls running one at a time.
…rted before it lands The Janitor checked the signal only after the strategy ran, and background() takes its finished job as soon as it swaps it in. A compile aborted before the strategy ran (in a compress:start handler, say) or during an earlier chain() step threw the job away, and the next compile summarized again. Check the signal before the strategy runs and before each chain() step.
…e signal's reason A client that throws its own error class on abort made the compile reject with that error, where every other aborted compile rejects with the signal's reason.
The ledger left the package root, but PrefixSession is exported and its public ledger field still reached the whole ledger API. The test that read it already pins what the reading is for through the session's own behavior.
…ADME A request past the window gets a compaction attempt through an open breaker, not a guaranteed compaction; the TanStack summarizer cost is per chat() call over budget; compress.model takes a LanguageModelV4.
…nDemandCompaction() fall back A test pins it: through a wrapper that passes the flag on, onDemandCompaction() asks for a compaction; through one that doesn't, it runs its fallback.
Nothing has read it since the stripped-thinking count moved to the messages a checkpoint invalidated.
background() took sameOnWire from prefix/wire while the pipeline checks took it from the prefix index; onDemandCompaction() and the Anthropic adapter reached into prefix files the same way. Every runtime import from outside prefix/ now goes through its index.
…he real engine The structured call is a one-off only because the engine hands its firing the run's tools; until now only a simulated firing showed that. With tools the usage it reports is not counted, and without tools the same run is an ordinary call whose usage is.
…eturns withContextChef returns a LanguageModelV4, and fromAISDK/toAISDK and onCompress's details use LanguageModelV4Prompt; the READMEs and the docs-site page still said V3. ai's LanguageModel accepts V4 as well as V3 and V2. The zh README still documented compactHistory/planCompaction, which the middleware no longer exports; the en README dropped them already.
… at the next step
…ocks A message rebuilt from IR with no text went out with a text block holding '': a tool-call turn as [text '', tool_use], a thinking reply as [thinking, text ''], an attachment-only user message as [text '', image]. The model reads nothing from it, and if the API rejects one the request fails; it also made a tool-only turn differ from what the model sent. The text block is now left out when it is empty and the message has another block. A _cache_breakpoint meant for it goes on the last block (none for a thinking-only reply, as in the verbatim replay). A message with nothing else keeps it, and whitespace-only text is still sent. The appendOnly goldens change only by the 44 empty text blocks removed; the 4.x goldens don't change.
examples/verify-thinking-prefix.ts sends the request shapes ContextChef produces to the Messages API with plain fetch and prints ACCEPTED or REJECTED <status> <message> for each, covering the Anthropic lines of TODO.md "Needs an API key": stripping thinking mid tool loop, a user event and split tool results after tool_result, clear_at clearing, a system message after a tool result, the on-demand compaction shapes (kept reply after the block, keepRecentTurns 0, clear_at at the end, a system prompt change, cache_control on the last block), a server-side block with compact-2026-09-04, empty text blocks, and the appendOnly on/off agent loop. Every request sets prefix_mismatch_behavior "error" so the check runs on any account; --diagnostic sends the beta without block_binding and prints input_transformations. --dry-run prints the bodies against canned replies without a key. The AI SDK, TanStack and OpenAI / Gemini lines are left out.
…breakpoints The examples index (en and zh) now lists the script that runs the thinking prefix experiments against the API: it needs ANTHROPIC_API_KEY, and --dry-run needs no key. TODO.md "Deferred (append-only)" notes that a _cache_breakpoint on a tool-call turn or a tool result emits no cache_control on the Anthropic wire, which predates appendOnly; fixing it changes 4.x output.
The thinking check ignores empty and whitespace-only text blocks, and the leading and trailing whitespace of any text block, so leaving such a block out never changes what a signature covers. 6db8d13 kept whitespace-only text on the belief that dropping it would edit the turn; it doesn't. Text that is empty or only whitespace is now left out whenever the message has another block, which matches sendsNothing(), and a message with nothing else keeps its text block as it is. The doc comment and the changeset now give that reason instead of an unverified claim that the API rejects an empty block, and say that a conversation an earlier version sent with such a block keeps its thinking valid. New tests: whitespace-only text next to thinking, tool calls and attachments; text with whitespace around it kept; a breakpoint landing on the last attachment after thinking, and on a rebuilt compaction block. Neither golden suite changes: no fixture has whitespace-only text.
After each experiment the script now prints one note with the expected outcome of each request and control, and the library decision each result drives, taken from the TODO.md lines and the guide: for instance, REJECTED for strip-mid-tool-loop makes waiting for a user-turn boundary a hard requirement, ACCEPTED a quality choice. server-block-beta now resumes from the whole reply that carried the block, as fromAnthropic replays it, instead of the block alone, which its comment claimed was what fromAnthropic does. The clear-at-reminder reminder is five times larger (~5k tokens), so the cleared / not cleared verdict stands well clear of the noise from differing thinking lengths. The empty-text question now covers whitespace-only text, which the adapter leaves out too.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5 sign every thinking block over everything that came before it. On accounts created on or after 2026-08-31, a request that replays such a block after any change to earlier history is rejected with a 400. ContextChef used to re-render its injections (dynamic state, announcements, memory, skills, the guardrail) at the tail of every compile, so the turns they sat in last time changed on the next request.
This PR adds opt-in
appendOnly, which makes every request repeat what the previous one sent and only append. It ships as core 4.3.0, ai-sdk-middleware 3.2.0 and tanstack-ai 1.2.0, all as minors. WithappendOnlyoff, nothing changes. The branch also fixes several 4.x bugs found along the way; each has its own patch changeset.Guide:
docs-site/guide/append-only.md(en and zh).Motivation
Changes
Layer 1: keep earlier turns byte-stable
fromAnthropichistory replays verbatim while unmodified: thinking, separate text blocks, parallel tool results, server tool blocks and compaction signatures.checkPrefixContinuity(prev, next)lets you test your own setup.withGuardrails({ prefillMode: 'instruction' })serves the models that reject a native prefill.Layer 2:
ChefConfig.appendOnlyclearAton Anthropic.meta.prefixand theprefix:breakevent. Each checkpoint strips exactly the thinking it invalidated. History edits, system or tools changes, compactions and lost state are all reported.onDemandCompaction({ compact, fallback })produces a compaction block Anthropic signs, so the turns kept after it keep their thinking. Itsfallbackruns where signing isn't possible.pipelineChecks: an unannounced divergence raisespipeline:invariant.onCompressdetails carry the exactreplacementto write back. The guide gives the per-request recipe: save twice, restore without memory, configure after restore.Integrations
appendOnly,onPrefixBreakandkeptReasoningin ai-sdk-middleware and tanstack-ai, through one shared session helper in core.4.x fixes (patch changesets)
summarize/anchored/reset/compactHistory, and no split lands inside it.chain()keeps pinned messages across abackground()step, and a call made only to land a finished job goes only to the step that holds it.Testing
pipeline:invariantevents.pnpm typecheck, plus the ES2022 compat check).biome check .).vitepress buildandchangeset statusare clean.Each phase was reviewed, then fixed and re-reviewed with probe-verified findings. A final whole-branch review across core prefix/pipeline, overflow/adapters and the integrations was followed by one fix wave and a scoped re-review, and came back clean.
Not verified (no API key). TODO.md → Needs an API key lists one experiment per open question, and
examples/verify-thinking-prefix.tsruns every raw-API one against the Messages API with your own key (ANTHROPIC_API_KEY;--dry-runneeds none) and prints, per experiment, the result and what it means for the library. The design assumed the strict answer everywhere, so a lenient result would let the library strip or wait less, not fail.Breaking changes
None. Everything new is opt-in, and with
appendOnlyoff, 4.x output is unchanged apart from the bug fixes above. TODO.md → Before release lists two decisions to make before 4.3 ships.