feat(chat): hand run() a streamText with the managed options already applied - #4884
feat(chat): hand run() a streamText with the managed options already applied#4884ericallam wants to merge 9 commits into
Conversation
🦋 Changeset detectedLatest commit: bf66457 The changes in this PR will be included in the next version bump. This PR includes changesets to release 27 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review. 📜 Recent review details⏰ Context from checks skipped due to timeout. (48)
🧰 Additional context used🪛 LanguageTool.changeset/managed-streamtext-in-run.md[style] ~5-~5: To strengthen your wording, consider replacing the phrasal verb “leave out”. (OMIT_EXCLUDE) [style] ~12-~12: ‘by accident’ might be wordy. Consider a shorter alternative. (EN_WORDINESS_PREMIUM_BY_ACCIDENT) [style] ~14-~14: To form a complete sentence, be sure to include a subject. (MISSING_IT_THERE) [grammar] ~18-~18: Ensure spelling is correct (QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1) 🔇 Additional comments (1)
WalkthroughThe SDK now supplies managed Merge Risk: 🟡 Moderate · up to The managed streamText behavior is documented more broadly, but several published examples remain inaccurate. Most can cause incorrect integration behavior; the backend example may encourage exposing another user’s data to a model provider without server-side authorization. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 6 files. (1 skipped: 1 unsupported.)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
4984f70 to
f36bb10
Compare
@trigger.dev/build
trigger.dev
@trigger.dev/core
@trigger.dev/python
@trigger.dev/react-hooks
@trigger.dev/redis-worker
@trigger.dev/rsc
@trigger.dev/schema-to-json
@trigger.dev/sdk
commit: |
|
@coderabbitai review |
✅ Action performedReview finished.
|
f36bb10 to
57bb078
Compare
1a1da14 to
45bafb0
Compare
fa5e2a8 to
e6709a8
Compare
The bound streamText on the run and onAction arguments is #4884's, so on this branch alone the test neither typechecked nor ran.
06919ac to
0e34286
Compare
0e34286 to
3203120
Compare
The edit an action makes is snapshotted before the turn chat.turn()
requests begins, so a turn that is cancelled or runs out of memory
continues from the edited history rather than from the snapshot the edit
replaced. The turn's run() payload carries trigger "action-turn", not
"action", so a handler that returns early on the action trigger still
answers. An action sent through useChat keeps the request's metadata.
Also moves the action-turn test onto this branch's own surface; it had
used the bound streamText and chat.agent({ system }) from #4884.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxuSksX18bj1yhnLpkcQ6a
3203120 to
b1fdca8
Compare
b1fdca8 to
871f86c
Compare
Spreading chat.toStreamTextOptions() is the integration point for six things:
the managed prompt and its cache control, the resolved model, the prompt's
sampling config, telemetry, the skill tools, and the prepareStep that delivers
steering, compaction and injected context. Forgetting the spread drops all six
in silence, and spread order decides whether passing your own tools or
prepareStep clobbers the managed ones.
run() now receives a streamText with those options applied, so the managed
state cannot be lost by omission and the merge happens inside rather than at
the call site: tools go into the helper so skills survive, a caller system
becomes the base the prompt and injections append to, and a caller prepareStep
composes after the managed one instead of replacing it.
The signature is borrowed with typeof import("ai").streamText rather than
restated, so it resolves to whichever of ai v5/v6/v7 the user installed. The
runtime value rides the existing ESM/CJS shim that already isolates value
imports from ai.
PROTOTYPE. Typechecks and passes the suite on ai@6.0.116 and ai@7.0.66, but
adds a public registry option, does not settle what happens when caller and
managed system are both structured, and has no test for the composed
prepareStep.
An onAction handler has no tools in scope the way run() does, so a
regenerated answer built with the bound streamText could call nothing.
Omitting tools now falls back to chat.agent({ tools }); naming tools
still replaces the set for that call. onAction also receives tools.
chat.agent({ system, registry, cacheControl, systemProviderOptions })
reached only the bound streamText; the documented spread form ran without
them. The options are published for the run and toStreamTextOptions
defaults from them, caller options winning.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxuSksX18bj1yhnLpkcQ6a
An action no longer calls the model itself; one that returns chat.turn() is followed by run(), which already receives the bound streamText and the agent's tools. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AxuSksX18bj1yhnLpkcQ6a
onAction returns nothing or chat.turn(); the section on returning a response from an action is replaced, the frontend sends actions through useChat, and the reference drops the onAction streamText argument. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AxuSksX18bj1yhnLpkcQ6a
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AxuSksX18bj1yhnLpkcQ6a
…align the Head Start docs chat.toStreamTextOptions() read the agent-level system, registry, cacheControl and systemProviderOptions from a key set only inside the snapshot boot block, which a hydrateMessages agent skips, so the spread form dropped them there. The key is set on every boot now, with a test for the hydrateMessages case. Docs: the actions gating example returns chat.turn() instead of a streamText call onAction no longer receives; the extracted-loop examples import ChatStreamText, stepCountIs and ModelMessage; the Head Start pages and the chat-server docstrings list prompt among the owned options and describe tools as caller-supplied.
…ool statements The actions lifecycle flow only covered the edit-only path; it now says what happens when onAction returns chat.turn(). The Head Start step said the bound streamText already carried the caller's tools, and the migration example kept a spread comment for a call with no spread.
c9336c5 to
bf66457
Compare
Summary
Every
run()had to spreadchat.toStreamTextOptions(), and leaving it out dropped six things with no error: the managed prompt and its cache control, the registry-resolved model, the prompt's sampling config, telemetry, the skill tools, and theprepareStepthat delivers steering, compaction and injected context.Before:
After:
streamTextcomes fromrun's argument and shadows the one imported fromai, so the correct call is now the shorter one and the managed options cannot be lost by omission.chat.toStreamTextOptions()is unchanged and still supported, and is still the only option in a custom agent.What changes when your options collide with the managed ones
Spread order decides the outcome today, and losing is silent:
The managed
streamTextmerges instead.toolsare passed into the helper so skill tools survive, and aprepareStepyou pass runs after the managed one rather than replacing it. Everything else you name is left alone and wins, telemetry included.systemis the exception: it can be set onchat.agent({ system }), throughchat.prompt.set(), or at the call site, but only in one of them. Two at once throws and names the one that already owns it. No shape merges two system values across every supported AI SDK version, since v5 rejects an array of blocks and a structured block carries the provider options that make prompt caching work.chat.headStart and chat.startHeadStart
buildStreamTextOptionssuppliesmessages,stopWhen: stepCountIs(1)andabortSignal. Step 1 belongs to the route handler and step 2 onward to the agent, so re-settingstopWhenafter a spread hands over a stream that has already run past step 1.Before:
After:
Passing
messages,prompt,stopWhenorabortSignalto thatstreamTextis a type error, with a runtime throw behind it for JavaScript callers.toolsis yours to pass. The old shape only warned in prose.Also in here
chat.agent()takessystem,registry,cacheControlandsystemProviderOptions, so a managed prompt's model and its cache breakpoint no longer have to be passed at the call site.ChatStreamTextis exported for typing a loop factored out ofrun.The signature is taken from the AI SDK's own declaration:
The peer range spans
aiv5, v6 and v7, whose options differ.typeofresolves to whichever version is installed, so generics and tool inference are the caller's own and a v8 option needs no change here.Actions.
onActionno longer receivesstreamTextortools: an action is a state edit, and one that returnschat.turn()(added in #4816) is followed byrun(), which already has both. The action docs on this branch describe that model.chat.toStreamTextOptions()now also applieschat.agent'ssystem,registry,cacheControlandsystemProviderOptions, so the spread form is equivalent to thestreamTexthanded torun(), as the docs say; previously an agent's system prompt was silently dropped on that path. Those options are published on every boot, including for ahydrateMessagesagent, which skips the snapshot boot block where they were first set.Verification
Typecheck and the full suite pass on both
ai@6.0.116andai@7.0.66. The option merge is a pure function so the merged object can be asserted directly, which is howexperimental_telemetrybeing dropped was caught: moststreamTextoptions never reach the provider, so a test that observes the model cannot see them.Run end to end against a deployed agent with every
runrewritten to the new form and no spread anywhere: steering, undo across a cold boot, and regenerate all still pass, a caller's ownprepareStepruns while managed steering still fires inside the turn, and consecutive injections arrive one per turn. The handover-owned options are pinned by@ts-expect-errorassertions in a typechecked test rather than only by the runtime throw.