feat: add neutral native provider IR - #2
Merged
Merged
Conversation
) * docs(readme): make Relay and libsy quickstarts runnable Signed-off-by: Itay Zitvar <izitvar@nvidia.com> * docs(relay): configure deployment before enabling plugin and add macOS packaging path Signed-off-by: Itay Zitvar <izitvar@nvidia.com> * docs(relay): require integrity_only policy override for the unsigned plugin bundle Signed-off-by: Itay Zitvar <izitvar@nvidia.com> --------- Signed-off-by: Itay Zitvar <izitvar@nvidia.com>
…VIDIA-NeMo#625) Signed-off-by: zengyuanl <zengyuanl@nvidia.com>
* feat(libsy): add optional decision metadata Signed-off-by: nachiketb <nachiketb@nvidia.com> * feat(libsy): assign routing decision IDs Signed-off-by: nachiketb <nachiketb@nvidia.com> * refactor(libsy): rename decision metadata Signed-off-by: nachiketb <nachiketb@nvidia.com> * test: remove dumb test Signed-off-by: nachiketb <nachiketb@nvidia.com> --------- Signed-off-by: nachiketb <nachiketb@nvidia.com>
Adds system_prompt to native Switchyard targets so the prompt follows the model that actually answers the request. Signed-off-by: Alex Fournier <afournier@nvidia.com>
…-NeMo#645) * fix(translation): replay responses reasoning as input history Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com> * fix(client): flatten responses namespace tools upstream Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com> * fix(translation): omit empty responses reasoning Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com> --------- Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
Signed-off-by: Alex Fournier <afournier@nvidia.com>
Signed-off-by: Graham King <grahamk@nvidia.com>
Signed-off-by: Alex Fournier <afournier@nvidia.com>
…ransforming routes (NVIDIA-NeMo#646) * fix(translation): carry encrypted reasoning through the Responses codec Signed-off-by: Lin Jia <linj@nvidia.com> * fix(translation): decode text-only Responses reasoning items on output_item.done Signed-off-by: Lin Jia <linj@nvidia.com> * fix(translation): decode Responses reasoning from added, done, text.done, and completed carriers Signed-off-by: Lin Jia <linj@nvidia.com> * feat(translation): opt-in trace of raw Responses stream events Signed-off-by: Lin Jia <linj@nvidia.com> * fix(translation): emit Responses reasoning in the standard summary_text shape Signed-off-by: Lin Jia <linj@nvidia.com> * fix(translation): make synthesized Responses item ids unique across responses Signed-off-by: Lin Jia <linj@nvidia.com> * fix(translation): keep synthesized Responses item ids within 64 characters Embedding the upstream response id made synthesized item ids unique across turns, but some upstreams issue response ids several hundred characters long, and OpenAI rejects replayed item ids over 64 characters. A session that started on such an upstream and later moved to an OpenAI model failed every request with a 400 on the replayed history. Long response ids are now replaced by a 64-bit FNV-1a digest, which keeps ids distinct per response while bounding their length. Signed-off-by: Lin Jia <linj@nvidia.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(translation): replay encrypted reasoning under the item id it was issued with Encrypted reasoning returned by OpenAI-compatible providers is bound to the output item id it was issued under. The buffered path re-emitted such items with a synthesized id while keeping the payload, so a client that replayed the conversation got a 400 (invalid_encrypted_content: item_id did not match the target item id) on its next request and the session died. This affected any escalation route whose efficient tier returns encrypted reasoning. Both decoders now record the provider item id on the reasoning.encrypted detail, and both encoders reuse that id for the emitted reasoning item. If the id only becomes known after the item has already opened under another id, the payload is dropped with a warning instead of poisoning the replay. Signed-off-by: Lin Jia <linj@nvidia.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(translation): accept a verbatim reasoning item as an encrypted reasoning detail The buffered request decoder in NVIDIA-NeMo#645 keeps a provider's reasoning item whole as the reasoning detail when it carries encrypted_content. The stream and buffered response encoders here read the payload and item id through the shared helpers, so those helpers now recognise that shape alongside the documented reasoning.encrypted object. This keeps the buffered-decode, re-stream path (used by any route that buffers a reply) carrying the encrypted payload under its original id regardless of which PR lands first. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Lin Jia <linj@nvidia.com> * fix(translation): keep every reasoning item of a streamed Responses response GPT-5 models emit a reasoning item ahead of each tool call, so one response can carry several reasoning items. The Responses stream encoder kept a single reasoning slot: the second item never opened, and when its encrypted payload arrived under a different provider id the encoder dropped it with a warning (observed on 1.5 percent of GPT-5.6 turns behind Switchyard). The same slot also opened under a synthesized id whenever summary text streamed before the payload, which made the payload unverifiable and dropped it too. The encoder now tracks reasoning items per source content index and emits each as its own output item, closing them in provider order. The Responses stream decoder announces a reasoning item's provider id as soon as the added event names it, so the encoder opens the item under that id before any text or payload arrives. The response accumulator folds the announcement into the payload detail that follows it and drops announcements whose payload never came, so replayed history keeps one detail per item; the chat encoder skips announcements since a chat client cannot use them. Tests cover a two-reasoning-item response, summary text arriving before the payload, and the decoder announcing an id exactly once across added, done and completed events. Signed-off-by: Lin Jia <linj@nvidia.com> * chore(translation): changelog entries and review tidy-ups for the Responses codec fixes Adds the changelog entries for the Responses reasoning, item-id, and raw trace changes, removes a comment left behind when a helper moved to the shared codec module, and lists the new tracing dependency with the rest of the crate's dependencies. Signed-off-by: Lin Jia <linj@nvidia.com> --------- Signed-off-by: Lin Jia <linj@nvidia.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* feat(libsy): populate bounded outcome evidence Signed-off-by: nachiketb <nachiketb@nvidia.com> * docs(libsy): clarify evidence helper contracts Signed-off-by: nachiketb <nachiketb@nvidia.com> * fix(libsy): preserve deciding outcome evidence Signed-off-by: nachiketb <nachiketb@nvidia.com> --------- Signed-off-by: nachiketb <nachiketb@nvidia.com>
…DIA-NeMo#658) * feat(libsy): expose outcome metadata in OpenTelemetry and Python Signed-off-by: nachiketb <nachiketb@nvidia.com> * fix(libsy): preserve span identity and verify Python evidence Signed-off-by: nachiketb <nachiketb@nvidia.com> --------- Signed-off-by: nachiketb <nachiketb@nvidia.com>
* docs: add OpenTelemetry reference Signed-off-by: nachiketb <nachiketb@nvidia.com> * docs: focus OpenTelemetry reference on libraries Signed-off-by: nachiketb <nachiketb@nvidia.com> * docs: simplify OpenTelemetry reference Signed-off-by: nachiketb <nachiketb@nvidia.com> --------- Signed-off-by: nachiketb <nachiketb@nvidia.com>
* feat(routing): add auto algorithm type Signed-off-by: Ryan Lempka <rlempka@nvidia.com> * fix(routing): report correct route type in auto route errors Signed-off-by: Ryan Lempka <rlempka@nvidia.com> * docs(routing): describe auto routing without naming its current default Signed-off-by: Ryan Lempka <rlempka@nvidia.com> * feat(routing): give auto its own preset knobs instead of stage_router's fields Signed-off-by: Ryan Lempka <rlempka@nvidia.com> * docs(getting-started): use auto for the quick start route Signed-off-by: Ryan Lempka <rlempka@nvidia.com> --------- Signed-off-by: Ryan Lempka <rlempka@nvidia.com>
* refactor(stage-router): generalize tool activity categories Signed-off-by: Sean Lopp <slopp@nvidia.com> * feat(stage-router): configure custom tool semantics Signed-off-by: Sean Lopp <slopp@nvidia.com> * docs(stage-router): document custom tool semantics Signed-off-by: Sean Lopp <slopp@nvidia.com> * fix(stage-router): address tool semantics review Signed-off-by: Sean Lopp <slopp@nvidia.com> * fix(stage-router): preserve built-in tool semantics Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com> * test(stage-router): cover custom semantics scoring --------- Signed-off-by: Sean Lopp <slopp@nvidia.com> Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com> Co-authored-by: Sabhatina Selvam <sabhatinas@nvidia.com>
…o#649) Signed-off-by: Alex Fournier <afournier@nvidia.com>
…NVIDIA-NeMo#664) * fix(translation): carry the paired call name on Responses tool outputs Signed-off-by: Zengyuan Liu <zengyuanl@nvidia.com> * fix(translation): pair parallel tool calls with their outputs in Responses input Signed-off-by: Zengyuan Liu <zengyuanl@nvidia.com> * fix(translation): strip Codex compaction markers before Responses decode Signed-off-by: Zengyuan Liu <zengyuanl@nvidia.com> --------- Signed-off-by: Zengyuan Liu <zengyuanl@nvidia.com>
…e request shape (NVIDIA-NeMo#648) * feat(translation): round-trip Responses freeform custom tools Codex drives GPT-5 models with freeform tools: the definition is {"type": "custom", ...} and the model answers with custom_tool_call items whose input is a raw string. The Responses codec only modelled function tools, so a Codex session against a GPT-5 model through Switchyard lost its tool definitions and its tool calls and ended after one turn. Custom tools now pass through the IR as a function with a single input argument, with the verbatim definitions kept on the request extensions. History items custom_tool_call and custom_tool_call_output decode and re-encode with their types intact, upstream custom_tool_call output items decode on both the buffered and stream paths, and when a response is encoded with the request's extensions, calls to a custom tool are rewritten back into custom_tool_call items. Argument delta events for such calls are dropped on the stream because a partial JSON delta has no freeform equivalent; clients read the completed item. Signed-off-by: Lin Jia <linj@nvidia.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(translation): understand Responses-lite additional_tools input items Codex sends GPT-5 requests in a lite shape: no top-level tools, empty instructions, the tool definitions inside input[0] as an additional_tools developer item, and the base instructions as a developer message. The Responses codec did not know the item, so a routed GPT-5 session had no tools in the IR and the item was turned into a user message carrying the tool JSON. The request decoder now reads the item's tools as the request's tool definitions (including freeform tools) and keeps the array verbatim on the request extensions; the input decoder skips the item; the request encoder re-emits it in place and leaves top-level tools absent, so a Responses upstream receives the request in the shape the client used, while a chat upstream receives ordinary function tools. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Lin Jia <linj@nvidia.com> * fix(translation): give rewritten custom tool calls a ctc item id prefix OpenAI validates replayed item ids by prefix and rejects a custom_tool_call whose id starts with fc_ ("Expected an ID that begins with 'ctc'"). When a function_call item is rewritten into a custom_tool_call for the client, its synthesized id now takes the ctc_ prefix. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Lin Jia <linj@nvidia.com> * docs(translation): explain the custom-tool bookkeeping in the Responses request codec Signed-off-by: Lin Jia <linj@nvidia.com> * fix(translation): keep the additional_tools item when a lite request encodes input as a string A request that reduces to one user text encodes its Responses input as a plain string. The re-emitted additional_tools item was inserted only into an array input, so a Responses-lite request with a single user message lost its tool definitions on the way upstream. The encoder now widens the string to the equivalent one-message array before inserting the item. Found by review on NVIDIA-NeMo#648; covered by a request round-trip test. Signed-off-by: Lin Jia <linj@nvidia.com> * fix(translation): pair freeform tool calls with their outputs NVIDIA-NeMo#664 reorders replayed history so each function_call_output follows its call. Freeform (custom) calls and their custom_tool_call_output items are the same shape for an upstream that pairs by adjacency, so the pass now treats them alike. A tool result answering a freeform call keeps the custom_tool_call_output type from this branch and the paired call name from Signed-off-by: Lin Jia <linj@nvidia.com> NVIDIA-NeMo#664. Covered by a parallel custom-call round-trip test. --------- Signed-off-by: Lin Jia <linj@nvidia.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
…dge and log verdicts (NVIDIA-NeMo#639) * Log escalation judge verdicts at debug The escalation judge already returns a reason with every verdict, but the router discarded it after reading the boolean, so an operator tuning the judge prompt could not see why sessions were held on the efficient tier. Keep the reason on the verdict and emit one debug event per verdict under the util::escalation target; it is silent unless that target is enabled. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit 5e6cf24ed4a86b223c2aba4940f43c655049a58e) Signed-off-by: Lin Jia <linj@nvidia.com> * Anchor every pre-reply user message as task framing for the escalation judge The trajectory summary handed to the escalation judge pinned only the first user message as the task statement. Codex sends an environment context block as its first user message and the task as the second, so the judge saw shell and cwd boilerplate as the task while the real task sat in the rolling window, truncated to the per-message cap, and scrolled out after about thirty messages. From then on every task-aware pattern in the rubric (drift, unverified completion, violated constraints) had nothing to compare against, and only friction patterns could fire. Treat every user message that precedes the first assistant reply as task framing and anchor them all, with a wider per-message budget so a multi-thousand-character feature specification survives intact. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit fc61f715ed165e73e0664e4a866ec3b99aa38742) Signed-off-by: Lin Jia <linj@nvidia.com> --------- Signed-off-by: Lin Jia <linj@nvidia.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
… it serves (NVIDIA-NeMo#666) * feat(runner): let a target force the reasoning effort of the requests it serves A target's extra_body only fills keys the caller left out, so it cannot change the reasoning effort of a request from a client that always sets one. Codex, for example, sends reasoning.effort on every Responses request (high by default for gpt-5.6), which meant a route could not run its strong tier at max behind a client configured for high, and a per-target extra_body.reasoning_effort was silently ignored on the Responses wire. Every routed GPT-5.6 run in the DeepSWE program ran at high for that reason. This adds an optional reasoning_effort on [targets.<name>]. When set, the client applies it after the extra_body merge and it replaces the caller's value: reasoning.effort on the Responses wire, reasoning_effort on Chat Completions. Anthropic has no equivalent, so the runner rejects the key on anthropic_messages clients at load time, along with blank values. Tests cover the override on both OpenAI wire formats against a mock upstream, TOML parsing, the blank rejection, and the Anthropic rejection. The TOML schema reference documents the key. Signed-off-by: Lin Jia <linj@nvidia.com> * docs(reference): note that two efforts of one model need two llm clients The runner keeps one target per model id and llm client and drops a duplicate with a warning, so an effort-tier pair for one model must put each tier on its own llm_clients entry pointing at the same endpoint. Say so next to the reasoning_effort key, where the question arises. Signed-off-by: Lin Jia <linj@nvidia.com> * docs(client): explain the ordering and wire shapes of the reasoning effort override Signed-off-by: Lin Jia <linj@nvidia.com> * fix(runner): reject duplicate targets whose request settings conflict The LLM client keeps one backend per model id, so two targets naming the same model on the same llm client share a backend. That was tolerated with a warning, which is fine for an alias with identical settings but silently wrong once targets can carry reasoning_effort or extra_body: the second target's settings never reach the wire. Loading now fails when two such targets differ in reasoning_effort or extra_body, and still only warns for an identical alias. Found by review on NVIDIA-NeMo#666; covered by a runner test. Signed-off-by: Lin Jia <linj@nvidia.com> --------- Signed-off-by: Lin Jia <linj@nvidia.com>
…IA-NeMo#600) Signed-off-by: Alex Fournier <afournier@nvidia.com>
NVIDIA-NeMo#671) * fix(translation): open a reasoning summary part only when text streams The Responses stream encoder opened a summary part on every reasoning item, including encrypted-only items that never stream text, and closed it only when text had accumulated. An encrypted-only item therefore left the client with a reasoning_summary_part.added and no matching done, while the finished item carried an empty summary. The part now opens on the first text delta, so encrypted-only items open, carry their payload, and close with an empty summary and no part events. A stream test covers the encrypted-only case; the existing text case still sees its part open and close. Signed-off-by: Lin Jia <linj@nvidia.com> * fix(translation): do not open a summary part for an empty reasoning delta An empty reasoning delta opened the item's summary part without adding text, and the part then never closed. The delta now only opens the item. Found by review; covered by a stream test. Signed-off-by: Lin Jia <linj@nvidia.com> --------- Signed-off-by: Lin Jia <linj@nvidia.com>
Signed-off-by: Alex Fournier <afournier@nvidia.com>
* docs(relay): document the native integration Signed-off-by: Alex Fournier <afournier@nvidia.com> * docs(relay): explain routing value and cost measurement Signed-off-by: Alex Fournier <afournier@nvidia.com> * docs(relay): simplify integration overview Signed-off-by: Alex Fournier <afournier@nvidia.com> * docs(relay): clarify routing cost accounting Signed-off-by: Alex Fournier <afournier@nvidia.com> * docs(relay): clarify combined cost wording Signed-off-by: Alex Fournier <afournier@nvidia.com> * docs(relay): distinguish ATIF and OpenTelemetry Signed-off-by: Alex Fournier <afournier@nvidia.com> * docs(relay): clarify telemetry boundaries Signed-off-by: Alex Fournier <afournier@nvidia.com> * docs(relay): address integration review Signed-off-by: Alex Fournier <afournier@nvidia.com> * docs(relay): list supported routing algorithms Signed-off-by: Alex Fournier <afournier@nvidia.com> --------- Signed-off-by: Alex Fournier <afournier@nvidia.com>
Signed-off-by: Alex Fournier <afournier@nvidia.com>
…NeMo#678) Signed-off-by: Alex Fournier <afournier@nvidia.com>
Signed-off-by: Alex Fournier <afournier@nvidia.com>
Signed-off-by: Alex Fournier <afournier@nvidia.com>
Signed-off-by: Alex Fournier <afournier@nvidia.com>
Signed-off-by: Alex Fournier <afournier@nvidia.com>
…specific model. (NVIDIA-NeMo#630) Instead of giving the available models to the algorithm in new we pass them alongside the request in run_stream where they go in the Driver. Algorithms work with Category instead of ModelId. Later libsy selects a model from the list for that category. The mapping from Category to ModelId lives in the Driver. See NVIDIA-NeMo#588 for the originating idea. Assisted-by: Codex:GPT 5.6 Sol high Assisted-by: Claude:Opus 5 medium Reviewed-by: Claude:Opus 5 medium Signed-off-by: Graham King grahamk@nvidia.com
Signed-off-by: Ting-Hong Shieh <32212900+ting-hong-shieh@users.noreply.github.com>
c1-squire-dev
Bot
force-pushed
the
paul.querna/codex-neutral-ir
branch
from
September 13, 2026 15:05
64723b8 to
1d7e38a
Compare
Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
ci: adapt workflows for downstream integration
…ield-provenance fix(translation): preserve OpenAI reasoning field provenance
Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com> Signed-off-by: Paul Querna <paul.querna@conductorone.com>
…tus seam The typed routed-call contract landed with three consumers still reading the legacy shape, so a host that classifies its own failures lost behavior the status-shaped path had. - staged escalation: the efficient-tier fall-through matched only `ContextWindowExceeded`, so a host returning `RoutedCall` terminated the run instead of escalating. It now also accepts an advanceable `ContextWindow` or `TargetIncompatible`, and still honors `Stop` as terminal. - exhaustion status: `ProviderTargetsExhausted` collapsed to 503 with its bounded summary unread. It now resolves from the partition in precedence order -- exact availability evidence or a circuit bypass, then host or provider timeouts, then homogeneous rate-limit and request-shape cases, then a plain routing failure -- so the 429, 504, and 422 public codes are reachable and 503 stays truthful. - overflow exhaustion: an exhaustion whose every real failure was an overflow reported target unavailability, which is the conflation this contract exists to remove. It is now reported as a context-window condition at both seams that consume the distinction: the candidate hop's reason, and the runner's terminal telemetry kind. Bypassed circuits neither establish nor contradict the overflow, and an all-bypassed exhaustion stays unavailability. Mixed classes remain unavailability. `ProviderTargetsExhaustedSummary` grows `real_failures` and `is_context_window_exhaustion` so the aggregate rules are stated once, and rejects an over-long class-entry list before it sizes an allocation. Behavioral tests cover each seam, including the five aggregate-precedence combinations. No existing public type, constructor signature, or client requirement changes. Staged on the D-TB-103 immutable review fork ahead of upstream submission. Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com> Signed-off-by: Paul Querna <paul.querna@conductorone.com>
…codec feat: Bedrock Converse codec and typed routed-call failure contract
Signed-off-by: Paul Querna <paul.querna@conductorone.com>
Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
jerenkrantz
force-pushed
the
paul.querna/codex-neutral-ir
branch
from
September 14, 2026 17:59
85c3430 to
68770e1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification
cargo fmt --all --checkcargo clippy --workspace --all-targets -- -D warningscargo check --workspacecargo test -p switchyard-protocol -p switchyard-translation -p switchyard-llm-client(256 passed)cargo test --workspacereaches the final link step but cannot link the Nix Python 3.13 library in this environment because it requires GLIBC 2.38 symbols; the repository CI matrix provides its own compatible Python runner.