ReActV2: submit only for signatures that need it

.dialyzer_ignore.exs

@@ -57,9 +57,9 @@
   # defensive clause/guard: ReqLLM.Response types `usage` as map() on the
   # struct, but its schema defaults the field to nil and Response.usage/1 is
   # `map() | nil`, so the nil clause is reachable at runtime.
-  {"lib/imp/clients/req_llm.ex", :guard_fail, 1414},
-  {"lib/imp/clients/req_llm.ex", :pattern_match_cov, {1442, 8}},
-  {"lib/imp/clients/req_llm.ex", :pattern_match_cov, {147, 7}},
+  {"lib/imp/clients/req_llm.ex", :guard_fail, 1448},
+  {"lib/imp/clients/req_llm.ex", :pattern_match_cov, {1476, 8}},
+  {"lib/imp/clients/req_llm.ex", :pattern_match_cov, {150, 7}},
   # defensive error clause on an always-ok internal call
   {"lib/imp/clients/training.ex", :pattern_match, {1215, 13}},
   # defensive error clause on an always-ok internal call
@@ -219,7 +219,7 @@
   # Defensive fallbacks and MapSet opacity retained at the 0.3 cut. These are
   # individually pinned so a changed success type makes the gate ask again.
   {"bench/imp/benchmark_truth/multimodal_runner.ex", :pattern_match_cov, {340, 16}},
-  {"lib/imp/adapter/chat.ex", :pattern_match_cov, {774, 8}},
+  {"lib/imp/adapter/chat.ex", :pattern_match_cov, {787, 8}},
   {"lib/imp/adapter/xml.ex", :pattern_match_cov, {675, 8}},
   {"lib/imp/mcp.ex", :pattern_match_cov, {372, 8}},
   {"lib/imp/optimizer/artifact.ex", :call_without_opaque, {745, 52}},

CHANGELOG.md

@@ -4,33 +4,42 @@ User-visible changes to Imp are recorded here.
 
 ## Unreleased
 
-- A `ReActV2` turn now ends when the model stops calling tools. A step that
-  comes back as prose with no tool call, for a task signature with exactly one
-  output of type `:string`, finishes the run with that prose as the output and
-  `termination_reason: :answered`, in that one request. It used to cost one more
-  request with `tool_choice` naming `submit`, which both spent a call and, when
-  the model had already acted with a tool, came back with a summary of what it
-  did rather than what it said. Every other mainstream loop — Anthropic's tool
-  runner, the OpenAI Agents SDK, LangGraph's ReAct, Pydantic AI — ends the turn
-  this way, so it is the default. A signature with several outputs, or one
-  non-text output, still takes the forced submit, because prose cannot fill
-  those fields, and so does a step that says nothing at all. The new option
-  `prose: :forced_submit` keeps the old behaviour for a single-output
-  signature.
-- `ReActV2` gains `on_max_iters`, what the step limit does. The default,
-  `:forced_submit`, is what it did before: one more request with `tool_choice`
-  naming `submit`. `:last_prose` makes one more request with no tools in it at
-  all, so the only thing the model can do is speak, and that prose is the
-  single text output, with `termination_reason: :last_prose`. This is the
-  ending that fits a host whose model already finishes turns by writing prose:
-  it is never asked to call a tool it did not choose. A completion that says
-  nothing finishes with an empty answer rather than an error. `:last_prose`
-  needs a signature with exactly one output of type `:string`, and is refused
-  at construction otherwise. The companion option `last_prose_note`, a string,
-  puts one line of host text in front of that request as a user message and
-  keeps it in the returned history; Imp writes no sentence of its own. Both
-  options persist through `dump`/`load`, and a dump written before them loads
-  as `:forced_submit`.
+- `Imp.Clients.ReqLLM` returns a response whose body carries a provider error
+  as `{:error, %ReqLLM.Error.API.Request{}}`. OpenRouter relays an upstream
+  provider's refusal as a successful HTTP response with an error object and no
+  choices, which ReqLLM decodes to an empty message; read as a completion, a
+  refused request was a model that said nothing.
+- `ReActV2` offers `submit` only to a signature that needs it. A task
+  signature with exactly one output of type `:string` gets no `submit` tool:
+  a step that comes back as prose with no tool call is the answer, in that
+  one request, with `termination_reason: :answered`, which is how Anthropic's
+  tool runner, the OpenAI Agents SDK, LangGraph's ReAct and Pydantic AI end a
+  turn. Its history event carries the output, as a `submit`'s does, and the
+  answer is not also emitted as a `:reasoning` event. A signature with several
+  outputs, or one non-text output, keeps DSPy's `submit` unchanged.
+- A step of a one-text-output signature that calls nothing and says nothing
+  is an empty answer: the turn ends there with `termination_reason:
+  :answered` and no further request, because saying nothing is how a model
+  declines to answer.
+- An interrupted turn of a one-text-output signature (the step limit, a
+  failed request, prose the output does not accept) makes one more
+  request with the same tools as every step and `tool_choice: "none"`, so the
+  model can only write text, and that text is the answer, with `termination_reason: :last_prose`
+  and `termination_cause` naming the interruption (`:max_iters`,
+  `:prediction_error`, `:parse_error`, `:invalid_answer`). A completion that says nothing is an empty answer rather
+  than an error. A tool call the model makes on that request anyway is not
+  run; the text is the answer and the calls are listed in
+  `unexecuted_tool_calls`. `last_prose_note`, a string, puts one line of host text in
+  front of that request as a user message and keeps it in the returned
+  history; Imp writes no sentence of its own. If the process's `Imp.Deadline`
+  has already passed, no request is made and the run ends with
+  `termination_reason: :deadline_exceeded`. `forced_submit_notice` is for
+  signatures with `submit` and `last_prose_note` for those without; each is
+  refused at construction for the other. There is no `prose` or
+  `on_max_iters` option, and a dump no longer carries them.
+- `Imp.Observability` reports a prediction that ended `:answered`,
+  `:last_prose` or `:finished_by_tool` as complete; it reported them as
+  incomplete.
 - `ReActV2` gains `finish_on`, a map from tool name to
   `fn arguments, result, inputs -> {:finish, outputs} | :continue end`. A tool
   named there ends the turn with the outputs the function returns, which are

docs/IMP_FOR_DSPY_USERS.md

@@ -17,7 +17,7 @@ updates, telemetry, and fresh-runtime artifact application.
 | `dspy.Signature` / `"q -> a"` | `Imp.signature("q -> a")` — the same compact input/output idea with Imp type spellings such as `array[...]`, `enum[...]`, and `number` |
 | `dspy.Predict(sig)` | `Imp.predict(sig, lm: lm)` |
 | `dspy.ChainOfThought` | `Imp.chain_of_thought/2` |
-| `dspy.ReAct(sig, tools=[...])` | `Imp.react_v2(sig, tools, tool_policy: [...])` — typed tools, structured observations, and validated `submit` |
+| `dspy.ReAct(sig, tools=[...])` | `Imp.react_v2(sig, tools, tool_policy: [...])` — typed tools, structured observations, and validated `submit` for several or typed outputs; one text output is answered in prose |
 | `dspy.Example` / `.with_inputs` | `Imp.example/1` / `Imp.with_inputs/2` |
 | `dspy.Prediction` | `%Imp.Prediction{}` — read fields with `Imp.get/2` |
 | `dspy.Evaluate` | `Imp.evaluate/4` — returns score plus per-example rows |

docs/differentials/REACT_V2_FIDELITY.md

@@ -41,9 +41,8 @@ existing fail-fast `Imp.Predict.ReAct`.
 | History is structured rather than one growing trajectory string | `Imp.History` stores per-turn inputs, thought, typed calls, call results, and final fields | parallel, failure, serialization, and adapter replay tests |
 | Parallel tool calls preserve IDs and execute all calls | Every missing ID receives `call_<turn>_<index>`; results retain the corresponding ID | parallel call test |
 | Unknown tools and execution failures become observations | ReActV2 records error results and continues; existing ReAct remains fail-fast | recovery test |
-| `submit` is reserved and validates final outputs | Constructor rejects user `submit`; the generated submit tool uses the task JSON schema | reserved-submit and missing-output tests |
-| Empty calls, parse failure, context exhaustion, or budget exhaustion force one submit call | Parse failure, context exhaustion and budget exhaustion still do: the final predictor call pins provider `tool_choice` to `submit` and clears `reasoning_effort`, matching the pinned call configuration. A step of prose with no tool call does not, when the task declares exactly one text output: that prose is the output and the turn is over, as it is in Anthropic's tool runner, the OpenAI Agents SDK, LangGraph's ReAct and Pydantic AI. `prose: :forced_submit` restores the upstream shape | forced-submit test, prose-answer test |
-| No upstream equivalent | `on_max_iters: :last_prose` ends a turn that reaches the step limit with one request that carries no tools, so the model can only speak; that prose is the single text output, with `termination_reason: :last_prose`, and an empty completion is an empty answer. `:last_prose_note` puts one line of host text in front of that request. The default stays the forced submit | `test/react_v2_last_prose_test.exs` |
+| `submit` is reserved and validates final outputs | The name is reserved for every signature. A signature with several outputs, or one output that is not text, gets the upstream `submit` tool, built from the task JSON schema, with upstream's description and guidance text. A signature with exactly one output of type `:string` gets no `submit`: its answer is the prose the model writes when it stops calling tools (`termination_reason: :answered`), as in Anthropic's tool runner, the OpenAI Agents SDK, LangGraph's ReAct and Pydantic AI, and its guidance says to answer in plain text. Upstream needs `submit` because a signature can have several typed outputs; one text output does not | reserved-submit, missing-output and no-submit tests, `test/react_v2_request_shape_test.exs` |
+| Empty calls, parse failure, context exhaustion, or budget exhaustion force one submit call | With `submit`, parse failure, empty calls and budget exhaustion still do: the final predictor call pins provider `tool_choice` to `submit` and clears `reasoning_effort`, matching the pinned call configuration. With one text output there is no `submit` to force, so every interruption makes one request with the same tools and `tool_choice: "none"` (encoded `"none"` for OpenAI and OpenRouter and `{"type": "none"}` for Anthropic; a call made anyway is not run and is named in `unexecuted_tool_calls`), and its text is the answer (`termination_reason: :last_prose`, `termination_cause` naming the interruption); `:last_prose_note` puts one line of host text in front of it, and a passed `Imp.Deadline` ends the run with `:deadline_exceeded` instead of making the request. Context exhaustion ends the run at once in both cases, since a further request would be refused the same way | forced-submit tests, `test/react_v2_last_prose_test.exs`, `test/react_v2_last_request_wire_test.exs` |
 | No upstream equivalent | `finish_on` names tools that end the turn with the outputs they carry, the shape Pydantic AI calls an output tool | `finish_on` tests |
 | Prior calls replay as native assistant/tool messages | Chat adapter emits assistant `tool_calls` and matching tool-result messages by call ID | native history adapter test and ReqLLM tests |
 

examples/deployment/agent_optimization.exs

@@ -255,7 +255,7 @@ defmodule ImpDeployment.AgentOptimization.Runner do
 
     actions =
       events
-      |> Enum.filter(&(&1.kind == :tool_call and to_string(&1.tool_name) != "submit"))
+      |> Enum.filter(&(&1.kind == :tool_call))
       |> Enum.map(&to_string(&1.tool_name))
 
     errors = Enum.filter(events, &(&1.kind == :run_failed or not is_nil(&1.error)))
@@ -285,7 +285,7 @@ defmodule ImpDeployment.AgentOptimization.Runner do
 
     grounded? = expected_result? and String.contains?(answer, row.account_id)
     clean? = errors == []
-    valid_final? = completed and answer != "" and termination_reason == :submit
+    valid_final? = completed and answer != "" and termination_reason == :answered
 
     score =
       if expected? do
@@ -307,7 +307,7 @@ defmodule ImpDeployment.AgentOptimization.Runner do
         if(expected_result?, do: nil, else: "expected action did not return its sandbox result"),
         if(valid_final? and clean? and grounded?,
           do: nil,
-          else: "run did not submit a clean answer grounded in account #{row.account_id}"
+          else: "run did not answer cleanly, grounded in account #{row.account_id}"
         )
       ]
       |> Enum.reject(&is_nil/1)

examples/workspace_agent/lib/workspace_agent.ex

@@ -202,8 +202,8 @@ defmodule WorkspaceAgent do
           "observed files and cite relative file paths. You may create files, replace exact " <>
           "text, and run argument-vector commands when the task requires it; those effects " <>
           "require explicit client approval. Start with the README, inspect only what is needed, " <>
-          "make the smallest coherent change, run the relevant check, then synthesize and call " <>
-          "submit. Use the provider's named function calls rather than serializing a tool call " <>
+          "make the smallest coherent change, run the relevant check, then write the answer " <>
+          "as plain text without calling a tool. Use the provider's named function calls rather than serializing a tool call " <>
           "as response text, and pass a JSON object matching the selected tool schema. For " <>
           "commands, pass the executable once, for example run_command with " <>
           "{\"command\":\"cat\",\"args\":[\"README.md\"]}; never repeat the executable inside " <>
@@ -303,9 +303,7 @@ defmodule WorkspaceAgent do
                   :none -> "No workspace content was observed."
                 end
 
-              tool_turn("Return only the grounded smoke result.", "submit", "smoke-submit", %{
-                answer: answer
-              })
+              answer
           end
         end
       )

examples/workspace_agent/test/workspace_agent_test.exs

@@ -14,7 +14,7 @@ defmodule WorkspaceAgentTest do
     %{root: root}
   end
 
-  test "provider-free ReAct factory reads and submits workspace evidence", %{root: root} do
+  test "provider-free ReAct factory reads and answers with workspace evidence", %{root: root} do
     assert {:ok, program, %{cleanup: cleanup}} =
              WorkspaceAgent.program(%{cwd: root}, provider: :static, program: :react)