[rajivpant]

Two silent failures, found while verifying the 4.0.0 stack on 2026-11-06.

1. An exhausted output budget came back as an empty reply

Reasoning tokens count against max_tokens. A local Gemma 4 model with a 300-token budget, the size the settings panel offers, spent the whole budget reasoning. LiteLLM reported finish_reason: length with empty content and 1,375 characters of reasoning. Ragbot returned "" with no error, so it looked like a model with nothing to say.

synthesis_engine.llm.base.require_reply() now raises OutputBudgetExhaustedError when a completion has no text and stopped at its output limit. The error names the model and the budget, and says to raise max_tokens or lower thinking_effort. It covers OpenAI and LiteLLM length, Anthropic max_tokens and google-genai FinishReason.MAX_TOKENS. Every path calls it: LiteLLM complete and stream, and direct-backend complete and stream for Anthropic, OpenAI and Google. A partial reply cut at the limit is still returned; only an empty one raises.

2. chat_stream dropped failures from its streaming thread

chat_stream bridges the backend's callback stream to a generator through a worker thread. That thread's finally posted the end-of-stream sentinel, but its exception went nowhere. A streamed chat that failed, for any reason, simply ended. The thread now hands its exception back, and the generator re-raises it as ChatError after delivering any text that arrived first. The API already turns that into the SSE error event, and the web UI shows it in the reply bubble.

Verification

• tests/test_output_budget.py, 21 tests:
  • require_reply across finish-reason shapes, including partial replies and normal stops;
  • LiteLLM complete and stream;
  • direct-backend OpenAI and Google complete and stream;
  • chat_stream propagating a failure after delivering earlier text.
• Negative control: against main's core.py, both chat_stream tests fail with DID NOT RAISE ChatError.
• make test-fast: 1152 passed, 14 skipped.
• Live, against Ollama with gemma4:26b through the API image:
  • chat(max_tokens=300) now raises ChatError: Chat failed: ollama_chat/gemma4:26b used its whole 300-token output budget without writing a reply; reasoning counts against that budget. Raise max_tokens or lower thinking_effort.
  • max_tokens=4096 returns the full answer.

🤖 Generated with Claude Code