[rajivpant]

Retrieval helpers (query planning, expansion, HyDE, reranking, verification) were not working for local, Anthropic or OpenAI chats, and reranking ordered results by raw scores of mixed scale. This PR fixes both and bumps the version to 4.0.1.

Helper calls

_call_fast_llm called LiteLLM directly. The cases below were observed live, or reproduced offline against LiteLLM 1.109.0rc1:

• Local models: A local Gemma 4 12B thinks by default. It spent the fixed 500-token budget reasoning and returned an empty reply (finish_reason=length), so reranking never ran.

• Anthropic and OpenAI: LiteLLM rejected the fixed temperature=0.3:

  • for Claude Haiku 5.5, "does not support temperature=0.3";
  • for GPT-6 Luna, "doesn't support temperature=0.3 while reasoning is active".

  Every helper silently fell back.

• API keys: Keys were looked up under the environment-variable name (ANTHROPIC_API_KEY), but the keystore is keyed by provider (anthropic).

The helper now builds an LLMRequest for the configured backend. That request carries:

• the provider's key from the keystore;
• the model's default_max_tokens;
• the least reasoning the model allows, from the new least_reasoning_effort in model_parameters: none for local hybrid models (Ollama think: false), and the lowest declared mode for cloud models.

Offline validation against LiteLLM: Haiku, Luna, Flash-Lite and Gemma 4 12B all pass parameter validation.

Thinking

• off for local models: An explicit thinking_effort: off now sends reasoning_effort: none. Before, it sent nothing, which kept models that think by default thinking.
• Gemma 4 catalog entries: All four now declare hybrid thinking. Ollama 0.32.15 reports the capability for each. 12B, 26B and 31B think when no setting is sent; E4B does not.
• LiteLLM debug output: LiteLLMBackend sets suppress_debug_info. LiteLLM printed a "Provider List" banner on lookups it catches itself.

Ranking

• Rerank mix: It used 0.3 × raw score + 0.7 × LLM/10, but raw scores run from about 0.03 (RRF) to over 8 (full-text rank). In a live chat the reranker scored the one relevant chunk 10 and the rest 0, yet the relevant chunk finished eighth. Retrieval strength is now scaled to 0–1 by dividing by the largest score.
• Results past the top 20: They used to re-sort above reranked results on raw scores. They now follow in retrieval order.
• Skipped rerank: When reranking is skipped, the retrieval order stands.
• Caller sort: The caller no longer re-sorts by raw score.
• Context labels: The context labels each chunk with the combined 0–1 score.
• Hybrid search: It fuses with RRF whenever either list has hits, including after a keyword-search failure. Every result therefore carries an rrf_score.
• HyDE boost: It now applies to the sort key.

Verification

• Tests: 1,176 passed, 14 skipped. With main's rag.py and core.py, these fail: the five request tests in tests/test_fast_llm.py, the three thinking-off tests in tests/test_thinking.py, and 6 of the 7 tests in tests/test_rerank_ranking.py.
• Live local stack, gemma4:26b with the gemma4:12b helper:
  • Planner, expansion and HyDE all ran, and reranking scored the relevant chunk 10.
  • The answer was grounded in that chunk. The same question on main answered "no record".
  • The chat took 34 s, against 86 s on main.
• Live thinking_effort: off at a 256-token budget: It replies in 0.65 s. The default setting still raises the clear output-budget error.
• Cloud: Not exercised live; the cloud accounts are out of credit.

🤖 Generated with Claude Code