fix: extract LangChain content-block text and apply model parameters after eval - #80
Conversation
…after eval LangChain AIMessage.content can be a list of thinking/text blocks; keep the visible text for callers. Accept llm as a factory so ChatAnthropic/ChatOpenAI can be built with flag model.parameters after evaluation. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 7beaa6f. Configure here.
| model_name=str(model.get("name") or "claude-sonnet-4-5"), | ||
| thinking={"type": "enabled", "budget_tokens": _THINKING_BUDGET_TOKENS}, | ||
| max_tokens_to_sample=_MAX_TOKENS, | ||
| ) |
There was a problem hiding this comment.
Example factory kwargs collide with parameters
Medium Severity
The thinking example spreads flag model.parameters into ChatAnthropic and then passes overlapping keywords (timeout, stop, model_name, thinking, max_tokens_to_sample). A second value for any of those keys is a TypeError, and max_tokens aliases max_tokens_to_sample, so a typical Claude flag never constructs the model.
Reviewed by Cursor Bugbot for commit 7beaa6f. Configure here.
|
|
||
|
|
||
| def _make_ai_msg( | ||
| content: str = "answer", input_tokens: int = 10, output_tokens: int = 5 |
There was a problem hiding this comment.
just curious - why the change from string -> any?
There was a problem hiding this comment.
This PR is specifically targeting langchain's "content blocks" pattern it uses for some providers. So no longer a string. Any type we provided here given the breadth of what it can return would be incorrect for someone depending on provider, etc.


Summary
AIMessage.contentwhen it is a list of thinking/text blocks so callers no longer get an emptyoutputwhile tokens are still billed.llmas a factory(config) => model(in addition to a pre-built instance) and spread flagmodel.parametersunchanged into defaultChatAnthropic/ChatOpenAIconstructors after evaluation.langchain-thinkingexample (tool-free prompt) plus unit tests for mixed content blocks, factory vs instance, and default constructor kwargs.Test plan
packages/langchain-messagesandpackages/langchain-agentsunit testspython main.py langchain-thinking launch-darkly-documentation-summarizer-messages-claude "Reason it out yourself without any tools: what is 17 times 23?"— non-emptyresponsedoesnt-exist— exit 1Made with Cursor