fix: preserve generated chat parser during response parsing

[oglego]

Summary

While going through the examples/context-engineering/README.md example, I noticed tool calling wasn't working as documented:

<tool_call>
{"name": "calculator", "arguments": {"a": 3, "b": 4, "operation": "add"}}
</tool_call>
>

I dug into this and found that there was a complete refactoring of the parser architecture in llama.cpp (PR #18675), which impacts tool calling here - see deps/llama.cpp/docs/autoparser.md for background.

The PEG parser needs to be loaded via params.parser; deps/llama.cpp/tests/test-chat.cpp has examples of this. generation_prompt also needs to be passed through - without it, I think llama.cpp parses the raw response alone instead of generation_prompt + response, which is what the derived grammar actually expects.

Change

The only update needed to resolve this:

syntax.parser.load(params.parser);
syntax.generation_prompt = params.generation_prompt;

Testing

Verified end-to-end with a multi-turn conversation against granite-4.0-micro-Q8_0.gguf - tool calls parse and execute correctly:

> Calculate 3 + 4 
<tool_call>
{"name": "calculator", "arguments": {"a": 3, "b": 4, "operation": "add"}}
</tool_call>
[TOOL EXECUTION] Calling calculator
[TOOL RESULT]
{"result":7.0}
[CONTEXT] Trimmed messages:
[CONTEXT] - [tool]: {"result":49.0}
[CONTEXT] - [assistant]: tool_calls: [calculator({"a": 7, "b": 7, "operation": "multi...
The result of 3 + 4 is 7.
> Now multiple that by 7
<tool_call>
{"name": "calculator", "arguments": {"a": 7.0, "b": 7, "operation": "multiply"}}
</tool_call>
[TOOL EXECUTION] Calling calculator
[TOOL RESULT]
{"result":49.0}
[CONTEXT] Trimmed messages:
[CONTEXT] - [tool]: {"result":7.0}
[CONTEXT] - [assistant]: tool_calls: [calculator({"a": 3, "b": 4, "operation": "add"}...
49
>

Let me know if I am missing something on this, and thanks in advance for the review!

[daavoo]

thanks for the catch @oglego 🙏

[oglego]

Happy to help! Thanks for all of the enhancements to it!