Skip to content

fix(parsing): align XML argument coercion with vLLM 0.32 - #52

Open
hallerite wants to merge 10 commits into
mainfrom
fix/qwen35-arg-coercion-vllm-parity
Open

hallerite wants to merge 10 commits into
mainfrom
fix/qwen35-arg-coercion-vllm-parity

Conversation

@hallerite

@hallerite hallerite commented May 18, 2026 •

Copy link
Copy Markdown
Member

Fix XML boolean arguments such as True being marked INVALID_JSON. Coercion follows vLLM v0.32.0 for aliases, nullable/union/enum schemas, numeric conversion, and non-finite JSON. Arguments without schemas remain strings.

Qwen uses shared XML extraction for both renderer and pluggable parsing, preserving whitespace, incomplete-call diagnostics, and token spans. Prime Qwen3 keeps its template separator normalization. The branch includes current main.

Closes #47.

Validation:

  • Full suite: 13,497 passed, 126 skipped.
  • 35,433 cases compared with vLLM's helper and XML argument-converter code; 960 structural cases compared with the implementation before cleanup.
  • Ruff, pre-commit, and type checks pass. Commits are signed.

References: vLLM 0.32 coercion, Qwen grammar.


Note

Medium Risk
Changes tool-call parsing and typed argument coercion used across Qwen/GLM-style renderers; behavior shifts for booleans, unions, and schema-less paths, though coverage is extensive and aligned with vLLM.

Overview
Aligns schema-driven XML tool-argument coercion with vLLM 0.32, so values like True for boolean params parse as OK instead of INVALID_JSON. _coerce_arg_value now honors type aliases, unions/enums, null/int/number/boolean rules, and rejects non-finite JSON; without a schema, arguments stay plain strings (including the pluggable qwen3.5 parser).

Qwen 3.5 XML extraction is centralized in renderers/parsing.py: Qwen35ToolParser delegates to _parse_xml_tool_calls, which uses _parse_xml_function_blocks for more tolerant parameter boundaries, bare <function=…> when <tool_call> is missing, vLLM-style chatcmpl-tool-* ids, preserved content whitespace, and a safe fallback that returns full text if extraction throws. Prime Qwen3 still strips parsed assistant content for template parity.

Adds tests/test_arg_coercion.py and updates GLM unknown-tool expectations so unschema’d args remain strings.

Reviewed by Cursor Bugbot for commit e2cd1f8. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Delegate Qwen35ToolParser.extract XML parsing to shared _parse_xml_tool_calls helper

Aligns XML argument coercion with vLLM 0.32 by replacing the parser's inline XML extraction in parsers.py with the shared _parse_xml_tool_calls helper. The call passes the token offset and an empty parameter schema index, so this path does not supply tool schemas and argument values stay strings without a schema. Malformed and unclosed blocks are handled by the shared parser, and tool-call metadata is assigned through that path.

  • Risk: for tool calls that contain parameters, the shared coercion code references an undefined types variable, so extraction raises NameError instead of returning parsed calls. Check the coercion branch in _parse_xml_tool_calls.

Macroscope summarized e2cd1f8.

Renderers' Qwen3.5/GLM/MiniMax/Laguna XML parsers flagged
``<parameter=x>True</parameter>`` as INVALID_JSON for boolean params
because ``json.loads("True")`` fails. Both SGLang's
``Qwen3CoderDetector`` and vLLM's ``Qwen3CoderToolParser`` accept it via
``param_value.lower() == "true"`` — so Pythonic literals the model
freely emits round-trip cleanly at inference but came back malformed
through renderers.

Replaces the string-or-``json.loads`` dispatch with the full
``_convert_param_value`` ladder shared by both reference parsers:
case-insensitive ``null``, ``int()`` / ``float()`` for numeric families
with int demotion, case-folded bool, ``json.loads`` → ``ast.literal_eval``
for objects/arrays/anyOf, ``ast.literal_eval`` catch-all. INVALID_JSON
remains as the verifier / RL-loss signal for values that had to
degenerate (e.g. ``yes`` for bool, ``abc`` for int).

One deliberate deviation from vLLM/SGLang: declared ``string`` types
preserve ``"null"`` verbatim instead of coercing to Python ``None``, so
the existing ``test_tool_arg_type_preservation`` string-verbatim
contract still holds.

Fixes #47.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirror vLLM's Qwen3CoderToolParser end to end (chosen over qwen3_xml
because qwen3_coder is a strict superset for offline value coercion;
qwen3_xml's edge over qwen3_coder is its streaming state machine,
which does not apply here yet — module comment notes the TODO to
switch when streaming lands). Closes the three remaining structural
deltas:

- Adopt vLLM's three tolerant regex patterns (tool_call / function /
  parameter) so unclosed </parameter>, </function>, and </tool_call>
  tags are recovered instead of silently dropping their content.
- Add the qwen3coder_tool_parser.py:269-271 back-off: when output has
  no <tool_call> markers, scan the whole completion for raw
  <function=...></function> blocks and recover the call. Content text
  is whatever remains after the function blocks are stripped.
- Strip exactly one leading and one trailing newline from parameter
  values (qwen3coder_tool_parser.py:246-250); interior whitespace
  round-trips verbatim — previous .strip() was too aggressive.
- Fix the no-schema branch of _coerce_arg_value to match vLLM
  (qwen3coder_tool_parser.py:128-137): return the raw text verbatim
  after the case-insensitive null short-circuit instead of attempting
  json.loads.

Qwen3.6 needs no separate parser changes — Qwen36Renderer inherits
parse_response from Qwen35Renderer and shares the same parse_qwen35
plumbing, so the vLLM-parity ladder + structural tolerance applies
to both 3.5 (Python repr literals) and 3.6 (JSON literals) in one
shot.

Qwen35ToolParser in parsers.py (DefaultRenderer + tool_parser=
'qwen3.5' path) is updated to share the new _parse_xml_function_blocks
helper so both code paths produce identical output.

Tests: 7 new structural-parity tests pin the regex tolerance, no-tag
back-off, and whitespace stripping against the vLLM reference shape.
Existing test_no_schema_falls_back_to_json_loads is renamed and
inverted to match the new (vLLM-faithful) behavior.
Comment thread renderers/parsing.py
@macroscopeapp

macroscopeapp Bot commented May 20, 2026 •

Copy link
Copy Markdown

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — This changes shared production tool-call parsing behavior across multiple XML formats, including argument coercion, malformed-call recovery, content handling, and recognition of bare function calls. The extensive tests are useful, but the runtime scope is broader than a narrowly isolated coercion fix.

Notes:

  • Macroscope's correctness review did not run, so approvability was decided on eligibility alone.

You can add or adjust custom eligibility rules. Learn more.

…xception path

Close the four remaining behavioral deltas vs vLLM's
Qwen3CoderToolParser non-streaming extract path:

- Content text is now the raw slice (no .strip()). vLLM
  qwen3coder_tool_parser.py:316-319 explicitly leaves the commented-out
  .rstrip() in place; surrounding whitespace before/after <tool_call>
  round-trips verbatim.
- Backoff content (no <tool_call> markers) is the text up to the first
  <function= position only, matching vLLM line 317-319. Removed
  _strip_function_blocks helper that concatenated leftovers between
  function blocks — different from vLLM.
- Each successfully-parsed tool call is assigned an id in vLLM's
  default format: chatcmpl-tool-<16 hex chars> via
  uuid.uuid4().int & MASK_64_BITS, matching vLLM's make_tool_call_id /
  random_uuid pair. Malformed entries (no function name) intentionally
  stay id-less since vLLM would have dropped them entirely.
- Tool-call extraction is wrapped in try/except: on any unhandled
  error, surface tool_calls=[] and content=full_decoded_text — the
  vLLM qwen3coder_tool_parser.py:327-331 catch-all behavior. Reasoning
  extraction stays outside the wrapper since vLLM doesn't do reasoning
  in this parser.

Tests: 4 new cases pin content-no-strip, backoff content slice,
chatcmpl-tool-<hex> id format, and the exception fallback.
Comment thread renderers/parsing.py Outdated
hallerite and others added 2 commits May 20, 2026 13:43
…byte

vLLM's _convert_param_value (qwen3coder_tool_parser.py:125) runs the
case-insensitive "null" short-circuit BEFORE any type check —
including the string family. That means an XML wire value of
<parameter=x>null</parameter> against a {"type": "string"} schema
collapses to Python None, not the string "null".

Previously this repo carried a deliberate deviation to preserve the
string verbatim across the four XML renderers (Qwen3.5/3.6, GLM-5,
MiniMax-M2.5, Laguna-XS.2). The motivation was a tidier round-trip
contract: "string params always round-trip as strings." But the XML
wire format genuinely cannot distinguish the string "null" from a
JSON null, and matching vLLM is more important than matching our
sibling renderers — downstream consumers (verifier, RL-loss) align
with vLLM's interpretation, not ours.

Test updates:
- test_arg_coercion::test_null_is_preserved_verbatim_for_string_type
  is renamed to test_null_short_circuit_applies_even_for_string_type
  and inverted: a string-typed "null" now returns None.
- test_tool_arg_type_preservation::test_string_arg_preserves_type
  branches on _JSON_FORMAT_MODELS (Qwen3 Hermes, Kimi K2 section JSON)
  vs XML renderers for the string-null case only. JSON renderers
  preserve "null" verbatim because the wire bytes quote strings; XML
  renderers null-coerce. Other string cases (bool/int/array/object)
  still round-trip verbatim everywhere.
Folds in #63's bare-string-for-union-schemas fix and refactors
_coerce_arg_value to use vLLM's current shared helpers
(extract_types_from_schema + coerce_to_schema_type from
vllm/tool_parsers/utils.py, landed in vLLM #43025 on 2026-05-19),
replacing this branch's earlier single-type ladder.

Why: #52 was originally written against pre-refactor vLLM and would
have regressed #63 (anyOf schemas routed to "object" → json.loads
on bare strings flagged INVALID_JSON). vLLM's new shape recursively
flattens anyOf/oneOf/allOf into a type set and walks a priority-
ordered ladder (null > integer > number > boolean > object > array
> string) where string is the always-succeeding terminal — which
absorbs #63's case as a side effect.

Behavior changes relative to the pre-merge branch:
- Union schemas (anyOf/oneOf/allOf) recursively flatten — Union[str,X]
  now accepts bare strings without flagging (the #63 win).
- No top-level "null" short-circuit. Null coercion only fires when
  "null" is in the schema's type set (Optional[X] → anyOf [X, null]
  or type: ["X", "null"]). String-typed "null" stays a string.
- "yes" / non-boolean text for bool-typed param returns raw text +
  INVALID flag (was: False + INVALID).
- Number branch demotes whole floats to int regardless of source
  shape (1e3 → 1, 1.0 → 1) — matches vLLM's val.is_integer() rule.
- No ast.literal_eval anywhere — vLLM doesn't use it. Python-literal
  dicts ('k':1) no longer parse for object params.
- "binary" dropped as a bool alias (not in vLLM's _TYPE_ALIASES).

Renderers-specific deviation kept: used_fallback flag returned
alongside the value, propagated to ToolCallParseStatus.INVALID_JSON
for the verifier / RL-loss signal. vLLM has no such signal.

Tests updated for new behavior; full suite green (1775+99=1874 passed).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread renderers/parsing.py Outdated
Comment thread renderers/parsing.py
Three bugbot findings from PR #52:

- HIGH: ``int(val)`` in the number branch of ``_coerce_arg_value``
  was outside the try block, so ``float("nan")`` (raises ValueError
  on ``int()``) and ``float("inf")`` (raises OverflowError) would
  abort the whole parse. Move ``int(val)`` back inside the try and
  widen the catch to include ``OverflowError`` — minor improvement
  over vLLM, which crashes on ``inf``.
- LOW: ``_TOOL_CALL_BLOCK_RE`` was defined at module level but
  never used (carryover from the vLLM regex listing). Removed.
- MEDIUM: document the ``UNCLOSED_BLOCK`` > ``INVALID_JSON``
  status precedence in ``_parse_xml_function_blocks``. Structural
  failure dominates coercion failure when a block is both truncated
  and has uncoercible args; the verifier already discounts unclosed
  blocks, so combining the two would be noise. Intent now in a
  comment so future readers don't flip it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@HBX814

HBX814 commented Aug 22, 2026

Copy link
Copy Markdown

I ran this branch against the original repro from #47 — posting the numbers in case they help move it along.

main still reproduces the issue (at 1a5bd3b):

_coerce_arg_value("True",  {"type": "boolean"}) -> ('True',  True)   # used_json_fallback
_coerce_arg_value("False", {"type": "boolean"}) -> ('False', True)

End-to-end through Qwen35Renderer.parse_response, using @rohitrastogi's exact schema and completion:

status   : ToolCallParseStatus.INVALID_JSON
arguments: {'path': '/', 'max_depth': 2, 'include_files': 'True', 'show_size': 'True'}

This branch fixes it. Same script on fix/qwen35-arg-coercion-vllm-parity:

_coerce_arg_value("True", {"type": "boolean"}) -> (True, False)

status   : ToolCallParseStatus.OK
arguments: {'path': '/', 'max_depth': 2, 'include_files': True, 'show_size': True}

which lines up with what Qwen3CoderDetector._convert_param_value produces on the SGLang side.

Parsing tests on the branch — test_parsers.py, test_parse_response.py, test_parse_response_robustness.py, test_tool_arg_type_preservation.py, test_kimi_k25_tool_schema.py — 359 passed, 18 skipped.

One thing worth flagging before this can land: the branch is 55 commits behind origin/main (last merge from main was 2026-05-26), so newer test files such as tests/test_glm_tool_name_validation.py don't exist on it yet. Worth merging main in so CI runs against the current suite.

Environment: transformers 5.15.0, Qwen/Qwen3.5-0.8B tokenizer, no GPU.

@hallerite hallerite changed the title fix(parsing): vLLM-parity type ladder for XML tool-call arg coercion fix(parsing): align XML argument coercion with vLLM 0.32 Oct 18, 2026
@hallerite
hallerite force-pushed the fix/qwen35-arg-coercion-vllm-parity branch from 4fb7206 to a5a182f Compare October 18, 2026 21:54

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a5a182f. Configure here.

Comment thread renderers/parsing.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Qwen3.5 tool-call parser is stricter than SGLang qwen3_coder on boolean args

2 participants