You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
vLLM 0.19 may lose tool calls for Qwen/Qwen3.5-35B-A3B-FP8 when XML tool_call is emitted inside <think> #39056
In this case, the model may have produced a valid tool call, but the OpenAI-compatible response ends up with populated reasoning and empty tool_calls.
Observed model
The issue was observed and reproduced with:
Qwen/Qwen3.5-35B-A3B-FP8
It may also affect other Qwen3 / Qwen3.5 models that use the same parser combination, but the confirmed reproduction here is specifically on Qwen/Qwen3.5-35B-A3B-FP8.
Why this happens
The issue appears to come from the interaction between the reasoning parser and the tool parser:
qwen3_reasoning_parser extracts everything before </think> into reasoning.
downstream tool parsing only inspects content.
if <tool_call>...</tool_call> remains inside reasoning, it never reaches qwen3_coder.
So the bug is not that vLLM makes Qwen3.5 generate tool calls inside <think>.
The bug is that vLLM currently does not recover those tool calls when that output pattern occurs.
Scope
This report is specifically about the non-streaming path, which can break benchmark / agent flows that rely on structured tool_calls being returned.
Proposed fix
I have opened a PR with a minimal parser-side fix here:
That patch promotes embedded XML tool-call blocks out of reasoning into content in qwen3_reasoning_parser, so the existing qwen3_coder tool parser can still parse them.
Request
Please consider reviewing and merging the PR if the approach looks acceptable.
Reacted by Chan Kha Vu, Christopher T, nleve, Matthew Farrellee, Artemiy Marchenko, Iván Baldo, leason, Robin Bially, Jonathan Leaders, Abhilash Sulibela and 3 more
changed the title [-]vLLM 0.19 may lose Qwen3.5 tool calls when XML tool_call is emitted inside <think>[/-][+]vLLM 0.19 may lose tool calls for Qwen/Qwen3.5-35B-A3B-FP8 when XML tool_call is emitted inside <think>[/+]on Apr 6, 2026
Yes, it's very strange. Everything works properly when I switch from Qwen3.5 to Gemma4, which shows my code agent framework is fine. I used to think it was my fault!
I have a sudden thought: perhaps it's a bug caused by dependency conflicts after upgrading vllm to 0.19.0 to be compatible with Gemma4. This is mainly because I found that for vllm, qwen3.5 depends on the 4.x version of the transformers library, while gemma4 depends on the 5.x version.
@ZenoAFfectionate did you try to replicate on any previous versions of vLLM? Maybe temporary solution for people who need qwen3.5 to work properly is downgrade a bit?
@ZenoAFfectionate did you try to replicate on any previous versions of vLLM? Maybe temporary solution for people who need qwen3.5 to work properly is downgrade a bit?
When Qwen3.5 first came out, it worked fine. I remember the vLLM version back then was 0.17, but after updating to 0.18 and 0.19, it became unusable – that’s what shocked me the most. So maybe you could try downgrading to version 0.17~
@ZenoAFfectionate did you try to replicate on any previous versions of vLLM? Maybe temporary solution for people who need qwen3.5 to work properly is downgrade a bit?
When Qwen3.5 first came out, it worked fine. I remember the vLLM version back then was 0.17, but after updating to 0.18 and 0.19, it became unusable – that’s what shocked me the most. So maybe you could try downgrading to version 0.17~
I run vLLM on lates cuda-130:nightly docker image build which is v0.19. Seems like I wont get away from crafting my own docker image for blackwell architecture and 0.17 vLLM.
Something that seems to be a workaround is to use the anthropic api instead of openai api to communicate with VLLM (works with both Claude Code and OpenCode).
Through Claude Code I've run it for a couple of days without any issues. And now for a few hours in OpenCode without issues.
We have a complete rewrite of the streaming qwen3 parser (as the first of several to use a new streaming parser engine) in #45413 if anyone from this thread that reported the various permutations of this type of issue wants to give things a try with that one. Building with that PR locally will automatically update your qwen3_coder/qwen3_xml reasoning tool call parsers and qwen3 reasoning parser to the new parser engine.
Independent confirmation & version-range gap for PR consideration
The root-cause analysis in this issue is accurate. This isn't specific to the 35B model—the same qwen3 reasoning parser and qwen3_coder tool parser interaction is reproducible across other Qwen3.5 variants as well (including Qwen3.5-27B-AWQ).
For the fix and corresponding test coverage in PR #39055, it is worth explicitly accounting for the behavior across the >=0.20.0 to <0.24.0 version range, where this is only partially addressed:
<=0.19.0: The tool call emitted inside the think block never reaches the tool parser.
>=0.20.0 <0.24.0: Earlier parser changes partially resolved chunk routing, but the paired tool_call block pattern described in this report can still be missed.
>=0.24.0: Resolved for this specific pattern following the code path rewrite.
Ensuring test coverage spans the 0.20–0.23 release behavior will prevent regressions between earlier parser iterations and the full fix.
For reference and CI regression tracking, the reported failure shape (both non-streaming and chunked streaming variants) is tracked here:
Activity
[-]vLLM 0.19 may lose Qwen3.5 tool calls when XML tool_call is emitted inside <think>[/-][+]vLLM 0.19 may lose tool calls for Qwen/Qwen3.5-35B-A3B-FP8 when XML tool_call is emitted inside <think>[/+]hav4ik commentedon Apr 6, 2026
I've been observing similar effect with
nvidia/Nemotron-Cascade2-30B-A3Bas well (it usesqwen3_codertool parser).Mr-Righter commentedon Apr 6, 2026
Deploying Qwen/Qwen3.5-122B-A10B-FP8 with the following arguments:
--model Qwen/Qwen3.5-122B-A10B-FP8--async-scheduling--tensor-parallel-size 2--enable-prefix-caching--enable-auto-tool-choiceS--tool-call-parser qwen3_coder--reasoning-parser qwen3S--max_num_batched_tokens 4096--disable-custom-all-reduceObservation: The agent loop may terminate unexpectedly because the returned result does not contain a function call.
ZenoAFfectionate commentedon Apr 6, 2026
Yes, it's very strange. Everything works properly when I switch from Qwen3.5 to Gemma4, which shows my code agent framework is fine. I used to think it was my fault!
czerwiakowskim commentedon Apr 6, 2026
Have same issue with OpenCode (also appears in Claude Code)
this stops agent loop in random moments. Appears despite if using qwen3_coder or qwen3_xml tool parser
dcolt commentedon Apr 6, 2026
It's nice to see that I'm not alone in experiencing these issues.
I've been having them since basically the Qwen3.5-122b model got released. (Running on a DGX Spark). But I had no idea what was causing the issue.
bitbottrap commentedon Apr 6, 2026
Another nice enhancement would be combing "tool_choice=auto" with guided decoding functionality with structural tags.
ZenoAFfectionate commentedon Apr 7, 2026
I have a sudden thought: perhaps it's a bug caused by dependency conflicts after upgrading vllm to 0.19.0 to be compatible with Gemma4. This is mainly because I found that for vllm, qwen3.5 depends on the 4.x version of the transformers library, while gemma4 depends on the 5.x version.
czerwiakowskim commentedon Apr 7, 2026
@ZenoAFfectionate did you try to replicate on any previous versions of vLLM? Maybe temporary solution for people who need qwen3.5 to work properly is downgrade a bit?
ZenoAFfectionate commentedon Apr 8, 2026
When Qwen3.5 first came out, it worked fine. I remember the vLLM version back then was 0.17, but after updating to 0.18 and 0.19, it became unusable – that’s what shocked me the most. So maybe you could try downgrading to version 0.17~
czerwiakowskim commentedon Apr 8, 2026
I run vLLM on lates cuda-130:nightly docker image build which is v0.19. Seems like I wont get away from crafting my own docker image for blackwell architecture and 0.17 vLLM.
BTW: In the meanwhile i've created simple proxy resoling problem with malformed tool calls: https://github.com/czerwiakowskim/qwen-toolcall-fixer it's OpenAI API compatible, so if anyone need a quick fix working for most of the time grab it and use it :)
dcolt commentedon Apr 8, 2026
Something that seems to be a workaround is to use the anthropic api instead of openai api to communicate with VLLM (works with both Claude Code and OpenCode).
Through Claude Code I've run it for a couple of days without any issues. And now for a few hours in OpenCode without issues.
No idea why it would actually make a difference.
epheien commentedon Apr 9, 2026
Same issue in OpenCode.
thad0ctor commentedon Apr 10, 2026
I’m seeing what looks like the same issue on a local Qwen3.5 + OpenClaw setup.
Environment:
Server flags:
Observed behavior:
consistent with tool calls ending up inside LLM response
athanasiaorange commentedon Apr 10, 2026
我也遇到了相同的问题,本地vllm+Qwen3.5-35B,openclaw调用LLM时,一但思考就会中断,导致回答总是原地踏步。
9 remaining items
release: v0.0.17 (#66)
bbrowning commentedon Jun 12, 2026
We have a complete rewrite of the streaming qwen3 parser (as the first of several to use a new streaming parser engine) in #45413 if anyone from this thread that reported the various permutations of this type of issue wants to give things a try with that one. Building with that PR locally will automatically update your qwen3_coder/qwen3_xml reasoning tool call parsers and qwen3 reasoning parser to the new parser engine.
Add Qwen3.6 thinking-mode sampling defaults across all 6 recipes
0DukePan commentedon Sep 7, 2026
Independent confirmation & version-range gap for PR consideration
The root-cause analysis in this issue is accurate. This isn't specific to the 35B model—the same
qwen3reasoning parser andqwen3_codertool parser interaction is reproducible across other Qwen3.5 variants as well (including Qwen3.5-27B-AWQ).For the fix and corresponding test coverage in PR #39055, it is worth explicitly accounting for the behavior across the >=0.20.0 to <0.24.0 version range, where this is only partially addressed:
thinkblock never reaches the tool parser.tool_callblock pattern described in this report can still be missed.Ensuring test coverage spans the 0.20–0.23 release behavior will prevent regressions between earlier parser iterations and the full fix.
For reference and CI regression tracking, the reported failure shape (both non-streaming and chunked streaming variants) is tracked here:
Happy to re-test the proposed PR build against these raw fixture patterns once it is ready.
[Bugfix][Parser] Extract tool calls emitted within reasoning blocks (v…