Repository navigation
Description
What is the issue?
ERROR: raise ResponseError(e.response.text, e.response.status_code) from None ollama._types.ResponseError: failed to parse JSON: unexpected end of JSON input (status code: 500)
OS: Mac osx : RAM 64G
Ollama client: pypi ollama 0.6.1
0llama version: 0.17.4
model: qwen3.5:35b
Whenever I config num_ctx as 8k,16k,64k,128k or 256k, this error will reproduced. Not sure it's a ollama-qwen3.5:35b compatibility problem. When I used qwen3:30b, the problem would not reproduced.
Full Content here 👇:
Bug Report: qwen3.5 tool call parsing fails with "failed to parse JSON: unexpected end of JSON input"
Issue Description
When using the qwen3.5 model (qwen35moe) with tool calling enabled via Ollama Python library, the API returns HTTP 500 errors. The server logs show:
qwen3 tool call parsing failed error="failed to parse JSON: unexpected end of JSON input"
This error occurs intermittently but repeatedly during chat sessions that involve tool calls.
Environment
- Ollama Version: 0.17.4
- macOS Version: Darwin/24.6.0 (macOS Sequoia)
- Hardware: Apple M1 Max (Metal GPU)
- GPU Memory: 48.0 GiB available
- Python Client: ollama Python library
Server Log Context
The following is the relevant excerpt from ~/.ollama/logs/server.log:
# Server startup and model loading
time=2026-02-27T12:46:48.855+08:00 level=INFO source=routes.go:1665 msg="Ollama cloud disabled: false"
time=2026-02-27T12:46:48.862+08:00 level=INFO source=routes.go:1718 msg="Listening on [::]:11434 (version 0.17.4)"
time=2026-02-27T12:46:49.005+08:00 level=INFO source=types.go:42 msg="inference compute" id=0 filter_id=0 library=Metal compute=0.0 name=Metal description="Apple M1 Max" libdirs="" driver=0.0 pci_id="" type=discrete total="48.0 GiB" available="48.0 GiB"
# Model loaded successfully
time=2026-02-27T12:50:20.741+08:00 level=INFO source=ggml.go:136 msg="" architecture=qwen35moe file_type=Q4_K_M name="" description="" num_tensors=1959 num_key_values=57
# Metal GPU initialization
ggml_metal_device_init: GPU name: Apple M1 Max
ggml_metal_device_init: GPU family: MTLGPUFamilyApple7 (1007)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal3 (5001)
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_device_init: has tensor = false
# Successful chat requests
[GIN] 2026/02/27 - 12:56:09 | 200 | 3m59s | ::1 | POST "/api/chat"
[GIN] 2026/02/27 - 12:56:28 | 200 | 18.148873417s | ::1 | POST "/api/chat"
[GIN] 2026/02/27 - 12:57:38 | 200 | 1m8s | ::1 | POST "/api/chat"
[GIN] 2026/02/27 - 12:57:57 | 200 | 18.438211334s | ::1 | POST "/api/chat"
# Tool call parsing failure
time=2026-02-27T12:59:31.677+08:00 level=WARN source=qwen3.go:108 msg="qwen3 tool call parsing failed" error="failed to parse JSON: unexpected end of JSON input"
# HTTP 500 error response
[GIN] 2026/02/27 - 12:59:31 | 500 | 1m34s | ::1 | POST "/api/chat"
Error Pattern
The error follows this consistent pattern:
- Multiple successful
POST /api/chatrequests (HTTP 200) - Model generates a tool call
qwen3 tool call parsing failedwarning appears in logs- Final request returns HTTP 500
Full Error from Python Client
ERROR: raise ResponseError(e.response.text, e.response.status_code) from None
ollama._types.ResponseError: failed to parse JSON: unexpected end of JSON input (status code: 500)
Additional Observations
- Error Source:
qwen3.go:108- This is internal Ollama code for handling Qwen3 model's tool calling - Occurrence Frequency: This error has been observed 10+ times in the server log within a few hours of usage
- Model: qwen3.5 (qwen35moe, Q4_K_M quantization)
- File Type: GGUF V3 (latest)
Expected Behavior
The model should either:
- Successfully generate valid JSON for tool calls, OR
- Return a graceful error message instead of causing a 500 error
Possible Root Cause
The Qwen3.5 model appears to generate incomplete JSON output when attempting tool calls. Ollama's internal JSON parser in qwen3.go fails to parse this incomplete JSON, causing the entire request to fail with HTTP 500.
Workarounds Tried
- Using different model: qwen3:30b-a3b-thinking work without this issue
Request
This appears to be an issue with how Ollama handles malformed tool call output from the Qwen3.5 model. Possible solutions:
- Add retry logic or JSON recovery in the qwen3.go parser
- Better error handling to return a more informative message to the client
- Document this known limitation for Qwen3.5 tool calling
Logs Location: ~/.ollama/logs/server.log
App Version: 0.17.4
Date: 2026-02-27
Activity
rick-github commentedon Feb 27, 2026
Server logs will aid in debugging.
FieldMouse-AI commentedon Feb 27, 2026
rick-github commentedon Feb 27, 2026
@meidaid Wrong place to post that, see #14419. In your case, ollama does not yet support qwen35/qwen35moe models downloaded from HF.
Jackallink commentedon Feb 28, 2026
I have added the log context and preliminary analysis diagnostics. 👉 issue #14492
rick-github commentedon Feb 28, 2026
Full log. Example code that reproduces the problem would also be helpful.
rick-github commentedon Mar 12, 2026
Possibly fixed in 0.17.6 via #14605. Please upgrade and test.
hashelq commentedon Mar 22, 2026
~ /bin/ollama --version ollama version is 0.18.2Qwen3.5:9BI still observe broken tool calls within the thinking block.
rick-github commentedon Mar 22, 2026
Logs and example code that reproduces the problem would be helpful.
hashelq commentedon Mar 22, 2026
Steps to reproduce (my case)
Setup:
OLLAMA_KV_CACHE_TYPE=q4_0 OLLAMA_KEEP_ALIVE=-1 OLLAMA_MAX_LOADED_MODELS=1 OLLAMA_NUM_PARALLEL=1 OLLAMA_MAX_QUEUE=1 OLLAMA_FLASH_ATTENTION=1 OLLAMA_DEBUG=3 /usr/local/bin/ollama serve 2> log.txtLog:
Details
HW:
rick-github commentedon Mar 22, 2026
Full log, from the start.
hashelq commentedon Mar 22, 2026
https://raw.githubusercontent.com/hashelq/ollama-log-temp1/refs/heads/master/log.txt
rick-github commentedon Mar 22, 2026
Where was the GGUF sourced from?
hashelq commentedon Mar 22, 2026
https://huggingface.co/unsloth/Qwen3.5-9B-GGUF
upd1:
Also I have same exact problems with
https://huggingface.co/Jackrong 's qwen3.5 cladue-4.6opus-dist v1 models
and
https://huggingface.co/Tesslate/OmniCoder-9B-GGUF
It seems all qwen3.5 and their forks are affected in my case
upd2:
just pulled
qwen3.5:2bwith ollama pullwith cfg:
Same issue
rick-github commentedon Mar 22, 2026
Do you have the issue if you use the model from the ollama library?
hashelq commentedon Mar 23, 2026
Yes, I demonstrate it above with qwen3.5:2b
hashelq commentedon Mar 23, 2026
Interestingly, seems like switching to
for qwen3.5:9b fixes the basic tool calling (read/list/grep/whatever) problems for a trade off of broken think tag parsing.
But with this setup I seen the qwen could not correctly call the opencode's questions tool..
hashelq commentedon Mar 24, 2026
I found that sometimes qwen3.5 models fail to produce "</think>" at the end of thinking.
Here is the omnicoder 9b (qwen3.5:9b fork) raw:
It clearly misses the </think> after word "better".
And the debugging code is model/parsers/qwen35.go
... I was not using flash attention or KV_CACHE_TYPE for recent tests
Other observations:
rick-github commentedon Mar 24, 2026
#15022