Repository navigation
Closed
Labels
Description
I'm running a self-hosted "cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ" model using vLLM:
python3 -m vllm.entrypoints.openai.api_server --port 8080 --model cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ --tensor-parallel-size 1 --pipeline_parallel_size 3 --enable-auto-tool-choice --tool-call-parser qwen3_coder --max-model-len 262144"
It looks that the model is not able to call any tool (function) with OpenCode 0.4.12:
┃ ┃
┃ Todowrite ┃
┃ ┃
┃ The arguments provided to the tool are invalid: Invalid input for tool todowrite: Type validation failed: Value: {"todos":"[{\"content\": \"Examine the structure of ┃
┃ model/topology.yaml and model/node-metrics.yaml\", \"status\": \"in_progress\", \"priority\": \"high\", \"id\": \"task-1\"}]"}. ┃
┃ Error message: [ ┃
┃ { ┃
┃ "code": "invalid_type", ┃
┃ "expected": "array", ┃
┃ "received": "string", ┃
┃ "path": [ ┃
┃ "todos" ┃
┃ ], ┃
┃ "message": "Expected array, received string"
┃ Bash ┃
┃ ┃
┃ The arguments provided to the tool are invalid: Invalid input for tool bash: Type validation failed: Value: {"command":"ls -la model/"}. ┃
┃ Error message: [ ┃
┃ { ┃
┃ "code": "invalid_type", ┃
┃ "expected": "string", ┃
┃ "received": "undefined", ┃
┃ "path": [ ┃
┃ "description" ┃
┃ ], ┃
┃ "message": "Required"
┃ ┃
┃ Bash ┃
┃ ┃
┃ The arguments provided to the tool are invalid: Invalid input for tool bash: Type validation failed: Value: {"command":"mkdir -p ./tools"}. ┃
┃ Error message: [ ┃
┃ { ┃
┃ "code": "invalid_type", ┃
┃ "expected": "string", ┃
┃ "received": "undefined", ┃
┃ "path": [ ┃
┃ "description" ┃
┃ ], ┃
┃ "message": "Required"
I have the same experience with #1783: not only with the todo tool - but with all tools
I tested also with hermes but in this case there was no tool call at all.
The tests with OpenCode from end of last week run kind of better. At least the basic tools could be used.
Activity
thdxr commentedon Aug 11, 2025
i don't know if there is anything to do on ou rend - we've had good experiences with lmstudio
florath commentedon Aug 11, 2025
It would also help to understand where the problem is.
I assume that an LLM’s responses should not vary significantly depending on the underlying middleware.
Here is a trace of the response form the LLM that resulted in the attached error. (Removed one newline for better readability.)
When re-assembling this on the client side this should be the result:
I'm not an expert in this field, but this looks kind of valid.
But OpenCode logs:
Maybe this helps?
florath commentedon Aug 11, 2025
It looks that this is a problem with the LLM. The response is correctly formatted, but it is missing the description field, which is mandatory.
The description field is correctly marked as mandatory in the prompt:
and
But the response from the LLM is missing the response field:
instead of correctly
For me this is kind of strange that such a model is not capable of handling this correctly. I will try with different settings like tool-call-parser or context length (I just learned that the prompt is about 60k characters).
florath commentedon Aug 11, 2025
A random find. Could be related. Maybe anybody tried another quantization or the original?
Source: https://docs.unsloth.ai/basics/qwen3-coder-how-to-run-locally
florath commentedon Aug 12, 2025
Using a workaround of the above mentioned problem (see #1856) and moving to a version where the tool calling is fixed (unsloth/Qwen3-Coder-30B-A3B-Instruct), the problem disappeared (for now).
florath commentedon Aug 12, 2025
When running longer sessions, from time to time qwen3-coder just returns invalid JSON. E.g. stringified arrays, missing the "description" from time to time in the bash tool, and so on.
I created a small proxy which fixes the responses from the LLM. Maybe this is also of help for others?
https://github.com/florath/qwen3-call-patch-proxy
I have now a full working OpenCode with qwen3-coder. From my side this issue can be closed.
peaster commentedon Aug 13, 2025
@florath how are you configuring opencode with your proxy? I'd really like the AWQ quant to work via vLLM
florath commentedon Aug 14, 2025
Here my provider config:
For me the proxy is running on the same server as OpenCode. The proxy is then configured to point to my vLLM running the qwen3-coder model.
peaster commentedon Aug 18, 2025
this works well! thank you. For me, my root issue was a copypasta from an older qwen model that used a different tool call parser. I did run into the other issues you were seeing too
peaster commentedon Aug 30, 2025
As of opencode v0.5.29, https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8 calls tools flawlessly without any changes. I'm able to run that on dual 3090s with a 105,000 token context
rekram1-node commentedon Aug 30, 2025
@florath is this good to close then?