Skip to content

qwen3-coder-30B-A3B is not able to call any tool #1809

Closed

Description

@florath

I'm running a self-hosted "cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ" model using vLLM:

python3 -m vllm.entrypoints.openai.api_server --port 8080 --model cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ --tensor-parallel-size 1 --pipeline_parallel_size 3 --enable-auto-tool-choice --tool-call-parser qwen3_coder --max-model-len 262144"

It looks that the model is not able to call any tool (function) with OpenCode 0.4.12:

  ┃                                                                                                                                                                                ┃
  ┃  Todowrite                                                                                                                                                                     ┃
  ┃                                                                                                                                                                                ┃
  ┃  The arguments provided to the tool are invalid: Invalid input for tool todowrite: Type validation failed: Value: {"todos":"[{\"content\": \"Examine the structure of          ┃
  ┃  model/topology.yaml and model/node-metrics.yaml\", \"status\": \"in_progress\", \"priority\": \"high\", \"id\": \"task-1\"}]"}.                                               ┃
  ┃  Error message: [                                                                                                                                                              ┃
  ┃    {                                                                                                                                                                           ┃
  ┃      "code": "invalid_type",                                                                                                                                                   ┃
  ┃      "expected": "array",                                                                                                                                                      ┃
  ┃      "received": "string",                                                                                                                                                     ┃
  ┃      "path": [                                                                                                                                                                 ┃
  ┃        "todos"                                                                                                                                                                 ┃
  ┃      ],                                                                                                                                                                        ┃
  ┃      "message": "Expected array, received string"  

  ┃  Bash                                                                                                                                                                          ┃
  ┃                                                                                                                                                                                ┃
  ┃  The arguments provided to the tool are invalid: Invalid input for tool bash: Type validation failed: Value: {"command":"ls -la model/"}.                                      ┃
  ┃  Error message: [                                                                                                                                                              ┃
  ┃    {                                                                                                                                                                           ┃
  ┃      "code": "invalid_type",                                                                                                                                                   ┃
  ┃      "expected": "string",                                                                                                                                                     ┃
  ┃      "received": "undefined",                                                                                                                                                  ┃
  ┃      "path": [                                                                                                                                                                 ┃
  ┃        "description"                                                                                                                                                           ┃
  ┃      ],                                                                                                                                                                        ┃
  ┃      "message": "Required" 

  ┃                                                                                                                                                                                ┃
  ┃  Bash                                                                                                                                                                          ┃
  ┃                                                                                                                                                                                ┃
  ┃  The arguments provided to the tool are invalid: Invalid input for tool bash: Type validation failed: Value: {"command":"mkdir -p ./tools"}.                                   ┃
  ┃  Error message: [                                                                                                                                                              ┃
  ┃    {                                                                                                                                                                           ┃
  ┃      "code": "invalid_type",                                                                                                                                                   ┃
  ┃      "expected": "string",                                                                                                                                                     ┃
  ┃      "received": "undefined",                                                                                                                                                  ┃
  ┃      "path": [                                                                                                                                                                 ┃
  ┃        "description"                                                                                                                                                           ┃
  ┃      ],                                                                                                                                                                        ┃
  ┃      "message": "Required"     

I have the same experience with #1783: not only with the todo tool - but with all tools

I tested also with hermes but in this case there was no tool call at all.

The tests with OpenCode from end of last week run kind of better. At least the basic tools could be used.

Activity

  1. self-assigned this
    on Aug 11, 2025
  2. thdxr commented on Aug 11, 2025

    @thdxr
    Member

    i don't know if there is anything to do on ou rend - we've had good experiences with lmstudio

  3. florath commented on Aug 11, 2025

    @florath
    Author

    It would also help to understand where the problem is.
    I assume that an LLM’s responses should not vary significantly depending on the underlying middleware.

    Here is a trace of the response form the LLM that resulted in the attached error. (Removed one newline for better readability.)

    data: {"id":"chatcmpl-0647664af2c74d868f3580183b7da97d","object":"chat.completion.chunk","created":1754926598,"model":"cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ","choices":[{"index":0,"delta":{"content":" properly"},"logprobs":null,"finish_reason":null}]}
    
    101
    data: {"id":"chatcmpl-0647664af2c74d868f3580183b7da97d","object":"chat.completion.chunk","created":1754926598,"model":"cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ","choices":[{"index":0,"delta":{"content":" formatted"},"logprobs":null,"finish_reason":null}]}
    
    fc
    data: {"id":"chatcmpl-0647664af2c74d868f3580183b7da97d","object":"chat.completion.chunk","created":1754926598,"model":"cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ","choices":[{"index":0,"delta":{"content":":\n\n"},"logprobs":null,"finish_reason":null}]}
    
    166
    data: {"id":"chatcmpl-0647664af2c74d868f3580183b7da97d","object":"chat.completion.chunk","created":1754926598,"model":"cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ","choices":[{"index":0,"delta":{"tool_calls":[{"id":"call_5e8b4c3144f24e2a95348342","type":"function","index":0,"function":{"name":"bash","arguments":""}}]},"logprobs":null,"finish_reason":null}]}
    
    122
    data: {"id":"chatcmpl-0647664af2c74d868f3580183b7da97d","object":"chat.completion.chunk","created":1754926598,"model":"cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{"}}]},"logprobs":null,"finish_reason":null}]}
    
    15b
    data: {"id":"chatcmpl-0647664af2c74d868f3580183b7da97d","object":"chat.completion.chunk","created":1754926598,"model":"cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"command\": \"cd /home/florath/devel/TEST/ttt && ls -la\""}}]},"logprobs":null,"finish_reason":null}]}
    
    122
    data: {"id":"chatcmpl-0647664af2c74d868f3580183b7da97d","object":"chat.completion.chunk","created":1754926598,"model":"cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"}"}}]},"logprobs":null,"finish_reason":null}]}
    
    112
    data: {"id":"chatcmpl-0647664af2c74d868f3580183b7da97d","object":"chat.completion.chunk","created":1754926598,"model":"cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ","choices":[{"index":0,"delta":{"content":""},"logprobs":null,"finish_reason":"tool_calls","stop_reason":null}]}
    
    e
    data: [DONE]
    
    

    When re-assembling this on the client side this should be the result:

    {
      "tool_calls": [
        {
          "id": "call_5e8b4c3144f24e2a95348342",
          "type": "function",
          "index": 0,
          "function": {
            "name": "bash",
            "arguments": {
              "command": "cd /home/florath/devel/TEST/ttt && ls -la"
            }
          }
        }
      ]
    }
    
    

    I'm not an expert in this field, but this looks kind of valid.

    But OpenCode logs:

    Image

    Maybe this helps?

  4. florath commented on Aug 11, 2025

    @florath
    Author

    It looks that this is a problem with the LLM. The response is correctly formatted, but it is missing the description field, which is mandatory.

      parameters: z.object({
        command: z.string().describe("The command to execute"),
        timeout: z.number().describe("Optional timeout in milliseconds").optional(),
        description: z
          .string()
          .describe(
            "Clear, concise description of what this command does in 5-10 words. Examples:\nInput: ls\nOutput: Lists files in current directory\n\nInput: git status\nOutput: 
      Shows working tree status\n\nInput: npm install\nOutput: Installs package dependencies\n\nInput: mkdir foo\nOutput: Creates directory 'foo'",
          ),
      }),
    

    The description field is correctly marked as mandatory in the prompt:

     "required":["command","description"]
    

    and

      "description":{
        "type":"string",
        "description":"Clear, concise description of what this command does in 5-10 words. Examples:\nInput: ls\nOutput: Lists files in current directory..."
      }
    
    

    But the response from the LLM is missing the response field:

    {"command": "cd /home/florath/devel/TEST/ttt && ls -la"}
    

    instead of correctly

      {
        "command": "cd /home/florath/devel/TEST/ttt && ls -la",
        "description": "Lists files in directory with details"
      }
    
    

    For me this is kind of strange that such a model is not capable of handling this correctly. I will try with different settings like tool-call-parser or context length (I just learned that the prompt is about 60k characters).

  5. florath commented on Aug 11, 2025

    @florath
    Author

    A random find. Could be related. Maybe anybody tried another quantization or the original?

    Image

    Source: https://docs.unsloth.ai/basics/qwen3-coder-how-to-run-locally

  6. florath commented on Aug 12, 2025

    @florath
    Author

    Using a workaround of the above mentioned problem (see #1856) and moving to a version where the tool calling is fixed (unsloth/Qwen3-Coder-30B-A3B-Instruct), the problem disappeared (for now).

  7. florath commented on Aug 12, 2025

    @florath
    Author

    When running longer sessions, from time to time qwen3-coder just returns invalid JSON. E.g. stringified arrays, missing the "description" from time to time in the bash tool, and so on.

    I created a small proxy which fixes the responses from the LLM. Maybe this is also of help for others?
    https://github.com/florath/qwen3-call-patch-proxy

    I have now a full working OpenCode with qwen3-coder. From my side this issue can be closed.

  8. peaster commented on Aug 13, 2025

    @peaster

    @florath how are you configuring opencode with your proxy? I'd really like the AWQ quant to work via vLLM

  9. florath commented on Aug 14, 2025

    @florath
    Author

    Here my provider config:

    "provider": {
    	"qwen3-coder-30b-a3b-instruct-vllm": {
    	    "name": "Qwen 3 Coder 30B (vLLM)",
    	    "npm": "@ai-sdk/openai-compatible",
    	    "models": {
    		  "cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ": {}
    	    },
    	    "options": {
    	  	  "baseURL": "http://127.0.0.1:7999/v1"
    	    }
    	}
    },

    For me the proxy is running on the same server as OpenCode. The proxy is then configured to point to my vLLM running the qwen3-coder model.

  10. peaster commented on Aug 18, 2025

    @peaster

    this works well! thank you. For me, my root issue was a copypasta from an older qwen model that used a different tool call parser. I did run into the other issues you were seeing too

  11. peaster commented on Aug 30, 2025

    @peaster

    As of opencode v0.5.29, https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8 calls tools flawlessly without any changes. I'm able to run that on dual 3090s with a 105,000 token context

    vllm serve Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8 --tool-call-parser qwen3_coder --enable-auto-tool-choice --gpu-memory-utilization 0.9 --tensor-parallel-size 2 --max_model_len 105000
    
    Image
  12. rekram1-node commented on Aug 30, 2025

    @rekram1-node
    Collaborator

    @florath is this good to close then?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions