Repository navigation
Commit 0a57ff7
committed
feat(vllm): switch from Gemma to Qwen3-Coder for better tool calling support
- Replace Gemma 3 12B with Qwen3-Coder-30B-A3B (AWQ 4-bit)
- Change quantization from bitsandbytes to awq (more efficient)
- Update tool parser from hermes to qwen3_coder (native support)
- Increase context window from 16k to 32k tokens
- Remove --trust-remote-code flag (not needed for Qwen)
- VRAM usage: ~17GB (fits comfortably in 24GB GPU)
Based on anomalyco/opencode#1122
Qwen models have better tool calling compatibility with OpenCode
🤖 Generated with Claude Code1 parent 417a29b commit 0a57ff7
1 file changed
Lines changed: 8 additions & 9 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
| 15 | + | |
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
20 | | - | |
| 19 | + | |
| 20 | + | |
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| |||
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
52 | | - | |
53 | | - | |
| 52 | + | |
| 53 | + | |
54 | 54 | | |
55 | | - | |
| 55 | + | |
56 | 56 | | |
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
61 | | - | |
| 61 | + | |
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
68 | | - | |
69 | | - | |
| 68 | + | |
70 | 69 | | |
71 | 70 | | |
72 | 71 | | |
| |||
0 commit comments