Repository navigation
Description
Name and Version
$ llama-server --version
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
version: 7951 (22cae83)
built with Clang 21.0.0 for Linux x86_64
Operating systems
Linux
GGML backends
CUDA
Hardware
AMD Ryzen 9 7950X3D 16-Core Processor + NVIDIA GeForce RTX 4090
Models
Qwen3-Coder-Next quantized with b7951 at MXFP4MOE (also tried IQ4_NL, IQ4_XS and Qwen/Qwen3-Coder-Next-Q8_0)
llama.cpp/build/bin/llama-server -c 0 -m Qwen3-Coder-Next.MXFP4MOE.gguf -t 16 \
--min-p 0.3 -fitt 384 --no-warmup --temp 0.0Problem description & steps to reproduce
I tried using Qwen3-Coder-Next with opencode for the last 2 days, it always produces the same error when trying to write a file (across the last 3 opencode versions and many llama.cpp versions during the last 2 days).
Any opencode prompt that leads to writing a file will cause the JSON output to be broken at the same place, it always writes a duplicate "filePath" field and messes up the syntax in the latter (misses a colon and a quote).
$ opencode --version
1.1.51
$ opencode --prompt 'Write a C++ "Hello World" program, use dprintf(), compile it with clang++ and run it.'
┃
┃ Write a C++ "Hello World" program, use dprintf(), compile it with clang++ and run it.
┃
⚙ invalid [tool=write, error=Invalid input for tool write: JSON parsing failed: Text: {"content":"#include <cstdio>\n\nint main() {\n dprintf(STDOUT_FILENO, \"Hello World\\n\");\n return 0;\n}","filePath":"/home/user/hello.cpp","filePath"/home/user/hello.cpp"}.
Error message: JSON Parse error: Unrecognized token '/']
⚙ invalid [tool=write, error=Invalid input for tool write: JSON parsing failed: Text: {"content":"#include <cstdio>\n\nint main() {\n dprintf(STDOUT_FILENO, \"Hello World\\n\");\n return 0;\n}","filePath":"/home/user/hello.cpp","filePath"/home/user/hello.cpp"}.
Error message: JSON Parse error: Unrecognized token '/']
It keeps trying to write the file, but always produces the same error.
Same error with other prompts, it always writes 2 "filePath" fields and messes up the syntax in the latter.
Other models do not show this behavior (e.g. Minimax-M2.1, Qwen3-Coder-30B-A3B-Instruct or gpt-oss-120b work fine).
First Bad Commit
No response
Relevant log output
Logs
Log for:llama-server -c 0 -m /ml/models/Qwen3-Coder-Next.MXFP4MOE.gguf -t 16 --port 1111 --host 0.0.0.0 --min-p 0.3 -fitt 384 --no-warmup --temp 0.0
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
main: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
build: 7951 (22cae8321) with Clang 21.0.0 for Linux x86_64
system info: n_threads = 16, n_threads_batch = 16, total_threads = 32
system_info: n_threads = 16 (n_threads_batch = 16) / 32 | CUDA : ARCHS = 890 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | REPACK = 1 |
Running without SSL
init: using 31 threads for HTTP server
start: binding port with default address family
main: loading model
srv load_model: loading model '/ml/models/Qwen3-Coder-Next.MXFP4MOE.gguf'
common_init_result: fitting params to device memory, for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on
llama_params_fit_impl: projected to use 48643 MiB of device memory vs. 23396 MiB of free device memory
llama_params_fit_impl: cannot meet free memory target of 384 MiB, need to reduce device memory by 25631 MiB
llama_params_fit_impl: user has requested full context size of 262144 -> no change
llama_params_fit_impl: with only dense weights in device memory there is a total surplus of 13281 MiB
llama_params_fit_impl: filling dense-only layers back-to-front:
llama_params_fit_impl: - CUDA0 (NVIDIA GeForce RTX 4090): 49 layers, 10547 MiB used, 12849 MiB free
llama_params_fit_impl: converting dense-only layers to full layers and filling them front-to-back with overflow to next device/system memory:
llama_params_fit_impl: - CUDA0 (NVIDIA GeForce RTX 4090): 49 layers (33 overflowing), 22797 MiB used, 599 MiB free
llama_params_fit: successfully fit params to free device memory
llama_params_fit: fitting params to free memory took 1.57 seconds
llama_model_load_from_file_impl: using device CUDA0 (NVIDIA GeForce RTX 4090) (0000:01:00.0) - 23698 MiB free
llama_model_loader: loaded meta data with 41 key-value pairs and 843 tensors from /ml/models/Qwen3-Coder-Next.MXFP4MOE.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = qwen3next
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.sampling.top_k i32 = 40
llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000
llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000
llama_model_loader: - kv 5: general.size_label str = 512x2.5B
llama_model_loader: - kv 6: general.license str = apache-2.0
llama_model_loader: - kv 7: general.license.link str = https://huggingface.co/Qwen/Qwen3-Cod...
llama_model_loader: - kv 8: general.tags arr[str,1] = ["text-generation"]
llama_model_loader: - kv 9: qwen3next.block_count u32 = 48
llama_model_loader: - kv 10: qwen3next.context_length u32 = 262144
llama_model_loader: - kv 11: qwen3next.embedding_length u32 = 2048
llama_model_loader: - kv 12: qwen3next.feed_forward_length u32 = 5120
llama_model_loader: - kv 13: qwen3next.attention.head_count u32 = 16
llama_model_loader: - kv 14: qwen3next.attention.head_count_kv u32 = 2
llama_model_loader: - kv 15: qwen3next.rope.freq_base f32 = 5000000.000000
llama_model_loader: - kv 16: qwen3next.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 17: qwen3next.expert_used_count u32 = 10
llama_model_loader: - kv 18: qwen3next.attention.key_length u32 = 256
llama_model_loader: - kv 19: qwen3next.attention.value_length u32 = 256
llama_model_loader: - kv 20: qwen3next.expert_count u32 = 512
llama_model_loader: - kv 21: qwen3next.expert_feed_forward_length u32 = 512
llama_model_loader: - kv 22: qwen3next.expert_shared_feed_forward_length u32 = 512
llama_model_loader: - kv 23: qwen3next.ssm.conv_kernel u32 = 4
llama_model_loader: - kv 24: qwen3next.ssm.state_size u32 = 128
llama_model_loader: - kv 25: qwen3next.ssm.group_count u32 = 16
llama_model_loader: - kv 26: qwen3next.ssm.time_step_rank u32 = 32
llama_model_loader: - kv 27: qwen3next.ssm.inner_size u32 = 4096
llama_model_loader: - kv 28: qwen3next.rope.dimension_count u32 = 64
llama_model_loader: - kv 29: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 30: tokenizer.ggml.pre str = qwen2
llama_model_loader: - kv 31: tokenizer.ggml.tokens arr[str,151936] = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 32: tokenizer.ggml.token_type arr[i32,151936] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 33: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv 34: tokenizer.ggml.eos_token_id u32 = 151645
llama_model_loader: - kv 35: tokenizer.ggml.padding_token_id u32 = 151643
llama_model_loader: - kv 36: tokenizer.ggml.bos_token_id u32 = 151643
llama_model_loader: - kv 37: tokenizer.ggml.add_bos_token bool = false
llama_model_loader: - kv 38: tokenizer.chat_template str = {% macro render_extra_keys(json_dict,...
llama_model_loader: - kv 39: general.quantization_version u32 = 2
llama_model_loader: - kv 40: general.file_type u32 = 38
llama_model_loader: - type f32: 313 tensors
llama_model_loader: - type q8_0: 338 tensors
llama_model_loader: - type bf16: 48 tensors
llama_model_loader: - type mxfp4: 144 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = MXFP4 MoE
print_info: file size = 40.73 GiB (4.39 BPW)
load: 0 unused tokens
load: printing all EOG tokens:
load: - 151643 ('<|endoftext|>')
load: - 151645 ('<|im_end|>')
load: - 151662 ('<|fim_pad|>')
load: - 151663 ('<|repo_name|>')
load: - 151664 ('<|file_sep|>')
load: special tokens cache size = 26
load: token to piece cache size = 0.9311 MB
print_info: arch = qwen3next
print_info: vocab_only = 0
print_info: no_alloc = 0
print_info: n_ctx_train = 262144
print_info: n_embd = 2048
print_info: n_embd_inp = 2048
print_info: n_layer = 48
print_info: n_head = 16
print_info: n_head_kv = 2
print_info: n_rot = 64
print_info: n_swa = 0
print_info: is_swa_any = 0
print_info: n_embd_head_k = 256
print_info: n_embd_head_v = 256
print_info: n_gqa = 8
print_info: n_embd_k_gqa = 512
print_info: n_embd_v_gqa = 512
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-06
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: f_attn_scale = 0.0e+00
print_info: n_ff = 5120
print_info: n_expert = 512
print_info: n_expert_used = 10
print_info: n_expert_groups = 0
print_info: n_group_used = 0
print_info: causal attn = 1
print_info: pooling type = 0
print_info: rope type = 2
print_info: rope scaling = linear
print_info: freq_base_train = 5000000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn = 262144
print_info: rope_yarn_log_mul = 0.0000
print_info: rope_finetuned = unknown
print_info: ssm_d_conv = 4
print_info: ssm_d_inner = 4096
print_info: ssm_d_state = 128
print_info: ssm_dt_rank = 32
print_info: ssm_n_group = 16
print_info: ssm_dt_b_c_rms = 0
print_info: model type = 80B.A3B
print_info: model params = 79.67 B
print_info: general.name = n/a
print_info: vocab type = BPE
print_info: n_vocab = 151936
print_info: n_merges = 151387
print_info: BOS token = 151643 '<|endoftext|>'
print_info: EOS token = 151645 '<|im_end|>'
print_info: EOT token = 151645 '<|im_end|>'
print_info: PAD token = 151643 '<|endoftext|>'
print_info: LF token = 198 'Ċ'
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
print_info: FIM MID token = 151660 '<|fim_middle|>'
print_info: FIM PAD token = 151662 '<|fim_pad|>'
print_info: FIM REP token = 151663 '<|repo_name|>'
print_info: FIM SEP token = 151664 '<|file_sep|>'
print_info: EOG token = 151643 '<|endoftext|>'
print_info: EOG token = 151645 '<|im_end|>'
print_info: EOG token = 151662 '<|fim_pad|>'
print_info: EOG token = 151663 '<|repo_name|>'
print_info: EOG token = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true, direct_io = false)
load_tensors: offloading output layer to GPU
load_tensors: offloading 47 repeating layers to GPU
load_tensors: offloaded 49/49 layers to GPU
load_tensors: CPU_Mapped model buffer size = 41393.00 MiB
load_tensors: CUDA0 model buffer size = 15547.94 MiB
....................................................................................................
common_init_result: added <|endoftext|> logit bias = -inf
common_init_result: added <|im_end|> logit bias = -inf
common_init_result: added <|fim_pad|> logit bias = -inf
common_init_result: added <|repo_name|> logit bias = -inf
common_init_result: added <|file_sep|> logit bias = -inf
llama_context: constructing llama_context
llama_context: n_seq_max = 4
llama_context: n_ctx = 262144
llama_context: n_ctx_seq = 262144
llama_context: n_batch = 2048
llama_context: n_ubatch = 512
llama_context: causal_attn = 1
llama_context: flash_attn = auto
llama_context: kv_unified = true
llama_context: freq_base = 5000000.0
llama_context: freq_scale = 1
llama_context: CUDA_Host output buffer size = 2.32 MiB
llama_kv_cache: CUDA0 KV buffer size = 6144.00 MiB
llama_kv_cache: size = 6144.00 MiB (262144 cells, 12 layers, 4/1 seqs), K (f16): 3072.00 MiB, V (f16): 3072.00 MiB
llama_memory_recurrent: CUDA0 RS buffer size = 301.50 MiB
llama_memory_recurrent: size = 301.50 MiB ( 4 cells, 48 layers, 4 seqs), R (f32): 13.50 MiB, S (f32): 288.00 MiB
sched_reserve: reserving ...
sched_reserve: Flash Attention was auto, set to enabled
sched_reserve: CUDA0 compute buffer size = 804.06 MiB
sched_reserve: CUDA_Host compute buffer size = 520.01 MiB
sched_reserve: graph nodes = 9266 (with bs=512), 5918 (with bs=1)
sched_reserve: graph splits = 101 (with bs=512), 72 (with bs=1)
sched_reserve: reserve took 208.51 ms, sched copies = 1
srv load_model: initializing slots, n_slots = 4
no implementations specified for speculative decoding
slot load_model: id 0 | task -1 | speculative decoding context not initialized
slot load_model: id 0 | task -1 | new slot, n_ctx = 262144
no implementations specified for speculative decoding
slot load_model: id 1 | task -1 | speculative decoding context not initialized
slot load_model: id 1 | task -1 | new slot, n_ctx = 262144
no implementations specified for speculative decoding
slot load_model: id 2 | task -1 | speculative decoding context not initialized
slot load_model: id 2 | task -1 | new slot, n_ctx = 262144
no implementations specified for speculative decoding
slot load_model: id 3 | task -1 | speculative decoding context not initialized
slot load_model: id 3 | task -1 | new slot, n_ctx = 262144
srv load_model: prompt cache is enabled, size limit: 8192 MiB
srv load_model: use `--cache-ram 0` to disable the prompt cache
srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
'
srv init: init: chat template, thinking = 0
main: model loaded
main: server is listening on http://0.0.0.0:1111
main: starting the main loop...
srv update_slots: all slots are idle
srv params_from_: Chat format: Qwen3 Coder
slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
slot launch_slot_: id 3 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
slot update_slots: id 3 | task 0 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 10610
slot update_slots: id 3 | task 0 | n_tokens = 0, memory_seq_rm [0, end)
slot update_slots: id 3 | task 0 | prompt processing progress, n_tokens = 2048, batch.n_tokens = 2048, progress = 0.193025
slot update_slots: id 3 | task 0 | n_tokens = 2048, memory_seq_rm [2048, end)
slot update_slots: id 3 | task 0 | prompt processing progress, n_tokens = 4096, batch.n_tokens = 2048, progress = 0.386051
slot update_slots: id 3 | task 0 | n_tokens = 4096, memory_seq_rm [4096, end)
slot update_slots: id 3 | task 0 | prompt processing progress, n_tokens = 6144, batch.n_tokens = 2048, progress = 0.579076
slot update_slots: id 3 | task 0 | n_tokens = 6144, memory_seq_rm [6144, end)
slot update_slots: id 3 | task 0 | prompt processing progress, n_tokens = 8192, batch.n_tokens = 2048, progress = 0.772102
slot update_slots: id 3 | task 0 | n_tokens = 8192, memory_seq_rm [8192, end)
slot update_slots: id 3 | task 0 | prompt processing progress, n_tokens = 10240, batch.n_tokens = 2048, progress = 0.965127
slot update_slots: id 3 | task 0 | n_tokens = 10240, memory_seq_rm [10240, end)
slot update_slots: id 3 | task 0 | prompt processing progress, n_tokens = 10546, batch.n_tokens = 306, progress = 0.993968
slot update_slots: id 3 | task 0 | n_tokens = 10546, memory_seq_rm [10546, end)
slot update_slots: id 3 | task 0 | prompt processing progress, n_tokens = 10610, batch.n_tokens = 64, progress = 1.000000
slot update_slots: id 3 | task 0 | prompt done, n_tokens = 10610, batch.n_tokens = 64
slot init_sampler: id 3 | task 0 | init sampler, took 0.77 ms, tokens: text = 10610, total = 10610
slot update_slots: id 3 | task 0 | created context checkpoint 1 of 8 (pos_min = 10545, pos_max = 10545, size = 75.376 MiB)
slot print_timing: id 3 | task 0 |
prompt eval time = 23588.03 ms / 10610 tokens ( 2.22 ms per token, 449.80 tokens per second)
eval time = 3170.93 ms / 74 tokens ( 42.85 ms per token, 23.34 tokens per second)
total time = 26758.96 ms / 10684 tokens
slot release: id 3 | task 0 | stop processing: n_tokens = 10683, truncated = 0
srv update_slots: all slots are idle
srv log_server_r: done request: POST /v1/chat/completions 10.111.3.51 200
srv params_from_: Chat format: Qwen3 Coder
slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.981 (> 0.100 thold), f_keep = 0.994
slot launch_slot_: id 3 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id 3 | task 81 | processing task, is_child = 0
slot update_slots: id 3 | task 81 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 10819
slot update_slots: id 3 | task 81 | n_past = 10615, slot.prompt.tokens.size() = 10683, seq_id = 3, pos_min = 10682, n_swa = 1
slot update_slots: id 3 | task 81 | restored context checkpoint (pos_min = 10545, pos_max = 10545, size = 75.376 MiB)
slot update_slots: id 3 | task 81 | n_tokens = 10546, memory_seq_rm [10546, end)
slot update_slots: id 3 | task 81 | prompt processing progress, n_tokens = 10755, batch.n_tokens = 209, progress = 0.994084
slot update_slots: id 3 | task 81 | n_tokens = 10755, memory_seq_rm [10755, end)
slot update_slots: id 3 | task 81 | prompt processing progress, n_tokens = 10819, batch.n_tokens = 64, progress = 1.000000
slot update_slots: id 3 | task 81 | prompt done, n_tokens = 10819, batch.n_tokens = 64
slot init_sampler: id 3 | task 81 | init sampler, took 0.79 ms, tokens: text = 10819, total = 10819
slot update_slots: id 3 | task 81 | created context checkpoint 2 of 8 (pos_min = 10754, pos_max = 10754, size = 75.376 MiB)
slot print_timing: id 3 | task 81 |
prompt eval time = 1717.10 ms / 273 tokens ( 6.29 ms per token, 158.99 tokens per second)
eval time = 3111.03 ms / 74 tokens ( 42.04 ms per token, 23.79 tokens per second)
total time = 4828.13 ms / 347 tokens
slot release: id 3 | task 81 | stop processing: n_tokens = 10892, truncated = 0
srv update_slots: all slots are idle
srv log_server_r: done request: POST /v1/chat/completions 10.111.3.51 200
srv params_from_: Chat format: Qwen3 Coder
slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.982 (> 0.100 thold), f_keep = 0.994
slot launch_slot_: id 3 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id 3 | task 157 | processing task, is_child = 0
slot update_slots: id 3 | task 157 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 11028
slot update_slots: id 3 | task 157 | n_past = 10824, slot.prompt.tokens.size() = 10892, seq_id = 3, pos_min = 10891, n_swa = 1
slot update_slots: id 3 | task 157 | restored context checkpoint (pos_min = 10754, pos_max = 10754, size = 75.376 MiB)
slot update_slots: id 3 | task 157 | n_tokens = 10755, memory_seq_rm [10755, end)
slot update_slots: id 3 | task 157 | prompt processing progress, n_tokens = 10964, batch.n_tokens = 209, progress = 0.994197
slot update_slots: id 3 | task 157 | n_tokens = 10964, memory_seq_rm [10964, end)
slot update_slots: id 3 | task 157 | prompt processing progress, n_tokens = 11028, batch.n_tokens = 64, progress = 1.000000
slot update_slots: id 3 | task 157 | prompt done, n_tokens = 11028, batch.n_tokens = 64
slot init_sampler: id 3 | task 157 | init sampler, took 0.79 ms, tokens: text = 11028, total = 11028
slot update_slots: id 3 | task 157 | created context checkpoint 3 of 8 (pos_min = 10963, pos_max = 10963, size = 75.376 MiB)
srv log_server_r: done request: POST /v1/chat/completions 10.111.3.51 200
srv stop: cancel task, id_task = 157
slot release: id 3 | task 157 | stop processing: n_tokens = 11029, truncated = 0
srv update_slots: all slots are idle
Activity
tim-janik commentedon Feb 6, 2026
Just found #18675 by @pwilkin (via comment), running Qwen3-Coder-Next.MXFP4MOE.gguf in opencode with the Autoparser branch works fine!
crashr commentedon Feb 6, 2026
I got it working with this reverse proxy which I wrote some time ago to connect a streaming client to llama-server when it wasn't able to stream when tool calling was used so it's unlikely an model issue. Maybe it's an issue with streaming?
https://github.com/crashr/llama-stream
nilnor commentedon Feb 6, 2026
I commented on the autoparser PR, but since this appears to be a variant of the error I'm seeing and it's present with the autoparser branch I'll link it: #18675 (comment)
twstagg commentedon Feb 8, 2026
This seems to have helped me with opencode and Qwen3-Coder-Next tool calls quite a bit!
However, I think it breaks FIM? When using it with llama.vim on master, I don't believe I have this issue with infill:
kelvie commentedon Feb 9, 2026
I also noticed that FIM is broken (I use https://github.com/milanglacier/minuet-ai.el) when using the autoparser branch -- does it work with llama-stream?
I filed a bug in the autoparser author's repo: pwilkin#7
pwilkin commentedon Feb 9, 2026
Oh, thanks, I will have to look at that.
crashr commentedon Feb 9, 2026
@kelvie I would like to point out that llama-stream is not necessary if streaming can be disabled on the client side. Disabling it on the client side is definitely preferable, as llama-stream was originally only intended as a quick workaround if this is not possible and is not particularly mature. It may cause more problems than it solves. It is better to use curl and disable streaming.
tim-janik commentedon Feb 12, 2026
Related, no tool calls are working for stepfun-ai_Step-3.5-Flash-IQ3_XXS-00001-of-00002.gguf with recent master (tried pi, opencode). So I rebased autoparser onto master (e06088d) and that got Step-3.5-Flash working as well.
mikicvi commentedon Feb 12, 2026
I can reproduce it, although not as consistently. It starts happening after the first compacting session in opencode. I am runing the same
MXFP4MOEquant. It seems to happen regardless of the temperature used.Hardware setup: RTX A2000 12GB + 12th gen i5 12600h with 96GB of DDR5.
Settings:
Yes, I am aware some of these parameters are not necessary anymore, but I prefer to have it fixed, since defaults change a fair bit...
Opencode tool call output
mudler commentedon Feb 13, 2026
I can confirm Step-flash not working consistently, and as well Qwen3-coder-next failing time to time (will try to collect the logs soon and share). Sorry if not helpful and not providing more context but I don't have access to logs now, but wanted to meanwhile confirm on my end issues others are seeing.
paccerdk commentedon Feb 13, 2026
I get the exact same JSON output problem as @mikicvi using unsloth Qwen3-Coder-Next-UD-Q3_K_XL.gguf with OpenCode when trying to use the Write tool (fails maybe 9 out of 10 times):
I have
--verboseoutput from a working and a broken callLog of fail (causes above error)
Log of success
build: 9a96352 (7989)
The "Parsed message" output looks fine for both, but the streamed data does not - the XML for failed tool calls appear to have duplicate <parameter=filePath> entries, which for some reason seems to result in the invalid JSON
pwilkin commentedon Feb 13, 2026
Yeah, at this point I'd recommend everyone just use the autoparser branch ( #18675 ) until it's done. I'm dedicating most of my free time to finish the work there to make the package maintainable, don't really have the reasources for also fixing the legacy broken parser code currently on main. Sorry for that, I hope that I can have that done sooner than later.
bchtrue commentedon Feb 14, 2026
The same issue. Thank for contribution @pwilkin !
paolomainardi commentedon Feb 16, 2026
I have the very same issue with opencode and
qwen3-coder-next-unsloth-q4kxlfrom unsloth.tomstokes commentedon Feb 20, 2026
Thanks to the hints from @crashr about streaming, I stepped through the streaming diff code until I hit the issue. Qwen3-Coder-Next, or at least the unsloth quants I tried, some times emits duplicate
filePathparameters in the XML for tool calls.The XML parsing logic assumed all keys were unique. When a duplicate key was encountered the value overwrote the entry for the first key instead of appended a new key/value pair. This breaks the streaming diff output because a partial version of the second key may have been streamed already before the final JSON collapsed both keys into a last-writer-wins version.
I have a minimal PR to fix it here: #19753
A native tool call the companion cannot read is asked again, not an e…