Skip to content

Eval bug: Qwen3-Coder-Next produces invalid JSON tool calls #19382

Closed

Description

@tim-janik

Name and Version

$ llama-server --version
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
version: 7951 (22cae83)
built with Clang 21.0.0 for Linux x86_64

Operating systems

Linux

GGML backends

CUDA

Hardware

AMD Ryzen 9 7950X3D 16-Core Processor + NVIDIA GeForce RTX 4090

Models

Qwen3-Coder-Next quantized with b7951 at MXFP4MOE (also tried IQ4_NL, IQ4_XS and Qwen/Qwen3-Coder-Next-Q8_0)

llama.cpp/build/bin/llama-server -c 0 -m Qwen3-Coder-Next.MXFP4MOE.gguf -t 16 \
  --min-p 0.3 -fitt 384 --no-warmup --temp 0.0

Problem description & steps to reproduce

I tried using Qwen3-Coder-Next with opencode for the last 2 days, it always produces the same error when trying to write a file (across the last 3 opencode versions and many llama.cpp versions during the last 2 days).

Any opencode prompt that leads to writing a file will cause the JSON output to be broken at the same place, it always writes a duplicate "filePath" field and messes up the syntax in the latter (misses a colon and a quote).

$ opencode --version
1.1.51
$ opencode --prompt 'Write a C++ "Hello World" program, use dprintf(), compile it with clang++ and run it.'
  ┃
  ┃  Write a C++ "Hello World" program, use dprintf(), compile it with clang++ and run it.
  ┃

     ⚙ invalid [tool=write, error=Invalid input for tool write: JSON parsing failed: Text: {"content":"#include <cstdio>\n\nint main() {\n    dprintf(STDOUT_FILENO, \"Hello World\\n\");\n    return 0;\n}","filePath":"/home/user/hello.cpp","filePath"/home/user/hello.cpp"}.
     Error message: JSON Parse error: Unrecognized token '/']

     ⚙ invalid [tool=write, error=Invalid input for tool write: JSON parsing failed: Text: {"content":"#include <cstdio>\n\nint main() {\n    dprintf(STDOUT_FILENO, \"Hello World\\n\");\n    return 0;\n}","filePath":"/home/user/hello.cpp","filePath"/home/user/hello.cpp"}.
     Error message: JSON Parse error: Unrecognized token '/']

It keeps trying to write the file, but always produces the same error.

Same error with other prompts, it always writes 2 "filePath" fields and messes up the syntax in the latter.

Other models do not show this behavior (e.g. Minimax-M2.1, Qwen3-Coder-30B-A3B-Instruct or gpt-oss-120b work fine).

First Bad Commit

No response

Relevant log output

Logs Log for:

llama-server -c 0 -m /ml/models/Qwen3-Coder-Next.MXFP4MOE.gguf -t 16 --port 1111 --host 0.0.0.0 --min-p 0.3 -fitt 384 --no-warmup --temp 0.0

ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
main: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
build: 7951 (22cae8321) with Clang 21.0.0 for Linux x86_64
system info: n_threads = 16, n_threads_batch = 16, total_threads = 32

system_info: n_threads = 16 (n_threads_batch = 16) / 32 | CUDA : ARCHS = 890 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | REPACK = 1 | 

Running without SSL
init: using 31 threads for HTTP server
start: binding port with default address family
main: loading model
srv    load_model: loading model '/ml/models/Qwen3-Coder-Next.MXFP4MOE.gguf'
common_init_result: fitting params to device memory, for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on
llama_params_fit_impl: projected to use 48643 MiB of device memory vs. 23396 MiB of free device memory
llama_params_fit_impl: cannot meet free memory target of 384 MiB, need to reduce device memory by 25631 MiB
llama_params_fit_impl: user has requested full context size of 262144 -> no change
llama_params_fit_impl: with only dense weights in device memory there is a total surplus of 13281 MiB
llama_params_fit_impl: filling dense-only layers back-to-front:
llama_params_fit_impl:   - CUDA0 (NVIDIA GeForce RTX 4090): 49 layers,  10547 MiB used,  12849 MiB free
llama_params_fit_impl: converting dense-only layers to full layers and filling them front-to-back with overflow to next device/system memory:
llama_params_fit_impl:   - CUDA0 (NVIDIA GeForce RTX 4090): 49 layers (33 overflowing),  22797 MiB used,    599 MiB free
llama_params_fit: successfully fit params to free device memory
llama_params_fit: fitting params to free memory took 1.57 seconds
llama_model_load_from_file_impl: using device CUDA0 (NVIDIA GeForce RTX 4090) (0000:01:00.0) - 23698 MiB free
llama_model_loader: loaded meta data with 41 key-value pairs and 843 tensors from /ml/models/Qwen3-Coder-Next.MXFP4MOE.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = qwen3next
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 40
llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.950000
llama_model_loader: - kv   4:                      general.sampling.temp f32              = 1.000000
llama_model_loader: - kv   5:                         general.size_label str              = 512x2.5B
llama_model_loader: - kv   6:                            general.license str              = apache-2.0
llama_model_loader: - kv   7:                       general.license.link str              = https://huggingface.co/Qwen/Qwen3-Cod...
llama_model_loader: - kv   8:                               general.tags arr[str,1]       = ["text-generation"]
llama_model_loader: - kv   9:                      qwen3next.block_count u32              = 48
llama_model_loader: - kv  10:                   qwen3next.context_length u32              = 262144
llama_model_loader: - kv  11:                 qwen3next.embedding_length u32              = 2048
llama_model_loader: - kv  12:              qwen3next.feed_forward_length u32              = 5120
llama_model_loader: - kv  13:             qwen3next.attention.head_count u32              = 16
llama_model_loader: - kv  14:          qwen3next.attention.head_count_kv u32              = 2
llama_model_loader: - kv  15:                   qwen3next.rope.freq_base f32              = 5000000.000000
llama_model_loader: - kv  16: qwen3next.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  17:                qwen3next.expert_used_count u32              = 10
llama_model_loader: - kv  18:             qwen3next.attention.key_length u32              = 256
llama_model_loader: - kv  19:           qwen3next.attention.value_length u32              = 256
llama_model_loader: - kv  20:                     qwen3next.expert_count u32              = 512
llama_model_loader: - kv  21:       qwen3next.expert_feed_forward_length u32              = 512
llama_model_loader: - kv  22: qwen3next.expert_shared_feed_forward_length u32              = 512
llama_model_loader: - kv  23:                  qwen3next.ssm.conv_kernel u32              = 4
llama_model_loader: - kv  24:                   qwen3next.ssm.state_size u32              = 128
llama_model_loader: - kv  25:                  qwen3next.ssm.group_count u32              = 16
llama_model_loader: - kv  26:               qwen3next.ssm.time_step_rank u32              = 32
llama_model_loader: - kv  27:                   qwen3next.ssm.inner_size u32              = 4096
llama_model_loader: - kv  28:             qwen3next.rope.dimension_count u32              = 64
llama_model_loader: - kv  29:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  30:                         tokenizer.ggml.pre str              = qwen2
llama_model_loader: - kv  31:                      tokenizer.ggml.tokens arr[str,151936]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  32:                  tokenizer.ggml.token_type arr[i32,151936]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  33:                      tokenizer.ggml.merges arr[str,151387]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv  34:                tokenizer.ggml.eos_token_id u32              = 151645
llama_model_loader: - kv  35:            tokenizer.ggml.padding_token_id u32              = 151643
llama_model_loader: - kv  36:                tokenizer.ggml.bos_token_id u32              = 151643
llama_model_loader: - kv  37:               tokenizer.ggml.add_bos_token bool             = false
llama_model_loader: - kv  38:                    tokenizer.chat_template str              = {% macro render_extra_keys(json_dict,...
llama_model_loader: - kv  39:               general.quantization_version u32              = 2
llama_model_loader: - kv  40:                          general.file_type u32              = 38
llama_model_loader: - type  f32:  313 tensors
llama_model_loader: - type q8_0:  338 tensors
llama_model_loader: - type bf16:   48 tensors
llama_model_loader: - type mxfp4:  144 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = MXFP4 MoE
print_info: file size   = 40.73 GiB (4.39 BPW) 
load: 0 unused tokens
load: printing all EOG tokens:
load:   - 151643 ('<|endoftext|>')
load:   - 151645 ('<|im_end|>')
load:   - 151662 ('<|fim_pad|>')
load:   - 151663 ('<|repo_name|>')
load:   - 151664 ('<|file_sep|>')
load: special tokens cache size = 26
load: token to piece cache size = 0.9311 MB
print_info: arch                  = qwen3next
print_info: vocab_only            = 0
print_info: no_alloc              = 0
print_info: n_ctx_train           = 262144
print_info: n_embd                = 2048
print_info: n_embd_inp            = 2048
print_info: n_layer               = 48
print_info: n_head                = 16
print_info: n_head_kv             = 2
print_info: n_rot                 = 64
print_info: n_swa                 = 0
print_info: is_swa_any            = 0
print_info: n_embd_head_k         = 256
print_info: n_embd_head_v         = 256
print_info: n_gqa                 = 8
print_info: n_embd_k_gqa          = 512
print_info: n_embd_v_gqa          = 512
print_info: f_norm_eps            = 0.0e+00
print_info: f_norm_rms_eps        = 1.0e-06
print_info: f_clamp_kqv           = 0.0e+00
print_info: f_max_alibi_bias      = 0.0e+00
print_info: f_logit_scale         = 0.0e+00
print_info: f_attn_scale          = 0.0e+00
print_info: n_ff                  = 5120
print_info: n_expert              = 512
print_info: n_expert_used         = 10
print_info: n_expert_groups       = 0
print_info: n_group_used          = 0
print_info: causal attn           = 1
print_info: pooling type          = 0
print_info: rope type             = 2
print_info: rope scaling          = linear
print_info: freq_base_train       = 5000000.0
print_info: freq_scale_train      = 1
print_info: n_ctx_orig_yarn       = 262144
print_info: rope_yarn_log_mul     = 0.0000
print_info: rope_finetuned        = unknown
print_info: ssm_d_conv            = 4
print_info: ssm_d_inner           = 4096
print_info: ssm_d_state           = 128
print_info: ssm_dt_rank           = 32
print_info: ssm_n_group           = 16
print_info: ssm_dt_b_c_rms        = 0
print_info: model type            = 80B.A3B
print_info: model params          = 79.67 B
print_info: general.name          = n/a
print_info: vocab type            = BPE
print_info: n_vocab               = 151936
print_info: n_merges              = 151387
print_info: BOS token             = 151643 '<|endoftext|>'
print_info: EOS token             = 151645 '<|im_end|>'
print_info: EOT token             = 151645 '<|im_end|>'
print_info: PAD token             = 151643 '<|endoftext|>'
print_info: LF token              = 198 'Ċ'
print_info: FIM PRE token         = 151659 '<|fim_prefix|>'
print_info: FIM SUF token         = 151661 '<|fim_suffix|>'
print_info: FIM MID token         = 151660 '<|fim_middle|>'
print_info: FIM PAD token         = 151662 '<|fim_pad|>'
print_info: FIM REP token         = 151663 '<|repo_name|>'
print_info: FIM SEP token         = 151664 '<|file_sep|>'
print_info: EOG token             = 151643 '<|endoftext|>'
print_info: EOG token             = 151645 '<|im_end|>'
print_info: EOG token             = 151662 '<|fim_pad|>'
print_info: EOG token             = 151663 '<|repo_name|>'
print_info: EOG token             = 151664 '<|file_sep|>'
print_info: max token length      = 256
load_tensors: loading model tensors, this can take a while... (mmap = true, direct_io = false)
load_tensors: offloading output layer to GPU
load_tensors: offloading 47 repeating layers to GPU
load_tensors: offloaded 49/49 layers to GPU
load_tensors:   CPU_Mapped model buffer size = 41393.00 MiB
load_tensors:        CUDA0 model buffer size = 15547.94 MiB
....................................................................................................
common_init_result: added <|endoftext|> logit bias = -inf
common_init_result: added <|im_end|> logit bias = -inf
common_init_result: added <|fim_pad|> logit bias = -inf
common_init_result: added <|repo_name|> logit bias = -inf
common_init_result: added <|file_sep|> logit bias = -inf
llama_context: constructing llama_context
llama_context: n_seq_max     = 4
llama_context: n_ctx         = 262144
llama_context: n_ctx_seq     = 262144
llama_context: n_batch       = 2048
llama_context: n_ubatch      = 512
llama_context: causal_attn   = 1
llama_context: flash_attn    = auto
llama_context: kv_unified    = true
llama_context: freq_base     = 5000000.0
llama_context: freq_scale    = 1
llama_context:  CUDA_Host  output buffer size =     2.32 MiB
llama_kv_cache:      CUDA0 KV buffer size =  6144.00 MiB
llama_kv_cache: size = 6144.00 MiB (262144 cells,  12 layers,  4/1 seqs), K (f16): 3072.00 MiB, V (f16): 3072.00 MiB
llama_memory_recurrent:      CUDA0 RS buffer size =   301.50 MiB
llama_memory_recurrent: size =  301.50 MiB (     4 cells,  48 layers,  4 seqs), R (f32):   13.50 MiB, S (f32):  288.00 MiB
sched_reserve: reserving ...
sched_reserve: Flash Attention was auto, set to enabled
sched_reserve:      CUDA0 compute buffer size =   804.06 MiB
sched_reserve:  CUDA_Host compute buffer size =   520.01 MiB
sched_reserve: graph nodes  = 9266 (with bs=512), 5918 (with bs=1)
sched_reserve: graph splits = 101 (with bs=512), 72 (with bs=1)
sched_reserve: reserve took 208.51 ms, sched copies = 1
srv    load_model: initializing slots, n_slots = 4
no implementations specified for speculative decoding
slot   load_model: id  0 | task -1 | speculative decoding context not initialized
slot   load_model: id  0 | task -1 | new slot, n_ctx = 262144
no implementations specified for speculative decoding
slot   load_model: id  1 | task -1 | speculative decoding context not initialized
slot   load_model: id  1 | task -1 | new slot, n_ctx = 262144
no implementations specified for speculative decoding
slot   load_model: id  2 | task -1 | speculative decoding context not initialized
slot   load_model: id  2 | task -1 | new slot, n_ctx = 262144
no implementations specified for speculative decoding
slot   load_model: id  3 | task -1 | speculative decoding context not initialized
slot   load_model: id  3 | task -1 | new slot, n_ctx = 262144
srv    load_model: prompt cache is enabled, size limit: 8192 MiB
srv    load_model: use `--cache-ram 0` to disable the prompt cache
srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
'
srv          init: init: chat template, thinking = 0
main: model loaded
main: server is listening on http://0.0.0.0:1111
main: starting the main loop...
srv  update_slots: all slots are idle
srv  params_from_: Chat format: Qwen3 Coder
slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
slot launch_slot_: id  3 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist 
slot launch_slot_: id  3 | task 0 | processing task, is_child = 0
slot update_slots: id  3 | task 0 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 10610
slot update_slots: id  3 | task 0 | n_tokens = 0, memory_seq_rm [0, end)
slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 2048, batch.n_tokens = 2048, progress = 0.193025
slot update_slots: id  3 | task 0 | n_tokens = 2048, memory_seq_rm [2048, end)
slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 4096, batch.n_tokens = 2048, progress = 0.386051
slot update_slots: id  3 | task 0 | n_tokens = 4096, memory_seq_rm [4096, end)
slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 6144, batch.n_tokens = 2048, progress = 0.579076
slot update_slots: id  3 | task 0 | n_tokens = 6144, memory_seq_rm [6144, end)
slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 8192, batch.n_tokens = 2048, progress = 0.772102
slot update_slots: id  3 | task 0 | n_tokens = 8192, memory_seq_rm [8192, end)
slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 10240, batch.n_tokens = 2048, progress = 0.965127
slot update_slots: id  3 | task 0 | n_tokens = 10240, memory_seq_rm [10240, end)
slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 10546, batch.n_tokens = 306, progress = 0.993968
slot update_slots: id  3 | task 0 | n_tokens = 10546, memory_seq_rm [10546, end)
slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 10610, batch.n_tokens = 64, progress = 1.000000
slot update_slots: id  3 | task 0 | prompt done, n_tokens = 10610, batch.n_tokens = 64
slot init_sampler: id  3 | task 0 | init sampler, took 0.77 ms, tokens: text = 10610, total = 10610
slot update_slots: id  3 | task 0 | created context checkpoint 1 of 8 (pos_min = 10545, pos_max = 10545, size = 75.376 MiB)
slot print_timing: id  3 | task 0 | 
prompt eval time =   23588.03 ms / 10610 tokens (    2.22 ms per token,   449.80 tokens per second)
       eval time =    3170.93 ms /    74 tokens (   42.85 ms per token,    23.34 tokens per second)
      total time =   26758.96 ms / 10684 tokens
slot      release: id  3 | task 0 | stop processing: n_tokens = 10683, truncated = 0
srv  update_slots: all slots are idle
srv  log_server_r: done request: POST /v1/chat/completions 10.111.3.51 200
srv  params_from_: Chat format: Qwen3 Coder
slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.981 (> 0.100 thold), f_keep = 0.994
slot launch_slot_: id  3 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist 
slot launch_slot_: id  3 | task 81 | processing task, is_child = 0
slot update_slots: id  3 | task 81 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 10819
slot update_slots: id  3 | task 81 | n_past = 10615, slot.prompt.tokens.size() = 10683, seq_id = 3, pos_min = 10682, n_swa = 1
slot update_slots: id  3 | task 81 | restored context checkpoint (pos_min = 10545, pos_max = 10545, size = 75.376 MiB)
slot update_slots: id  3 | task 81 | n_tokens = 10546, memory_seq_rm [10546, end)
slot update_slots: id  3 | task 81 | prompt processing progress, n_tokens = 10755, batch.n_tokens = 209, progress = 0.994084
slot update_slots: id  3 | task 81 | n_tokens = 10755, memory_seq_rm [10755, end)
slot update_slots: id  3 | task 81 | prompt processing progress, n_tokens = 10819, batch.n_tokens = 64, progress = 1.000000
slot update_slots: id  3 | task 81 | prompt done, n_tokens = 10819, batch.n_tokens = 64
slot init_sampler: id  3 | task 81 | init sampler, took 0.79 ms, tokens: text = 10819, total = 10819
slot update_slots: id  3 | task 81 | created context checkpoint 2 of 8 (pos_min = 10754, pos_max = 10754, size = 75.376 MiB)
slot print_timing: id  3 | task 81 | 
prompt eval time =    1717.10 ms /   273 tokens (    6.29 ms per token,   158.99 tokens per second)
       eval time =    3111.03 ms /    74 tokens (   42.04 ms per token,    23.79 tokens per second)
      total time =    4828.13 ms /   347 tokens
slot      release: id  3 | task 81 | stop processing: n_tokens = 10892, truncated = 0
srv  update_slots: all slots are idle
srv  log_server_r: done request: POST /v1/chat/completions 10.111.3.51 200
srv  params_from_: Chat format: Qwen3 Coder
slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.982 (> 0.100 thold), f_keep = 0.994
slot launch_slot_: id  3 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist 
slot launch_slot_: id  3 | task 157 | processing task, is_child = 0
slot update_slots: id  3 | task 157 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 11028
slot update_slots: id  3 | task 157 | n_past = 10824, slot.prompt.tokens.size() = 10892, seq_id = 3, pos_min = 10891, n_swa = 1
slot update_slots: id  3 | task 157 | restored context checkpoint (pos_min = 10754, pos_max = 10754, size = 75.376 MiB)
slot update_slots: id  3 | task 157 | n_tokens = 10755, memory_seq_rm [10755, end)
slot update_slots: id  3 | task 157 | prompt processing progress, n_tokens = 10964, batch.n_tokens = 209, progress = 0.994197
slot update_slots: id  3 | task 157 | n_tokens = 10964, memory_seq_rm [10964, end)
slot update_slots: id  3 | task 157 | prompt processing progress, n_tokens = 11028, batch.n_tokens = 64, progress = 1.000000
slot update_slots: id  3 | task 157 | prompt done, n_tokens = 11028, batch.n_tokens = 64
slot init_sampler: id  3 | task 157 | init sampler, took 0.79 ms, tokens: text = 11028, total = 11028
slot update_slots: id  3 | task 157 | created context checkpoint 3 of 8 (pos_min = 10963, pos_max = 10963, size = 75.376 MiB)
srv  log_server_r: done request: POST /v1/chat/completions 10.111.3.51 200
srv          stop: cancel task, id_task = 157
slot      release: id  3 | task 157 | stop processing: n_tokens = 11029, truncated = 0
srv  update_slots: all slots are idle

Activity

  1. tim-janik commented on Feb 6, 2026

    @tim-janik
    Author

    Just found #18675 by @pwilkin (via comment), running Qwen3-Coder-Next.MXFP4MOE.gguf in opencode with the Autoparser branch works fine!

    $ llama-server -c 0 -m /ml/models/Qwen3-Coder-Next.MXFP4MOE.gguf -t 16 --min-p 0.3 -fitt 384 --no-warmup --temp 0.0                                                 
    $ opencode --prompt 'Write a C++ "Hello World" program, use dprintf(), compile it with clang++ and run it.'
    
      ┃
      ┃  # C++ Hello World with dprintf, clang++ compile/run                                                                                                                                                                                                         10,980 ($0.00) v1.1.51
      ┃
    
      ┃
      ┃  Write a C++ "Hello World" program, use dprintf(), compile it with clang++ and run it.
      ┃
    
      ┃
      ┃  # Wrote hello.cpp
      ┃
      ┃   1 #include <cstdio>
      ┃   2
      ┃   3 int main() {
      ┃   4     dprintf(STDOUT_FILENO, "Hello World\n");
      ┃   5     return 0;
      ┃   6 }
      ┃   7
      ┃
      ┃  Error [3:12]: Use of undeclared identifier 'STDOUT_FILENO'
      ┃
    
      ┃
      ┃  # Compile and run Hello World program
      ┃
      ┃  $ clang++ -o hello hello.cpp && ./hello
      ┃
      ┃  hello.cpp:4:13: error: use of undeclared identifier 'STDOUT_FILENO'
      ┃      4 |     dprintf(STDOUT_FILENO, "Hello World\n");
      ┃        |             ^
      ┃  1 error generated.
      ┃
    
      ┃
      ┃  ← Edit hello.cpp
      ┃
      ┃    1 #include <cstdio>                                                                                                                      1   #include <cstdio>
      ┃                                                                                                                                             2 + #include <unistd.h>
      ┃    2                                                                                                                                        3
      ┃    3 int main() {                                                                                                                           4   int main() {
      ┃    4     dprintf(STDOUT_FILENO, "Hello World\n");                                                                                           5       dprintf(STDOUT_FILENO, "Hello World\n");
      ┃    5     return 0;                                                                                                                          6       return 0;
      ┃
    
      ┃
      ┃  # Compile and run Hello World program
      ┃
      ┃  $ clang++ -o hello hello.cpp && ./hello
      ┃
      ┃  Hello World
      ┃
    
         Done. The program compiles and runs successfully, outputting "Hello World".
    
         ▣  Build · local · 12.2s
    
  2. crashr commented on Feb 6, 2026

    @crashr
    Contributor

    I got it working with this reverse proxy which I wrote some time ago to connect a streaming client to llama-server when it wasn't able to stream when tool calling was used so it's unlikely an model issue. Maybe it's an issue with streaming?
    https://github.com/crashr/llama-stream

  3. nilnor commented on Feb 6, 2026

    @nilnor

    I commented on the autoparser PR, but since this appears to be a variant of the error I'm seeing and it's present with the autoparser branch I'll link it: #18675 (comment)

  4. twstagg commented on Feb 8, 2026

    @twstagg

    This seems to have helped me with opencode and Qwen3-Coder-Next tool calls quite a bit!

    However, I think it breaks FIM? When using it with llama.vim on master, I don't believe I have this issue with infill:

    llamacpp[2726228]: [48047] srv    operator(): got exception: {"error":{"code":500,"message":"Failed to parse due to missing parser definition.","type":"server_error"}}
    
  5. kelvie commented on Feb 9, 2026

    @kelvie

    I also noticed that FIM is broken (I use https://github.com/milanglacier/minuet-ai.el) when using the autoparser branch -- does it work with llama-stream?

    I filed a bug in the autoparser author's repo: pwilkin#7

  6. pwilkin commented on Feb 9, 2026

    @pwilkin
    Member

    Oh, thanks, I will have to look at that.

  7. crashr commented on Feb 9, 2026

    @crashr
    Contributor

    @kelvie I would like to point out that llama-stream is not necessary if streaming can be disabled on the client side. Disabling it on the client side is definitely preferable, as llama-stream was originally only intended as a quick workaround if this is not possible and is not particularly mature. It may cause more problems than it solves. It is better to use curl and disable streaming.

  8. tim-janik commented on Feb 12, 2026

    @tim-janik
    Author

    Oh, thanks, I will have to look at that.

    Related, no tool calls are working for stepfun-ai_Step-3.5-Flash-IQ3_XXS-00001-of-00002.gguf with recent master (tried pi, opencode). So I rebased autoparser onto master (e06088d) and that got Step-3.5-Flash working as well.

  9. mikicvi commented on Feb 12, 2026

    @mikicvi

    I can reproduce it, although not as consistently. It starts happening after the first compacting session in opencode. I am runing the same MXFP4MOE quant. It seems to happen regardless of the temperature used.
    Hardware setup: RTX A2000 12GB + 12th gen i5 12600h with 96GB of DDR5.

    Settings:

                llama-server
                --model /models/Qwen3-Coder-Next-MXFP4_MOE.gguf
                --host 127.0.0.1
                --port ${PORT}
                --ctx-size 95000
                --flash-attn on
                --mmap
                --no-direct-io
                --no-warmup
                --jinja
                --threads 10
                --temp 1.0
                --top-k 40
                --top-p 0.95
                --min-p 0.01
                -b 4096
                -ub 4096
    

    Yes, I am aware some of these parameters are not necessary anymore, but I prefer to have it fixed, since defaults change a fair bit...

    Opencode tool call output
    Let me re-read the actual code behavior and rewrite the tests correctly:
    ⚙ invalid [tool=write, error=Invalid input for tool write: JSON parsing failed: Text: {"content":"const axios = require('axios');\nconst fs = require('fs-extra');\nconst path = require('path');\nconst { downloadVideo } = require('../src/downloader');\n\njest.mock('axios');\njest.mock('fs-extra');\n\ndescribe('downloadVideo', () => {\n    const url = 'http://example.com/video';\n    const outputDir = '/path/to/output';\n    const fileName = 'video';\n    const container_extension = 'mp4';\n    const maxRetries = 3;\n\n    beforeEach(() => {\n        jest.clearAllMocks();\n    });\n\n    it('should download the video successfully', async () => {\n        const filePath = path.join(outputDir, `${fileName}.${container_extension}`);\n        const response = {\n            data: {\n                pipe: jest.fn(),\n                on: jest.fn(),\n            },\n            headers: {\n                'content-length': 1024 * 1024 * 10,\n            },\n        };\n        const writer = {\n            on: jest.fn((event, cb) => {\n                if (event === 'finish') cb();\n            }),\n            finish: jest.fn(),\n            error: jest.fn(),\n        };\n\n        axios.mockResolvedValue(response);\n        fs.existsSync.mockReturnValue(false);\n        fs.createWriteStream.mockReturnValue(writer);\n        fs.statSync.mockReturnValue({ size: 0 });\n\n        const result = await downloadVideo(url, outputDir, fileName, container_extension, maxRetries);\n\n        expect(axios).toHaveBeenCalledWith({\n            url,\n            method: 'GET',\n            responseType: 'stream',\n            maxRedirects: 5,\n            headers: {\n                Range: 'bytes=0-',\n            },\n        });\n        expect(fs.existsSync).toHaveBeenCalledWith(filePath);\n        expect(fs.createWriteStream).toHaveBeenCalledWith(filePath, { flags: 'a' });\n        expect(response.data.pipe).toHaveBeenCalledWith(writer);\n        expect(result).toBe(filePath);\n    }, 10000);\n\n    it('should retry downloading the video when an error occurs', async () => {\n        const filePath = path.join(outputDir, `${fileName}.${container_extension}`);\n        const response = {\n            data: {\n                pipe: jest.fn(),\n                on: jest.fn(),\n            },\n            headers: {\n                'content-length': 1024 * 1024 * 10,\n            },\n        };\n        const writer = {\n            on: jest.fn((event, cb) => {\n                if (event === 'finish') cb();\n            }),\n            finish: jest.fn(),\n            error: jest.fn(),\n        };\n\n        axios.mockRejectedValueOnce(new Error('Network error'));\n        axios.mockResolvedValue(response);\n        fs.existsSync.mockImplementation((p) => p === filePath);\n        fs.statSync.mockReturnValue({ size: 1024 * 1024 });\n        fs.createWriteStream.mockReturnValue(writer);\n\n        const result = await downloadVideo(url, outputDir, fileName, container_extension, maxRetries);\n\n        expect(axios).toHaveBeenCalledTimes(2);\n        expect(fs.existsSync).toHaveBeenCalledWith(filePath);\n        expect(fs.createWriteStream).toHaveBeenCalledWith(filePath, { flags: 'a' });\n        expect(result).toBe(filePath);\n    }, 15000);\n\n    it('should throw an error when max retries exceeded', async () => {\n        axios.mockRejectedValue(new Error('Network error'));\n        fs.existsSync.mockReturnValue(false);\n\n        await expect(downloadVideo(url, outputDir, fileName, container_extension, 2)).rejects.toThrowError(\n            'Failed to download video after 2 attempts',\n        );\n\n        expect(axios).toHaveBeenCalledTimes(2);\n        expect(fs.existsSync).toHaveBeenCalledWith(path.join(outputDir, `${fileName}.${container_extension}`));\n    });\n\n    it('should return filePath when file already exists', async () => {\n        const filePath = path.join(outputDir, `${fileName}.${container_extension}`);\n        fs.existsSync.mockReturnValue(true);\n        fs.statSync.mockReturnValue({ size: 1024 });\n\n        const result = await downloadVideo(url, outputDir, fileName, container_extension, maxRetries);\n\n        expect(result).toBe(filePath);\n        expect(fs.existsSync).toHaveBeenCalledWith(filePath);\n        expect(fs.statSync).toHaveBeenCalledWith(filePath);\n    });\n});","filePath":"/Users/mikicv/Documents/megaripper/tests/downloader.test.js","filePath"/Users/mikicv/Documents/megaripper/tests/downloader.test.js"}.
    Error message: JSON Parse error: Unrecognized token '/']
    ⚙ invalid [tool=write, error=Invalid input for tool write: JSON parsing failed: Text: {"content":"const axios = require('axios');\nconst fs = require('fs-extra');\nconst path = require('path');\nconst { downloadVideo } = require('../src/downloader');\n\njest.mock('axios');\njest.mock('fs-extra');\n\ndescribe('downloadVideo', () => {\n    const url = 'http://example.com/video';\n    const outputDir = '/path/to/output';\n    const fileName = 'video';\n    const container_extension = 'mp4';\n    const maxRetries = 3;\n\n    beforeEach(() => {\n        jest.clearAllMocks();\n    });\n\n    it('should download the video successfully', async () => {\n        const filePath = path.join(outputDir, `${fileName}.${container_extension}`);\n        const response = {\n            data: {\n                pipe: jest.fn(),\n                on: jest.fn(),\n            },\n            headers: {\n                'content-length': 1024 * 1024 * 10,\n            },\n        };\n        const writer = {\n            on: jest.fn((event, cb) => {\n                if (event === 'finish') cb();\n            }),\n            finish: jest.fn(),\n            error: jest.fn(),\n        };\n\n        axios.mockResolvedValue(response);\n        fs.existsSync.mockReturnValue(false);\n        fs.createWriteStream.mockReturnValue(writer);\n        fs.statSync.mockReturnValue({ size: 0 });\n\n        const result = await downloadVideo(url, outputDir, fileName, container_extension, maxRetries);\n\n        expect(axios).toHaveBeenCalledWith({\n            url,\n            method: 'GET',\n            responseType: 'stream',\n            maxRedirects: 5,\n            headers: {\n                Range: 'bytes=0-',\n            },\n        });\n        expect(fs.existsSync).toHaveBeenCalledWith(filePath);\n        expect(fs.createWriteStream).toHaveBeenCalledWith(filePath, { flags: 'a' });\n        expect(response.data.pipe).toHaveBeenCalledWith(writer);\n        expect(result).toBe(filePath);\n    }, 10000);\n\n    it('should retry downloading the video when an error occurs', async () => {\n        const filePath = path.join(outputDir, `${fileName}.${container_extension}`);\n        const response = {\n            data: {\n                pipe: jest.fn(),\n                on: jest.fn(),\n            },\n            headers: {\n                'content-length': 1024 * 1024 * 10,\n            },\n        };\n        const writer = {\n            on: jest.fn((event, cb) => {\n                if (event === 'finish') cb();\n            }),\n            finish: jest.fn(),\n            error: jest.fn(),\n        };\n\n        axios.mockRejectedValueOnce(new Error('Network error'));\n        axios.mockResolvedValue(response);\n        fs.existsSync.mockImplementation((p) => p === filePath);\n        fs.statSync.mockReturnValue({ size: 1024 * 1024 });\n        fs.createWriteStream.mockReturnValue(writer);\n\n        const result = await downloadVideo(url, outputDir, fileName, container_extension, maxRetries);\n\n        expect(axios).toHaveBeenCalledTimes(2);\n        expect(fs.existsSync).toHaveBeenCalledWith(filePath);\n        expect(fs.createWriteStream).toHaveBeenCalledWith(filePath, { flags: 'a' });\n        expect(result).toBe(filePath);\n    }, 15000);\n\n    it('should throw an error when max retries exceeded', async () => {\n        axios.mockRejectedValue(new Error('Network error'));\n        fs.existsSync.mockReturnValue(false);\n\n        await expect(downloadVideo(url, outputDir, fileName, container_extension, 2)).rejects.toThrowError(\n            'Failed to download video after 2 attempts',\n        );\n\n        expect(axios).toHaveBeenCalledTimes(2);\n        expect(fs.existsSync).toHaveBeenCalledWith(path.join(outputDir, `${fileName}.${container_extension}`));\n    });\n\n    it('should return filePath when file already exists', async () => {\n        const filePath = path.join(outputDir, `${fileName}.${container_extension}`);\n        fs.existsSync.mockReturnValue(true);\n        fs.statSync.mockReturnValue({ size: 1024 });\n\n        const result = await downloadVideo(url, outputDir, fileName, container_extension, maxRetries);\n\n        expect(result).toBe(filePath);\n        expect(fs.existsSync).toHaveBeenCalledWith(filePath);\n        expect(fs.statSync).toHaveBeenCalledWith(filePath);\n    });\n});","filePath":"/Users/mikicv/Documents/megaripper/tests/downloader.test.js","filePath"/Users/mikicv/Documents/megaripper/tests/downloader.test.js"}.
    Error message: JSON Parse error: Unrecognized token '/']
    Let me use the write tool properly:
    
  10. mudler commented on Feb 13, 2026

    @mudler
    Contributor

    Oh, thanks, I will have to look at that.

    Related, no tool calls are working for stepfun-ai_Step-3.5-Flash-IQ3_XXS-00001-of-00002.gguf with recent master (tried pi, opencode). So I rebased autoparser onto master (e06088d) and that got Step-3.5-Flash working as well.

    I can confirm Step-flash not working consistently, and as well Qwen3-coder-next failing time to time (will try to collect the logs soon and share). Sorry if not helpful and not providing more context but I don't have access to logs now, but wanted to meanwhile confirm on my end issues others are seeing.

  11. paccerdk commented on Feb 13, 2026

    @paccerdk

    I get the exact same JSON output problem as @mikicvi using unsloth Qwen3-Coder-Next-UD-Q3_K_XL.gguf with OpenCode when trying to use the Write tool (fails maybe 9 out of 10 times):

    → Read src/helloworld.cpp 
    ⚙ invalid [tool=write, error=Invalid input for tool write: JSON parsing failed: Text: {"content":"#include <iostream>\n\nint main() {\n    std::cout << \"Hello World!\";\n    std::cout << \"foo\";\n    return 0;\n}","filePath":"/data/dev/llmassist/src/helloworld.cpp","filePath"/data/dev/llmassist/src/helloworld.cpp"}.
    

    I have --verbose output from a working and a broken call

    Log of fail (causes above error)
    [52341] res          send: sending result for task id = 47
    [52341] res          send: task id = 47 pushed to result queue
    [52341] slot      release: id  0 | task 47 | stop processing: n_tokens = 12335, truncated = 0
    [52341] slot        reset: id  0 | task 47 | 
    [52341] srv  update_slots: run slots completed
    [52341] que    start_loop: waiting for new tasks
    [52341] que    start_loop: processing new tasks
    [52341] que    start_loop: processing task, id = 136
    [52341] que    start_loop: update slots
    [52341] srv  update_slots: all slots are idle
    [52341] que    start_loop: waiting for new tasks
    [52341] srv  update_chat_: Parsing chat message: <tool_call>
    [52341] <function=write>
    [52341] <parameter=content>
    [52341] #include <iostream>
    [52341] 
    [52341] int main() {
    [52341]     std::cout << "Hello World!";
    [52341]     std::cout << "foo";
    [52341]     return 0;
    [52341] }
    [52341] 
    [52341] </parameter>
    [52341] <parameter=filePath>
    [52341] /data/dev/llmassist/src/helloworld.cpp
    [52341] </parameter>
    [52341] <parameter=filePath>
    [52341] /data/dev/llmassist/src/helloworld.cpp
    [52341] </parameter>
    [52341] </function>
    [52341] </tool_call>
    [52341] Parsing input with format Qwen3 Coder: <tool_call>
    [52341] <function=write>
    [52341] <parameter=content>
    [52341] #include <iostream>
    [52341] 
    [52341] int main() {
    [52341]     std::cout << "Hello World!";
    [52341]     std::cout << "foo";
    [52341]     return 0;
    [52341] }
    [52341] 
    [52341] </parameter>
    [52341] <parameter=filePath>
    [52341] /data/dev/llmassist/src/helloworld.cpp
    [52341] </parameter>
    [52341] <parameter=filePath>
    [52341] /data/dev/llmassist/src/helloworld.cpp
    [52341] </parameter>
    [52341] </function>
    [52341] </tool_call>
    [52341] Matched tool start: "<tool_call>\n<function="
    [52341] Failed to parse up to error: [json.exception.parse_error.101] parse error at line 2, column 1: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal: <
    [52341] >>>
    [52341] Failed to parse up to error: [json.exception.parse_error.101] parse error at line 2, column 1: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal: <
    [52341] >>>
    [52341] Failed to parse up to error: [json.exception.parse_error.101] parse error at line 2, column 1: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal: <
    [52341] >>>
    [52341] Parsed message: {"role":"assistant","content":"","tool_calls":[{"type":"function","function":{"name":"write","arguments":"{\"content\":\"#include <iostream>\\n\\nint main() {\\n    std::cout << \\\"Hello World!\\\";\\n    std::cout << \\\"foo\\\";\\n    return 0;\\n}\",\"filePath\":\"/data/dev/llmassist/src/helloworld.cpp\"}"}}]}
    
    Streamed deltas:
    "delta":{"role":"assistant","content":null}
    "delta":{"tool_calls":[{"index":0,"id":"LhwU4FrXs2uqmNYxljdyAsw30P3r38nf","type":"function","function":{"name":"write","arguments":"{"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"content"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\":"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"#include"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" <iostream"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":">"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\\n\\nint"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" main"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"()"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" {"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\\n    std"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"::"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"cout"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" <"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"< \\"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"Hello"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" World"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"!\\\";"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\\n    std"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"::"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"cout"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" <"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"< \\"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"foo"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\\\";"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\\n    return"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" 0"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":";"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\\n}"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\""}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":","}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"filePath"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\":"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"/data"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"/dev"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"/"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"ll"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"mass"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"ist"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"/src"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"/h"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"elloworld"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":".cpp"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\""}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":","}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"filePath"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"/data"}}]}        <-- Incorrect
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"/dev"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"/"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"ll"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"mass"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"ist"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"/src"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"/h"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"elloworld"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":".cpp"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\""}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"}"}}]}
    
    Log of success
    [52341] res          send: sending result for task id = 608
    [52341] res          send: task id = 608 pushed to result queue
    [52341] slot      release: id  0 | task 608 | stop processing: n_tokens = 13696, truncated = 0
    [52341] slot        reset: id  0 | task 608 | 
    [52341] srv  update_slots: run slots completed
    [52341] que    start_loop: waiting for new tasks
    [52341] que    start_loop: processing new tasks
    [52341] que    start_loop: processing task, id = 686
    [52341] srv  update_chat_: Parsing chat message: <tool_call>
    [52341] <function=write>
    [52341] <parameter=content>
    [52341] #include <iostream>
    [52341] 
    [52341] int main() {
    [52341]     std::cout << "Hello World!";
    [52341]     std::cout << "foo";
    [52341]     return 0;
    [52341] }
    [52341] </parameter>
    [52341] <parameter=filePath>
    [52341] /data/dev/llmassist/src/helloworld.cpp
    [52341] </parameter>
    [52341] </function>
    [52341] </tool_call>
    [52341] Parsing input with format Qwen3 Coder: <tool_call>
    [52341] <function=write>
    [52341] <parameter=content>
    [52341] #include <iostream>
    [52341] 
    [52341] int main() {
    [52341]     std::cout << "Hello World!";
    [52341]     std::cout << "foo";
    [52341]     return 0;
    [52341] }
    [52341] </parameter>
    [52341] <parameter=filePath>
    [52341] /data/dev/llmassist/src/helloworld.cpp
    [52341] </parameter>
    [52341] </function>
    [52341] </tool_call>
    [52341] Matched tool start: "<tool_call>\n<function="
    [52341] que    start_loop: update slots
    [52341] srv  update_slots: all slots are idle
    [52341] que    start_loop: waiting for new tasks
    [52341] Failed to parse up to error: [json.exception.parse_error.101] parse error at line 2, column 1: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal: <
    [52341] >>>
    [52341] Failed to parse up to error: [json.exception.parse_error.101] parse error at line 2, column 1: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal: <
    [52341] >>>
    [52341] Parsed message: {"role":"assistant","content":"","tool_calls":[{"type":"function","function":{"name":"write","arguments":"{\"content\":\"#include <iostream>\\n\\nint main() {\\n    std::cout << \\\"Hello World!\\\";\\n    std::cout << \\\"foo\\\";\\n    return 0;\\n}\",\"filePath\":\"/data/dev/llmassist/src/helloworld.cpp\"}"}}]}
    
    Streamed deltas:
    "delta":{"role":"assistant","content":null}
    "delta":{"tool_calls":[{"index":0,"id":"TMzSJRz7wlIbZThrjTFMdtnbYjI0apjd","type":"function","function":{"name":"write","arguments":"{"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"content"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\":"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"#include"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" <iostream"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":">"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\\n\\nint"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" main"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"()"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" {"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\\n    std"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"::"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"cout"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" <"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"< \\"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"Hello"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" World"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"!\\\";"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\\n    std"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"::"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"cout"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" <"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"< \\"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"foo"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\\\";"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\\n    return"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":" 0"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":";"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\\n}"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\""}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":","}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"filePath"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\":"}}]}                 <-- Correct
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"/data"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"/dev"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"/"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"ll"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"mass"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"ist"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"/src"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"/h"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"elloworld"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":".cpp"}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"\""}}]}
    "delta":{"tool_calls":[{"index":0,"function":{"arguments":"}"}}]}
    

    build: 9a96352 (7989)

    The "Parsed message" output looks fine for both, but the streamed data does not - the XML for failed tool calls appear to have duplicate <parameter=filePath> entries, which for some reason seems to result in the invalid JSON

  12. pwilkin commented on Feb 13, 2026

    @pwilkin
    Member

    Yeah, at this point I'd recommend everyone just use the autoparser branch ( #18675 ) until it's done. I'm dedicating most of my free time to finish the work there to make the package maintainable, don't really have the reasources for also fixing the legacy broken parser code currently on main. Sorry for that, I hope that I can have that done sooner than later.

  13. bchtrue commented on Feb 14, 2026

    @bchtrue

    The same issue. Thank for contribution @pwilkin !

  14. paolomainardi commented on Feb 16, 2026

    @paolomainardi

    I have the very same issue with opencode and qwen3-coder-next-unsloth-q4kxl from unsloth.

  15. tomstokes commented on Feb 20, 2026

    @tomstokes

    Thanks to the hints from @crashr about streaming, I stepped through the streaming diff code until I hit the issue. Qwen3-Coder-Next, or at least the unsloth quants I tried, some times emits duplicate filePath parameters in the XML for tool calls.

    The XML parsing logic assumed all keys were unique. When a duplicate key was encountered the value overwrote the entry for the first key instead of appended a new key/value pair. This breaks the streaming diff output because a partial version of the second key may have been streamed already before the final JSON collapsed both keys into a last-writer-wins version.

    I have a minimal PR to fix it here: #19753

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions