Repository navigation
server: thinking type rejected as invalid but used by Ministral 3 #17700
Description
EDIT: Apologies for marking this as a bug, probably should have marked as an enhancement - feel free to re-tag.
Name and Version
❯ ./llama.cpp/build/bin/llama-server --version
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 5090, compute capability 12.0, VMM: yes
version: 7236 (a2b0fe8)
built with cc (Gentoo 13.4.1_p20250807 p8) 13.4.1 20250807 for x86_64-pc-linux-gnu
Operating systems
Linux
Which llama.cpp modules do you know to be affected?
llama-server
Command line
llama-server --port 8000 -fa on -c 8192 -ngl 99 -cram -1 -m /models/Ministral-3-14B-Reasoning-2512-UD-Q6_K_XL.ggufProblem description & steps to reproduce
The new Ministral 3 reasoning models use a (I believe non-standard) content[].type of thinking which is used to pass the reasoning history traces back to the model:
https://huggingface.co/mistralai/Ministral-3-14B-Reasoning-2512/blob/main/chat_template.jinja#L94
However, llama-server currently rejects this with an error:
srv operator(): got exception: {"error":{"code":500,"message":"unsupported content[].type","type":"server_error"}}
Is this something that should/could be fixed on the llama-server side? Or is there a more standardized way of passing reasoning traces back in that e.g. we should encourage people converting the Magistral 3 GGUFs to add into the chat template?
First Bad Commit
N/A, not a regression
Activity
eauchs commentedon Dec 2, 2025
eauchs commentedon Dec 2, 2025
Yes, this is a limitation in the current OAI compatibility parsing in llama-server. It explicitly throws an exception on content types other than text or image_url.
To support this properly in llama.cpp, we need to patch the JSON parsing logic (likely in server.cpp or associated OAI utils) to accept type: "thinking". It should be handled effectively as text content so the reasoning trace passes through to the model's chat template without error.
Workaround: For now, you can modify your client/middleware to rename thinking to text in the messages content array before sending the request to the server.
isaac-mcfadyen commentedon Dec 2, 2025
The issue with this is that it won't insert the reasoning trace properly unless the reasoning tags are manually added to the text, correct?
E.g the chat template adds thinking content between
[THINK]and[/THINK]but if the client sends it as part of atextcontent block it won't be put between those tags.eauchs commentedon Dec 2, 2025
Correct.
For the client-side workaround: You would indeed need to manually wrap the content in the specific tags (e.g., [THINK]...[/THINK]) and send it as text.
For the actual server-side fix: Simply treating it as text internally would likely fail as you described. The llama-server implementation needs to accept the thinking type during parsing and pass it effectively to the templating engine as a distinct type, so that the Jinja template logic (which checks if message['type'] == 'thinking') can trigger and insert the correct tags automatically.
aldehir commentedon Dec 2, 2025
Other models accept reasoning traces back in the
reasoning_contentfield and map it internally to the appropriate field in the template. I don't think this should be any different.isaac-mcfadyen commentedon Dec 2, 2025
CC @danielhanchen (hope you're OK with the ping!) as someone who's created GGUFs for these models.
I know you've done fixes for chat templates of other models in the past, is this something that could be considered for Ministral 3?
aldehir commentedon Dec 4, 2025
My PR #17713 formats the template with traces sent back in
reasoning_content. Alternatively, you can pass--reasoning-format noneas an argument or"reasoning_format": "none"in the request to keep the traces incontent. Although, some clients may strip the tags out and not include them in subsequent requests.