You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 601f5f7
Browse filesBrowse the repository at this point in the historyBrowse files
chat_template_kwargs is a vLLM/SGLang extension. OpenAI, Azure, and strict
OpenAI-compatible gateways reject the unknown body field with HTTP 400, and
non-Qwen vLLM models served with it can emit <think> output with no <answer>
tag (acc=0.000). c31c50b fixed that by only emitting the field when thinking
was enabled, which closed#28 but left no supported way to send an explicit
enable_thinking: false -- the request in #90/#109.
The protocol has three states, so make the setting three-state:
server_default (default) -> omit chat_template_kwargs
enabled -> send enable_thinking: true
disabled -> send enable_thinking: false
server_default keeps every existing deployment on exactly the bytes it sends
today, so #28 stays fixed, while disabled gives #90 the explicit false it asks
for. The legacy enable_thinking boolean keeps its historical wire meaning
(true -> send true, false -> omit), so no config changes behavior; setting
both keys to conflicting values raises rather than silently picking a winner.
Unknown tokens raise too -- a typo must not silently flip a reproducibility
control.
Because server_default delegates a result-affecting choice to the server's
chat template, the backend warns once per role when it is used, and the
resolved per-role mode is recorded in the run's config.json under
resolved_qwen_thinking_modes.
Also settles the docs contradiction between "local vLLM endpoint" and
"OpenAI-compatible": qwen_chat speaks the OpenAI protocol and reaches both
self-hosted servers and hosted gateways, which is exactly why the wire policy
cannot be inferred and must be explicit.
Closes#90
|`model.qwen_chat_*`| Shared `base_url`, `api_key`, `temperature`, `timeout_seconds`, `max_tokens`, `thinking_mode`, and the legacy `enable_thinking`|
98
+
|`model.qwen_chat_thinking_mode`| Wire policy for `chat_template_kwargs.enable_thinking`: `server_default` (default; omit the field), `enabled`, or `disabled`. See [Qwen thinking mode](#qwen-thinking-mode)|
99
+
|`model.optimizer_qwen_chat_*` / `model.target_qwen_chat_*`| Per-role Qwen overrides, including `*_qwen_chat_thinking_mode`|
70
100
|`model.minimax_*`| MiniMax `region`, `base_url`, `api_key`, shared `minimax_model`, `temperature`, `max_tokens`, and `enable_thinking`; `minimax_model` applies when MiniMax is the target |
71
101
|`model.codex_exec_*`| Codex path, sandbox, profile, SDK mode, reasoning, network/search, and approval policy; see compatibility notes below |
72
102
|`model.claude_code_exec_*`| Claude path, profile, SDK mode, effort, and thinking-token cap |
0 commit comments