fix(models): forward reasoning_content as top-level field in multi-turn#1396
Conversation
…rn requests
Redefine GenerateConfig.reasoning_history from inspect_ai-style filter
values ('none'|'all'|'last'|'auto') to evalscope-style wire-format values
('none'|'think_tag'|'reasoning_field'), and consume it in
OpenAICompatibleAPI so the encoded shape of prior-turn reasoning_content
can be controlled per-request.
BREAKING CHANGE: default wire format switches from `<think>...</think>`
inline encoding to a top-level `reasoning_content` field, matching the
modern OpenAI-compatible reasoning protocol. Fixes DeepSeek V4 thinking
400 errors and removes context pollution for Qwen3 thinking. DeepSeek R1
(legacy) users must now explicitly opt out via `reasoning_history='none'`.
Also clean up GenerateConfig:
- Remove the cargo-culted `migrate_reasoning` bool->literal validator
(zero consumers since v1.0).
- Remove 4 zombie fields with no consumers anywhere in the repo:
best_of, cache_prompt, internal_tools, max_tool_output.
- Document `batch_size` as an internal field synced from
TaskConfig.eval_batch_size (not user-facing).
- Document 6 previously-undocumented but actively-consumed fields:
stop_seqs, repetition_penalty, response_schema, reasoning_effort,
reasoning_tokens, reasoning_summary. Fix `timeout` type (int -> int/float).
Refs: #1392
Prior art: BerriAI/litellm#28080
There was a problem hiding this comment.
Code Review
This pull request updates the generation configuration and OpenAI-compatible model handling to support modern reasoning models (such as DeepSeek V4 and Qwen3 thinking modes). It replaces the legacy reasoning_history options with a new reasoning_format parameter supporting think_tag, reasoning_field, and none modes. This allows proper encoding of reasoning content either inline, as a top-level reasoning_content field, or stripping it entirely. The documentation and unit tests have been updated accordingly. The reviewer suggested ensuring backward compatibility in openai_assistant_content by allowing reasoning_format to accept a boolean value (the legacy include_reasoning parameter) and normalizing it internally.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
There was a problem hiding this comment.
Pull request overview
This PR fixes multi-turn OpenAI-compatible request serialization for models/providers that require prior-turn reasoning_content to be echoed back as a top-level field (not embedded inside content as <think>...</think>). It introduces an explicit reasoning_history wire-format mode and updates the OpenAI-compatible API path to emit reasoning_content when configured (defaulting to the new reasoning_field behavior at consumption time).
Changes:
- Add
ReasoningFormatsupport to OpenAI chat message serialization, including promoting the lastContentReasoningto top-levelreasoning_contentand a blank-reasoning single-space fallback for strict servers. - Redefine
GenerateConfig.reasoning_historyliteral values to match wire formats (none | think_tag | reasoning_field) and consume it inOpenAICompatibleAPI.generate/generate_async(defaulting toreasoning_fieldwhen unset). - Add unit tests for the three reasoning wire shapes and update parameter docs (EN/ZH) to document the new behavior and options.
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
tests/models/test_openai_reasoning.py |
Adds unit coverage pinning the outgoing assistant message wire shape across think_tag, reasoning_field, and none, plus schema validation for reasoning_history. |
evalscope/models/utils/openai.py |
Adds reasoning_format to OpenAI chat message serialization, promotes reasoning_content as a top-level field in reasoning_field mode, and introduces _extract_last_reasoning. |
evalscope/models/openai_compatible.py |
Passes reasoning_history through to openai_chat_messages, defaulting to reasoning_field when unset to fix strict providers (e.g., DeepSeek V4 thinking). |
evalscope/api/model/generate_config.py |
Updates reasoning_history allowed literals/documentation and removes a legacy migration validator and unused fields from the typed schema. |
docs/zh/get_started/parameters.md |
Documents reasoning_history modes and newly documented generation config fields; updates timeout type to int/float. |
docs/en/get_started/parameters.md |
Same as ZH: documents reasoning_history modes/behavior and updates timeout type to int/float. |
…rn (modelscope#1396) * fix(models): forward reasoning_content as top-level field in multi-turn requests Redefine GenerateConfig.reasoning_history from inspect_ai-style filter values ('none'|'all'|'last'|'auto') to evalscope-style wire-format values ('none'|'think_tag'|'reasoning_field'), and consume it in OpenAICompatibleAPI so the encoded shape of prior-turn reasoning_content can be controlled per-request. BREAKING CHANGE: default wire format switches from `<think>...</think>` inline encoding to a top-level `reasoning_content` field, matching the modern OpenAI-compatible reasoning protocol. Fixes DeepSeek V4 thinking 400 errors and removes context pollution for Qwen3 thinking. DeepSeek R1 (legacy) users must now explicitly opt out via `reasoning_history='none'`. Also clean up GenerateConfig: - Remove the cargo-culted `migrate_reasoning` bool->literal validator (zero consumers since v1.0). - Remove 4 zombie fields with no consumers anywhere in the repo: best_of, cache_prompt, internal_tools, max_tool_output. - Document `batch_size` as an internal field synced from TaskConfig.eval_batch_size (not user-facing). - Document 6 previously-undocumented but actively-consumed fields: stop_seqs, repetition_penalty, response_schema, reasoning_effort, reasoning_tokens, reasoning_summary. Fix `timeout` type (int -> int/float). Refs: modelscope#1392 Prior art: BerriAI/litellm#28080 * chore: fix isort import grouping for new test file
Summary
Fixes #1392 — DeepSeek V4 thinking mode rejects evalscope's multi-turn requests with
HTTP 400 "The reasoning_content in the thinking mode must be passed back to the API.", and Qwen3 thinking silently pollutes context by parsing<think>...</think>as plain text.Root cause:
openai_assistant_contentencodesContentReasoningas<think>...</think>inside thecontentstring and never writes a top-levelreasoning_contentfield. This worked for OpenAI/Together/Groq but breaks modern OpenAI-compatible providers that expectreasoning_contentas an independent field.Fix: redefine
GenerateConfig.reasoning_historyliteral from inspect_ai-style filter values ('none'|'all'|'last'|'auto') to wire-format values ('none'|'think_tag'|'reasoning_field'), and consume it inOpenAICompatibleAPI.generate/generate_asyncto control the per-request wire shape.⚠ Breaking Change
think_tag)reasoning_field)deepseek-reasoner)R1 users must explicitly set
--generation-config '{\"reasoning_history\":\"none\"}'because R1's protocol forbidsreasoning_contentecho-back. Documented indocs/{zh,en}/get_started/parameters.md.Affected benchmarks before the fix
bfcl_v3(the one reported in DeepSeek模型多轮对话时reasoning_content字 段处理不当导致评测失败 #1392),gaia,swe_bench_*_agentic,swe_bench_pro_agentic— all route prior-turn assistant messages throughopenai_chat_messagesand hit the bugtau_bench/tau2/tau3/terminal_benchalready workaround via response-sideopenai_chat_choices(include_reasoning=False); their behavior is unchangedDrive-by cleanups in
GenerateConfigmigrate_reasoningbool→literal validator (zero consumers since v1.0)best_of,cache_prompt,internal_tools,max_tool_outputbatch_sizeas an internal field (synced fromTaskConfig.eval_batch_size, not user-facing)stop_seqs,repetition_penalty,response_schema,reasoning_effort,reasoning_tokens,reasoning_summary. Fixtimeouttype (int→int/float)Design notes
ContentReasoningwins when multiple are present on an assistant message (matches litellm's behavior; see_extract_last_reasoning).\" \"placeholder with a debug log (DeepSeek V4 rejects emptyreasoning_content).reasoning_history.litellm_compatible.pypath is intentionally out of scope (litellm has its own schema validation that may strip extra top-level keys — deferred to a follow-up PR).Prior art: BerriAI/litellm#28080 — the same fix for the DeepSeek path in litellm, including the single-space fallback design.
Test plan
make lintpassespytest tests/models/test_openai_reasoning.py -v— 12 new unit tests covering 3 modes × 5 message shapes + schema invariants (legacy literal rejection, new literal acceptance, default behavior)pytest tests/agent/external/test_openai_chat_round_trip.py -v— 6 round-trip tests still green (regression check)pytest tests/cli/test_all.py::TestRun::test_ci_lite -v -s -p no:warnings— CI smoke green (no regression on the happy-path benchmark run)qwen3-max(DashScope, thinking enabled): all 3 modes produce the expected wire shape (<think>in content / top-levelreasoning_content/ clean drop) and all 3 are accepted by the serverdeepseek-v4-pro+extra_body={\"thinking\":{\"type\":\"enabled\"}}) returns 200 instead of 400deepseek-reasoner) regression — default config should now 400;reasoning_history='none'should return 200