Repository navigation
fix(ollama): implement _get_disable_thinking_kwargs so structured-output fallback can disable thinking - #3588
Conversation
…put fallback can disable thinking
|
Thanks for adding the Ollama thinking toggle. The upstream reports in ollama/ollama#10976 and ollama/ollama#10929 suggest that disabling thinking can be a useful workaround in some configurations. However, related upstream fixes such as ollama/ollama#14477 make it important to identify which versions and models still need it. Could you provide a real-API before/after comparison using the same request and environment: the base revision fails to produce valid structured output, while this PR succeeds specifically through the Please also strengthen the regression test by mocking One documentation correction: the current implementation assigns |
|
Updated both points, thanks. Real-environment before/after (full details and raw responses: #3583 (comment)) Ollama
Raw Test (commit Description corrected: you are right that |
Fixes #3583
AgentScope Version
2.0.12dev (main @
2f8c029)Description
ChatModelBase.generate_structured_outputonly includes theno_thinkstrategy in its fallback ladder when the provider overrides_get_disable_thinking_kwargs(src/agentscope/model/_base.py:509-524). Five providers implement it, butOllamaChatModeldoes not — so for Ollama reasoning models (qwen3, deepseek-r1, ...) whose thinking output crowds out the structured payload, the ladder is only forced → auto → none and never attempts thethink=Falserequest that Ollama fully supports (the parameter path already mapsthinking_enableto the top-levelthinkoption atmodel/_ollama/_model.py:178-179).Changes — implement the override, verbatim provider style:
Complementary to #3510 (Ollama
thinktri-state): that fix made the parameter tri-state; this makes the structured-output fallback actually use it. One interaction, documented correctly:_call_apiwriteskwargs["think"] = parameters.thinking_enable(model/_ollama/_model.py:178-179) beforekwargs.update(generate_kwargs)(:181), so theno_thinkattempt'sthink=Falseoverrides an explicitthinking_enable=Truefor that attempt — unlike the DashScopesetdefaultsemantics; happy to align if maintainers prefer.How to test
Regression cases added to the existing
tests/model_ollama_test.py:test_disable_thinking_kwargs_is_think_false— fails onmain({} != {'think': False}).test_fallback_ladder_sends_think_false_and_parses— mocksclient.chatand asserts the ladder sendsthink=Falseon the third (no_think) attempt and that the returned tool arguments are parsed and validated. Onmainthe same test fails with[True, True, True] != [True, True, False].python -m pytest tests/model_ollama_test.py # 15 passed.Real environment reproduction
Reproduced with a real Ollama server (
0.40.3, Windows, CPU), Python clientollama 0.6.2, andqwen3:0.6b(Q4_K_M; 751.63M parameters; default thinkingtrue). Request: extract{name: string, age: integer}from "Alice is 30 years old and lives in Berlin." through thegenerate_structured_outputtool;temperature=0(required for a deterministic repro — with the model's default sampling, one base-revision run succeeded on the second attempt).Raw
/api/chatwith tools (3 runs each, identical):think=true""done_reason=stop, 138 eval tokens)think=false""{"name": "Alice", "age": 30}(29 eval tokens)Through
generate_structured_output(thinking_enable=True, temperature=0):thinkper attemptea903c9true, true, true0, 0, 0StructuredOutputErrortrue, true, false0, 0, 1{'name': 'Alice', 'age': 30}The default
thinking_enable=Noneconfiguration behaves the same (base fails, this PR succeeds), becauseqwen3:0.6benables thinking by default. On Ollama the first three strategies are byte-identical (tool_choiceis ignored and thenonestrategy still sends the tool schema), so theno_thinkattempt is the only request that differs.Checklist