Skip to content

fix(ollama): implement _get_disable_thinking_kwargs so structured-output fallback can disable thinking - #3588

Merged
qbc2016 merged 2 commits into
agentscope-ai:mainfrom
konlue:fix/ollama-no-think-fallback
Nov 2, 2026
Merged

qbc2016 merged 2 commits into
agentscope-ai:mainfrom
konlue:fix/ollama-no-think-fallback

Conversation

@konlue

@konlue konlue commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #3583

AgentScope Version

2.0.12dev (main @ 2f8c029)

Description

ChatModelBase.generate_structured_output only includes the no_think strategy in its fallback ladder when the provider overrides _get_disable_thinking_kwargs (src/agentscope/model/_base.py:509-524). Five providers implement it, but OllamaChatModel does not — so for Ollama reasoning models (qwen3, deepseek-r1, ...) whose thinking output crowds out the structured payload, the ladder is only forced → auto → none and never attempts the think=False request that Ollama fully supports (the parameter path already maps thinking_enable to the top-level think option at model/_ollama/_model.py:178-179).

Changes — implement the override, verbatim provider style:

def _get_disable_thinking_kwargs(self) -> dict:
    """Ollama uses a top-level ``think`` option."""
    return {"think": False}

Complementary to #3510 (Ollama think tri-state): that fix made the parameter tri-state; this makes the structured-output fallback actually use it. One interaction, documented correctly: _call_api writes kwargs["think"] = parameters.thinking_enable (model/_ollama/_model.py:178-179) before kwargs.update(generate_kwargs) (:181), so the no_think attempt's think=False overrides an explicit thinking_enable=True for that attempt — unlike the DashScope setdefault semantics; happy to align if maintainers prefer.

How to test

Regression cases added to the existing tests/model_ollama_test.py:

  • test_disable_thinking_kwargs_is_think_false — fails on main ({} != {'think': False}).
  • test_fallback_ladder_sends_think_false_and_parses — mocks client.chat and asserts the ladder sends think=False on the third (no_think) attempt and that the returned tool arguments are parsed and validated. On main the same test fails with [True, True, True] != [True, True, False].
python -m pytest tests/model_ollama_test.py
# 15 passed.

Real environment reproduction

Reproduced with a real Ollama server (0.40.3, Windows, CPU), Python client ollama 0.6.2, and qwen3:0.6b (Q4_K_M; 751.63M parameters; default thinking true). Request: extract {name: string, age: integer} from "Alice is 30 years old and lives in Berlin." through the generate_structured_output tool; temperature=0 (required for a deterministic repro — with the model's default sampling, one base-revision run succeeded on the second attempt).

Raw /api/chat with tools (3 runs each, identical):

request thinking content tool_calls
think=true 465 chars "" none (done_reason=stop, 138 eval tokens)
think=false 0 "" {"name": "Alice", "age": 30} (29 eval tokens)

Through generate_structured_output (thinking_enable=True, temperature=0):

revision SDK think per attempt tool calls result
base ea903c9 true, true, true 0, 0, 0 StructuredOutputError
this PR true, true, false 0, 0, 1 {'name': 'Alice', 'age': 30}

The default thinking_enable=None configuration behaves the same (base fails, this PR succeeds), because qwen3:0.6b enables thinking by default. On Ollama the first three strategies are byte-identical (tool_choice is ignored and the none strategy still sends the tool schema), so the no_think attempt is the only request that differs.

Checklist

  • An issue has been created for this PR
  • I have read the CONTRIBUTING.md
  • Docstrings are in Google style
  • Related documentation has been updated (e.g. links, examples, etc.) in documentation repository
  • Code is ready for review

@qbc2016

qbc2016 commented Nov 2, 2026

Copy link
Copy Markdown
Member

Thanks for adding the Ollama thinking toggle. The upstream reports in ollama/ollama#10976 and ollama/ollama#10929 suggest that disabling thinking can be a useful workaround in some configurations. However, related upstream fixes such as ollama/ollama#14477 make it important to identify which versions and models still need it.

Could you provide a real-API before/after comparison using the same request and environment: the base revision fails to produce valid structured output, while this PR succeeds specifically through the no_think fallback? Please include the server/SDK versions, exact model, request parameters, and responses for each attempt. Since AgentScope uses the tool-calling path here, a native JSON Schema reproduction alone would not establish that this change resolves the failure.

Please also strengthen the regression test by mocking client.chat rather than replacing _call_api, so it verifies that think=False reaches the SDK and that the returned tool arguments are successfully parsed and validated.

One documentation correction: the current implementation assigns parameters.thinking_enable before kwargs.update(generate_kwargs), so the fallback's think=False overrides an explicit thinking_enable=True. The PR description currently states the opposite.

@konlue

konlue commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

Updated both points, thanks.

Real-environment before/after (full details and raw responses: #3583 (comment))

Ollama 0.40.3, ollama Python client 0.6.2, qwen3:0.6b (Q4_K_M), temperature=0, structured output through the generate_structured_output tool:

revision SDK think per attempt tool calls result
base ea903c9 true, true, true 0, 0, 0 StructuredOutputError
this PR true, true, false 0, 0, 1 {'name': 'Alice', 'age': 30}

Raw /api/chat with tools: think=true returns only a 465-character thinking block and no tool call (done_reason=stop); think=false returns the valid tool call. The default thinking_enable=None configuration gives the same result, since this model enables thinking by default.

Test (commit 14b8b6a): the regression test now mocks client.chat rather than _call_api. It returns plain text for the first two attempts and a valid tool call for the third, then asserts [True, True, False] for the think argument and that the returned arguments are parsed and validated. On main it fails with [True, True, True] != [True, True, False]; on this branch python -m pytest tests/model_ollama_test.py → 15 passed.

Description corrected: you are right that _call_api writes kwargs["think"] = parameters.thinking_enable (model/_ollama/_model.py:178-179) before kwargs.update(generate_kwargs) (:181), so the fallback's think=False overrides an explicit thinking_enable=True for that attempt. The previous sentence stated the opposite; fixed in the description.

@qbc2016 qbc2016 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qbc2016
qbc2016 merged commit 633646c into agentscope-ai:main Nov 2, 2026
12 checks passed
@github-actions github-actions Bot removed the pr/awaiting-review Waiting for maintainers to review this PR label Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Ollama structured-output fallback never tries no_think — _get_disable_thinking_kwargs is not overridden

2 participants