Repository navigation
Description
Prerequisites
- I have searched the existing issues and discussions, and this is not a duplicate.
- This is a bug, not a usage question. (For questions, please use Discussions instead.)
Background / Description
AnthropicChatModel with thinking_enable=True sizes thinking.budget_tokens from the parameters-level max_tokens only. When a caller passes a smaller per-call max_tokens via generate_kwargs — a documented public path, ChatModelBase.__call__ forwards **kwargs as "Additional keyword arguments passed to the underlying API" (src/agentscope/model/_base.py:202-203) — the emitted request carries the per-call max_tokens but the parameters-derived budget_tokens, which is then larger. Such a request violates the API's requirement and is rejected with HTTP 400.
Concretely: with parameters.thinking_enable=True (and thinking_budget unset, parameters.max_tokens unset), await model(messages, max_tokens=1000) sends max_tokens=1000 together with thinking.budget_tokens=4096 (half of the 8192 default assumed at src/agentscope/model/_anthropic/_model.py:250).
Expected behaviour: the request must satisfy max_tokens > thinking.budget_tokens, because:
- the model's own guard states it —
src/agentscope/model/_anthropic/_model.py:198:# Anthropic requires max_tokens > budget_tokens strictly. - the repo's own test asserts it on emitted requests —
test_budget_mode_expands_max_tokens(tests/model_anthropic_test.py:482-491: "max_tokens must stay strictly above budget_tokens"); the auto-expansion in_build_thinking_configexists precisely to maintain this invariant, but on this path it is evaluated against the wrongmax_tokens; - the Anthropic API requires it —
thinking.budget_tokens"Must be ≥1024 and less thanmax_tokens" (anthropic SDKThinkingConfigEnabledParam; extended thinking docs: https://platform.claude.com/docs/en/build-with-claude/extended-thinking).
Root cause: in _call_api (src/agentscope/model/_anthropic/_model.py), line 250 derives max_tokens from self.parameters.max_tokens or 8192 before the caller-supplied generate_kwargs are merged into kwargs (lines 252-257); lines 259-266 then build and possibly auto-expand the thinking config against that parameters-derived value, and the expanded max_tokens is only written back when it differs from the parameters-derived one — the caller's max_tokens already sitting in kwargs is never considered when sizing budget_tokens.
Happy to be assigned and open a PR — the fix is to resolve the effective cap after merging generate_kwargs (kwargs.get("max_tokens") or parameters.max_tokens or the 8192 default) and size budget_tokens / any auto-expansion against that value.
Related issues/PRs: searched open issues for "anthropic budget_tokens", "anthropic max_tokens thinking", and "thinking budget" — no existing report found (2026-10-13).
AI disclosure: this report was prepared with AI assistance to locate and reproduce the bug; I ran the reproduction below against commit 34457377b2919f69cd2cc3391552a333bc1791af and verified each cited line in the source myself.
Error Messages
The malformed request is visible without a network call — capturing what client.messages.create receives (mocked client) shows the violated invariant:
sent max_tokens : 1000
sent thinking : {'type': 'enabled', 'budget_tokens': 4096}
sent budget_tokens : 4096
invariant max>budget : False
guard's assumed max_tokens: 8192
With a control run added (same model, no per-call override), the override is what breaks the invariant:
--- A per-call max_tokens=1000
sent max_tokens = 1000
sent thinking = {'type': 'enabled', 'budget_tokens': 4096}
invariant max>budget holds: False
--- B control (no override)
sent max_tokens = 8192
sent thinking = {'type': 'enabled', 'budget_tokens': 4096}
invariant max>budget holds: True
REPRODUCED: request would carry max_tokens=1000 with thinking.budget_tokens=4096 -> violates Anthropic's max_tokens > budget_tokens requirement
A real call with the first payload is rejected by the Messages API with HTTP 400, since thinking.budget_tokens (4096) is not less than max_tokens (1000).
Steps to Reproduce
- Code (
repro_anthropic_budget_vs_max_tokens.py):
import asyncio
from unittest.mock import AsyncMock, MagicMock
from agentscope.model import AnthropicChatModel
from agentscope.credential import AnthropicCredential
from agentscope.message import UserMsg
async def main() -> None:
captured = {}
async def fake_create(**kwargs):
captured.update(kwargs)
resp = MagicMock()
resp.content = []
resp.usage = None
resp.id = "msg_1"
return resp
model = AnthropicChatModel(
credential=AnthropicCredential(api_key="test"),
model="claude-opus-4-6",
stream=False,
parameters=AnthropicChatModel.Parameters(
thinking_enable=True, # budget mode, thinking_budget unset
),
)
model.client = MagicMock()
model.client.messages.create = AsyncMock(side_effect=fake_create)
# Per-call max_tokens via generate_kwargs — public __call__ forwards it.
await model(
[UserMsg("user", "Hi")],
max_tokens=1000, # per-call override
)
sent_max = captured["max_tokens"]
budget = captured["thinking"]["budget_tokens"]
print("sent max_tokens :", sent_max)
print("sent thinking :", captured["thinking"])
print("sent budget_tokens :", budget)
print("invariant max>budget :", sent_max > budget)
print("guard's assumed max_tokens:", model.parameters.max_tokens or 8192)
asyncio.run(main())- Run:
python repro_anthropic_budget_vs_max_tokens.py - See the output above: the request carries
max_tokens=1000withthinking.budget_tokens=4096, so the invariant the model's own guard maintains elsewhere is violated on the per-call-override path.
Environment
- AgentScope Version: 2.0.10 (source checkout at commit
34457377b2919f69cd2cc3391552a333bc1791af) - Python Version: 3.12.14 (anthropic SDK 1.13.0)
- OS: macOS 26.6.2 (arm64)
Activity
BlueX888 commentedon Oct 12, 2026
Reproduced this. Root cause: src/agentscope/model/_anthropic/_model.py:250 — _call_api sizes the thinking config from the parameters-derived max_tokens (self.parameters.max_tokens or 8192) before caller generate_kwargs are merged, so a per-call max_tokens never bounds budget_tokens.
Minimal reproduction:
Observed output:
Happy to open a PR with this fix, or to be assigned if you prefer to take it.
github-actions commentedon Oct 12, 2026
It's yours, @BlueX888. If there is no pull request and no word from you by 2026-10-26 the claim is released so nobody is blocked — a comment here renews it.
wuyak commentedon Oct 14, 2026
After reviewing the implementation and the sequence of related changes, I believe the issue currently defines only the parameter-calculation error that directly triggers the 400 response. It does not yet address the older behavior that should actually be reconsidered: when the caller does not provide a thinking budget, AgentScope derives
budget_tokensfrommax_tokens.The current PR #3292 follows the issue as written. It uses the effective per-call
max_tokensto calculate the budget and therefore fixes the mismatch in the reported reproduction. However, it also preserves and expands the existing implicit budgeting policy. I therefore suggest that the issue author consider reframing the problem from “the automatic budget usesmax_tokensfrom the wrong level” to “should AgentScope continue deriving a thinking budget frommax_tokensat all?”, and then use that framing to determine the appropriate scope of the current PR.How this configuration evolved
The initial Anthropic implementation accepted the provider-native thinking configuration directly:
AgentScope forwarded the mode and budget supplied by the caller. It did not generate a budget from the output limit.
PR #1440 then replaced that raw dictionary with a typed configuration:
At this point,
budget_tokenswas still modeled independently. It had a fixed and visible default, but that default did not depend onmax_tokens.The behavior changed in PR #1564. During the model API refactor, it removed
ThinkingConfigand replaced it with two flat fields:This introduced an underspecified state that had not existed before: manual thinking could be enabled while its budget remained unset. The same refactor filled that state with the following fallback:
It also began expanding
max_tokensautomatically when the budget did not fit. The main purpose of that change was to refactor the model API, and its PR description did not present “half of the output limit” as a deliberately designed budgeting policy. Based on this history,max_tokens // 2was introduced as a fallback after the old configuration was split andthinking_budget=Nonebecame possible. It was not part of the original Anthropic interface behavior.By PR #2240, the project had already recognized that a single
thinking_enableflag could no longer represent Anthropic's thinking configuration, and added:That change also established that:
thinking_modetakes precedence over the legacythinking_enableflag;budget_tokens;reasoning_effortis an independent control;thinking_enableremains only as a compatibility entry point for the legacy budget-based mode.However, it retained the
max_tokens // 2fallback on that compatibility path. The current interface can therefore express an explicit thinking mode and effort, while the legacy path still invents a budget from the output limit when the caller does not provide one.What this means for the current issue
The reported failure exposes more than an ordering error in parameter merging.
Once a per-call
max_tokensoverride is introduced, the old fallback derives a budget from the parameters-level limit while the final request uses the call-level limit, so the two values conflict. This reproduction shows that as long as the thinking budget is derived implicitly from another setting, it can diverge from the caller's intent for a particular request.The current PR changes the derivation to use the effective call-level limit:
This removes the mismatch in the reported reproduction. But to make the derived value satisfy the API constraints across different inputs, it also needs to introduce additional behavior:
thinking_budget=512to 1024;max_tokens;max_tokenswhen an explicit budget does not fit.These behaviors can ensure that the request sent to the provider is valid. At the same time, they extend AgentScope's responsibility from one legacy fallback formula into a broader budgeting policy: generating a budget, changing a budget, and rewriting an output limit.
Based on the evolution above, I believe the more accurate root cause is:
The 400 caused by the per-call override is one concrete consequence of that fallback, rather than the complete problem.
Suggested direction for the issue
I suggest that the issue author consider extending the acceptance boundary in the following direction:
max_tokens // 2budget derivation.thinking_mode="enabled"use an explicitly suppliedthinking_budget.reasoning_effortsetting.max_tokens, but use it to validate whether the explicit configuration is compatible rather than to generate a thinking budget.thinkingconfiguration supplied explicitly for a single call as the highest-precedence value, and send it unchanged.The compatibility behavior for legacy
thinking_enable=Truewithoutthinking_budgetcan be decided separately. It could require an explicit budget immediately, or temporarily retain a fixed and documented legacy default with a deprecation warning. Either migration path would avoid deriving an unexpressed thinking budget frommax_tokens.If the issue is reframed this way, several behaviors currently treated as expected would also need to be reconsidered:
max_tokens=4096without an explicit budget would no longer automatically producebudget_tokens=2048;max_tokens=1500would no longer automatically produce a 1024-token thinking budget;thinking_budget=512would report a configuration error instead of being silently changed to 1024;thinking_budget=4096combined with per-callmax_tokens=1000would report the conflict instead of silently expanding the output limit.Whether the implementation continues using the current flat fields or introduces separate configuration types for the different modes can be discussed at the implementation stage. The more important decision for this issue is the boundary:
max_tokensmay constrain and validate a thinking budget, but it should not generate one.I therefore suggest that the issue author consider updating the problem statement and expected behavior first, and then reassess whether the current PR should be adjusted accordingly.
BlueX888 commentedon Oct 14, 2026
Thanks @wuyak — this history is genuinely useful, and I agree with the core observation: the
max_tokens // 2fallback entered in #1564 when the typed config was split andthinking_budget=Nonebecame representable; it was never presented as a designed budgeting policy, andthinking_enable=Truewithout athinking_budgetis an underspecified state that the fallback silently fills.As the issue author, here is where I have landed on scope.
This issue stays about the 400. As filed, it reports that the thinking config is sized from the parameters-level
max_tokenswhile the request actually sends the per-call one, so the two values conflict and the API rejects the request. #3292 fixes exactly that. Its resolution step is also a prerequisite for your proposed end state — your direction keeps resolving the effective per-callmax_tokensand uses it to validate the explicit configuration — so the bugfix is forward-compatible with the redesign rather than in tension with it.On the additional behaviors #3292 introduces: they are the minimum needed to keep the emitted budget one the API accepts, now that the per-call cap is honored. Anthropic rejects
budget_tokens < 1024, so a per-callmax_tokens=1500would otherwise derive a 750 budget and reproduce the reported 400 in an adjacent configuration; an explicitthinking_budget=512is likewise a 400 on current main (the floor fixes that too). The fail-fast for unsatisfiable auto budgets and the auto-expand for explicit budgets (pre-existing on main) follow the same logic. I agree the silent512 → 1024rewrite is debatable — raising there instead is worth discussing in the follow-up.The redesign itself is a maintainer decision. Removing the derivation entirely, raising on explicit budget/cap conflicts instead of auto-expanding, and choosing the migration path for legacy
thinking_enable=Truewithout a budget are public-behavior changes: configurations that succeed today would start failing or require explicit values. That deserves its own issue where the intended end state, the deprecation timeline, and the migration plan are spelled out, rather than being folded into — and holding up — a green bugfix for the reported 400.So my proposal: keep #3292 scoped to making the budget consistent with the limit actually sent on the call, and take "should AgentScope derive a thinking budget from
max_tokensat all?" to a separate design discussion. I am happy to open that follow-up issue, with your write-up above as the basis, if the maintainers want it — and if they would rather land the redesign in one step, I can adjust #3292 accordingly. Maintainers, your call on scope.