Skip to content

[Bug]: AnthropicChatModel builds thinking.budget_tokens from parameters-level max_tokens only, so a per-call max_tokens is sent with budget_tokens (4096) > max_tokens (1000) — an API-rejected request #3288

Closed

Description

@BlueX888

Prerequisites

  • I have searched the existing issues and discussions, and this is not a duplicate.
  • This is a bug, not a usage question. (For questions, please use Discussions instead.)

Background / Description

AnthropicChatModel with thinking_enable=True sizes thinking.budget_tokens from the parameters-level max_tokens only. When a caller passes a smaller per-call max_tokens via generate_kwargs — a documented public path, ChatModelBase.__call__ forwards **kwargs as "Additional keyword arguments passed to the underlying API" (src/agentscope/model/_base.py:202-203) — the emitted request carries the per-call max_tokens but the parameters-derived budget_tokens, which is then larger. Such a request violates the API's requirement and is rejected with HTTP 400.

Concretely: with parameters.thinking_enable=True (and thinking_budget unset, parameters.max_tokens unset), await model(messages, max_tokens=1000) sends max_tokens=1000 together with thinking.budget_tokens=4096 (half of the 8192 default assumed at src/agentscope/model/_anthropic/_model.py:250).

Expected behaviour: the request must satisfy max_tokens > thinking.budget_tokens, because:

  • the model's own guard states it — src/agentscope/model/_anthropic/_model.py:198: # Anthropic requires max_tokens > budget_tokens strictly.
  • the repo's own test asserts it on emitted requests — test_budget_mode_expands_max_tokens (tests/model_anthropic_test.py:482-491: "max_tokens must stay strictly above budget_tokens"); the auto-expansion in _build_thinking_config exists precisely to maintain this invariant, but on this path it is evaluated against the wrong max_tokens;
  • the Anthropic API requires it — thinking.budget_tokens "Must be ≥1024 and less than max_tokens" (anthropic SDK ThinkingConfigEnabledParam; extended thinking docs: https://platform.claude.com/docs/en/build-with-claude/extended-thinking).

Root cause: in _call_api (src/agentscope/model/_anthropic/_model.py), line 250 derives max_tokens from self.parameters.max_tokens or 8192 before the caller-supplied generate_kwargs are merged into kwargs (lines 252-257); lines 259-266 then build and possibly auto-expand the thinking config against that parameters-derived value, and the expanded max_tokens is only written back when it differs from the parameters-derived one — the caller's max_tokens already sitting in kwargs is never considered when sizing budget_tokens.

Happy to be assigned and open a PR — the fix is to resolve the effective cap after merging generate_kwargs (kwargs.get("max_tokens") or parameters.max_tokens or the 8192 default) and size budget_tokens / any auto-expansion against that value.

Related issues/PRs: searched open issues for "anthropic budget_tokens", "anthropic max_tokens thinking", and "thinking budget" — no existing report found (2026-10-13).

AI disclosure: this report was prepared with AI assistance to locate and reproduce the bug; I ran the reproduction below against commit 34457377b2919f69cd2cc3391552a333bc1791af and verified each cited line in the source myself.

Error Messages

The malformed request is visible without a network call — capturing what client.messages.create receives (mocked client) shows the violated invariant:

sent max_tokens          : 1000
sent thinking            : {'type': 'enabled', 'budget_tokens': 4096}
sent budget_tokens       : 4096
invariant max>budget     : False
guard's assumed max_tokens: 8192

With a control run added (same model, no per-call override), the override is what breaks the invariant:

--- A per-call max_tokens=1000
sent max_tokens    = 1000
sent thinking     = {'type': 'enabled', 'budget_tokens': 4096}
invariant max>budget holds: False
--- B control (no override)
sent max_tokens    = 8192
sent thinking     = {'type': 'enabled', 'budget_tokens': 4096}
invariant max>budget holds: True
REPRODUCED: request would carry max_tokens=1000 with thinking.budget_tokens=4096 -> violates Anthropic's max_tokens > budget_tokens requirement

A real call with the first payload is rejected by the Messages API with HTTP 400, since thinking.budget_tokens (4096) is not less than max_tokens (1000).

Steps to Reproduce

  1. Code (repro_anthropic_budget_vs_max_tokens.py):
import asyncio
from unittest.mock import AsyncMock, MagicMock

from agentscope.model import AnthropicChatModel
from agentscope.credential import AnthropicCredential
from agentscope.message import UserMsg


async def main() -> None:
    captured = {}

    async def fake_create(**kwargs):
        captured.update(kwargs)
        resp = MagicMock()
        resp.content = []
        resp.usage = None
        resp.id = "msg_1"
        return resp

    model = AnthropicChatModel(
        credential=AnthropicCredential(api_key="test"),
        model="claude-opus-4-6",
        stream=False,
        parameters=AnthropicChatModel.Parameters(
            thinking_enable=True,  # budget mode, thinking_budget unset
        ),
    )
    model.client = MagicMock()
    model.client.messages.create = AsyncMock(side_effect=fake_create)

    # Per-call max_tokens via generate_kwargs — public __call__ forwards it.
    await model(
        [UserMsg("user", "Hi")],
        max_tokens=1000,  # per-call override
    )

    sent_max = captured["max_tokens"]
    budget = captured["thinking"]["budget_tokens"]
    print("sent max_tokens          :", sent_max)
    print("sent thinking            :", captured["thinking"])
    print("sent budget_tokens       :", budget)
    print("invariant max>budget     :", sent_max > budget)
    print("guard's assumed max_tokens:", model.parameters.max_tokens or 8192)


asyncio.run(main())
  1. Run: python repro_anthropic_budget_vs_max_tokens.py
  2. See the output above: the request carries max_tokens=1000 with thinking.budget_tokens=4096, so the invariant the model's own guard maintains elsewhere is violated on the per-call-override path.

Environment

  • AgentScope Version: 2.0.10 (source checkout at commit 34457377b2919f69cd2cc3391552a333bc1791af)
  • Python Version: 3.12.14 (anthropic SDK 1.13.0)
  • OS: macOS 26.6.2 (arm64)

Activity

  1. added
    triage/verifyingA bot is verifying whether this bug is real
    triage/confirmedVerified: the reported defect exists
    and removed
    triage/verifyingA bot is verifying whether this bug is real
    on Oct 12, 2026
  2. BlueX888 commented on Oct 12, 2026

    @BlueX888
    ContributorAuthor

    Reproduced this. Root cause: src/agentscope/model/_anthropic/_model.py:250 — _call_api sizes the thinking config from the parameters-derived max_tokens (self.parameters.max_tokens or 8192) before caller generate_kwargs are merged, so a per-call max_tokens never bounds budget_tokens.

    Minimal reproduction:

    python repro_anthropic_budget_vs_max_tokens.py
    

    Observed output:

    sent max_tokens          : 1000
    sent thinking            : {"type": "enabled", "budget_tokens": 4096}
    sent budget_tokens       : 4096
    invariant max>budget     : False
    guard's assumed max_tokens: 8192
    

    Happy to open a PR with this fix, or to be assigned if you prefer to take it.

  3. github-actions commented on Oct 12, 2026

    @github-actions
    Contributor

    It's yours, @BlueX888. If there is no pull request and no word from you by 2026-10-26 the claim is released so nobody is blocked — a comment here renews it.

  4. wuyak commented on Oct 14, 2026

    @wuyak
    Contributor

    After reviewing the implementation and the sequence of related changes, I believe the issue currently defines only the parameter-calculation error that directly triggers the 400 response. It does not yet address the older behavior that should actually be reconsidered: when the caller does not provide a thinking budget, AgentScope derives budget_tokens from max_tokens.

    The current PR #3292 follows the issue as written. It uses the effective per-call max_tokens to calculate the budget and therefore fixes the mismatch in the reported reproduction. However, it also preserves and expands the existing implicit budgeting policy. I therefore suggest that the issue author consider reframing the problem from “the automatic budget uses max_tokens from the wrong level” to “should AgentScope continue deriving a thinking budget from max_tokens at all?”, and then use that framing to determine the appropriate scope of the current PR.

    How this configuration evolved

    The initial Anthropic implementation accepted the provider-native thinking configuration directly:

    thinking={
        "type": "enabled",
        "budget_tokens": 1024,
    }

    AgentScope forwarded the mode and budget supplied by the caller. It did not generate a budget from the output limit.

    PR #1440 then replaced that raw dictionary with a typed configuration:

    class ThinkingConfig(BaseModel):
        enable_thinking: bool
        budget_tokens: int = 1024

    At this point, budget_tokens was still modeled independently. It had a fixed and visible default, but that default did not depend on max_tokens.

    The behavior changed in PR #1564. During the model API refactor, it removed ThinkingConfig and replaced it with two flat fields:

    thinking_enable: bool = False
    thinking_budget: int | None = None

    This introduced an underspecified state that had not existed before: manual thinking could be enabled while its budget remained unset. The same refactor filled that state with the following fallback:

    budget = self.parameters.thinking_budget or (max_tokens // 2)

    It also began expanding max_tokens automatically when the budget did not fit. The main purpose of that change was to refactor the model API, and its PR description did not present “half of the output limit” as a deliberately designed budgeting policy. Based on this history, max_tokens // 2 was introduced as a fallback after the old configuration was split and thinking_budget=None became possible. It was not part of the original Anthropic interface behavior.

    By PR #2240, the project had already recognized that a single thinking_enable flag could no longer represent Anthropic's thinking configuration, and added:

    thinking_mode = "adaptive" | "enabled" | "disabled"
    reasoning_effort = "low" | "medium" | "high" | "xhigh" | "max"

    That change also established that:

    • thinking_mode takes precedence over the legacy thinking_enable flag;
    • adaptive thinking does not send budget_tokens;
    • reasoning_effort is an independent control;
    • thinking_enable remains only as a compatibility entry point for the legacy budget-based mode.

    However, it retained the max_tokens // 2 fallback on that compatibility path. The current interface can therefore express an explicit thinking mode and effort, while the legacy path still invents a budget from the output limit when the caller does not provide one.

    What this means for the current issue

    The reported failure exposes more than an ordering error in parameter merging.

    Once a per-call max_tokens override is introduced, the old fallback derives a budget from the parameters-level limit while the final request uses the call-level limit, so the two values conflict. This reproduction shows that as long as the thinking budget is derived implicitly from another setting, it can diverge from the caller's intent for a particular request.

    The current PR changes the derivation to use the effective call-level limit:

    budget = thinking_budget or (effective_max_tokens // 2)

    This removes the mismatch in the reported reproduction. But to make the derived value satisfy the API constraints across different inputs, it also needs to introduce additional behavior:

    • raise an automatically derived budget to the minimum value of 1024;
    • rewrite an explicit thinking_budget=512 to 1024;
    • raise an error when an automatically derived budget cannot fit under a small max_tokens;
    • automatically expand a per-call max_tokens when an explicit budget does not fit.

    These behaviors can ensure that the request sent to the provider is valid. At the same time, they extend AgentScope's responsibility from one legacy fallback formula into a broader budgeting policy: generating a budget, changing a budget, and rewriting an output limit.

    Based on the evolution above, I believe the more accurate root cause is:

    thinking_enable=True without a thinking_budget does not fully specify the manual-thinking configuration. The max_tokens // 2 fallback hides that underspecified state.

    The 400 caused by the per-call override is one concrete consequence of that fallback, rather than the complete problem.

    Suggested direction for the issue

    I suggest that the issue author consider extending the acceptance boundary in the following direction:

    • Remove the max_tokens // 2 budget derivation.
    • Have manual thinking_mode="enabled" use an explicitly supplied thinking_budget.
    • Keep adaptive thinking budget-free and continue to express reasoning intensity through the independent reasoning_effort setting.
    • Resolve the effective per-call max_tokens, but use it to validate whether the explicit configuration is compatible rather than to generate a thinking budget.
    • When an explicit budget conflicts with an explicit output limit, report a clear error containing both values instead of silently rewriting either one.
    • Preserve a provider-native thinking configuration supplied explicitly for a single call as the highest-precedence value, and send it unchanged.

    The compatibility behavior for legacy thinking_enable=True without thinking_budget can be decided separately. It could require an explicit budget immediately, or temporarily retain a fixed and documented legacy default with a deprecation warning. Either migration path would avoid deriving an unexpressed thinking budget from max_tokens.

    If the issue is reframed this way, several behaviors currently treated as expected would also need to be reconsidered:

    • max_tokens=4096 without an explicit budget would no longer automatically produce budget_tokens=2048;
    • max_tokens=1500 would no longer automatically produce a 1024-token thinking budget;
    • an explicit thinking_budget=512 would report a configuration error instead of being silently changed to 1024;
    • an explicit thinking_budget=4096 combined with per-call max_tokens=1000 would report the conflict instead of silently expanding the output limit.

    Whether the implementation continues using the current flat fields or introduces separate configuration types for the different modes can be discussed at the implementation stage. The more important decision for this issue is the boundary: max_tokens may constrain and validate a thinking budget, but it should not generate one.

    I therefore suggest that the issue author consider updating the problem statement and expected behavior first, and then reassess whether the current PR should be adjusted accordingly.

  5. BlueX888 commented on Oct 14, 2026

    @BlueX888
    ContributorAuthor

    Thanks @wuyak — this history is genuinely useful, and I agree with the core observation: the max_tokens // 2 fallback entered in #1564 when the typed config was split and thinking_budget=None became representable; it was never presented as a designed budgeting policy, and thinking_enable=True without a thinking_budget is an underspecified state that the fallback silently fills.

    As the issue author, here is where I have landed on scope.

    This issue stays about the 400. As filed, it reports that the thinking config is sized from the parameters-level max_tokens while the request actually sends the per-call one, so the two values conflict and the API rejects the request. #3292 fixes exactly that. Its resolution step is also a prerequisite for your proposed end state — your direction keeps resolving the effective per-call max_tokens and uses it to validate the explicit configuration — so the bugfix is forward-compatible with the redesign rather than in tension with it.

    On the additional behaviors #3292 introduces: they are the minimum needed to keep the emitted budget one the API accepts, now that the per-call cap is honored. Anthropic rejects budget_tokens < 1024, so a per-call max_tokens=1500 would otherwise derive a 750 budget and reproduce the reported 400 in an adjacent configuration; an explicit thinking_budget=512 is likewise a 400 on current main (the floor fixes that too). The fail-fast for unsatisfiable auto budgets and the auto-expand for explicit budgets (pre-existing on main) follow the same logic. I agree the silent 512 → 1024 rewrite is debatable — raising there instead is worth discussing in the follow-up.

    The redesign itself is a maintainer decision. Removing the derivation entirely, raising on explicit budget/cap conflicts instead of auto-expanding, and choosing the migration path for legacy thinking_enable=True without a budget are public-behavior changes: configurations that succeed today would start failing or require explicit values. That deserves its own issue where the intended end state, the deprecation timeline, and the migration plan are spelled out, rather than being folded into — and holding up — a green bugfix for the reported 400.

    So my proposal: keep #3292 scoped to making the budget consistent with the limit actually sent on the call, and take "should AgentScope derive a thinking budget from max_tokens at all?" to a separate design discussion. I am happy to open that follow-up issue, with your write-up above as the basis, if the maintainers want it — and if they would rather land the redesign in one step, I can adjust #3292 accordingly. Maintainers, your call on scope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

triage/confirmedVerified: the reported defect exists

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions