Skip to content

fix(ai): fit Claude thinking budgets on Bedrock Converse - #51190

Merged
rekram1-node merged 1 commit into
v2from
bedrock-thinking-fit
Sep 24, 2026
Merged

rekram1-node merged 1 commit into
v2from
bedrock-thinking-fit

Conversation

@rekram1-node

Copy link
Copy Markdown
Collaborator

Issue for this PR

N/A. Found while testing default output limits for #51021.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Claude thinking budgets on Bedrock Converse now reach the protocol as a typed option, so they get fitted to half the output limit like Anthropic's own API does since #51157.

Bedrock enforces the same rule as Anthropic:

max_tokens must be greater than thinking.budget_tokens

Our Claude budget variants sent the budget as a raw body overlay (additionalModelRequestFields.thinking). That overlay is merged after the protocol builds the body, so the protocol could not fit it. Automatic compaction starts when the summary's output limit is below about 28k. So on Claude 4.5 models, the max variant (31,999) was always rejected at automatic compaction, and high (16,000) was rejected once the limit fell below 16k.

  • packages/ai: Bedrock Converse reads a typed thinking: { type: "enabled", budgetTokens } option. It writes additionalModelRequestFields.thinking, fitting the budget with a 1,024 minimum. The Bedrock provider passes a thinking setting through as that option.
  • packages/core: Claude budget variants on Bedrock Converse now use the thinking setting. The AI SDK Bedrock config mapping does the same for reasoningConfig: { type: "enabled", budgetTokens } on Claude.

Other Bedrock fields (adaptive thinking, effort, Grok, Nova, OpenAI) keep using the body overlay.

How did you verify your code works?

  • Unit tests:

    • Converse fits the budget and merges it with top_k: 31,999 stays under a 64,000 limit, becomes 10,000 under 20,000, and becomes 1,024 under 1,500.
    • Claude budget variants spell the thinking setting.
    • The AI SDK mapping routes Claude's enabled budget to the same setting.
  • Checks: the affected packages/ai and packages/core suites pass, and bun run check passes with feat(core): fit output limits to the context window #51021 applied.

  • Direct Bedrock checks on us.anthropic.claude-haiku-4-5:

    maxTokens Budget Result
    20,000 31,999 400, "max_tokens must be greater than thinking.budget_tokens"
    20,000 10,000 200
  • Live through opencode, with feat(core): fit output limits to the context window #51021 applied and a 20,000 summary limit to simulate automatic compaction on a nearly full context:

    • Main steps: maxTokens: 64000 with thinking.budget_tokens: 31999.
    • Compaction: maxTokens: 20000 with thinking.budget_tokens: 10000. The summary completed.

Screenshots / recordings

N/A

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@rekram1-node
rekram1-node merged commit 20610e6 into v2 Sep 24, 2026
10 checks passed
@rekram1-node
rekram1-node deleted the bedrock-thinking-fit branch September 24, 2026 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant