Skip to content

Commit baea820

Browse files
ruben-benaconh2
authored andcommitted
fix: raise stale output ceilings on adaptive-thinking Claude models
LLM: written by Claude Opus 5; needs human review getModelMaxOutputTokens capped adaptive-thinking Claude models at ANTHROPIC_DEFAULT_MAX_TOKENS (8192) with reasoning off and DEFAULT_HYBRID_REASONING_MODEL_MAX_TOKENS (16_384) with it on. Thinking tokens count toward max_tokens, so thinking could consume the whole budget, and Bedrock returned empty, fully billed turns with stop_reason max_tokens. Reasoning-off is not thinking-off on most of these models: Omitting the thinking field selects adaptive thinking on Opus 5 and Sonnet 5, and Fable 5 / 5.1 and Opus 5.5 reject thinking type "disabled" outright. Opus 4.7 and 4.8 are the exception: they don't think unless adaptive thinking is requested, so with reasoning off 8192 was not too small for them, but with reasoning on 16_384 was. Resolve supportsReasoningBinary models from their own declared ceiling in both reasoning states, still subject to the 20% context-window clamp. For Opus 4.7 / 4.8 with reasoning off this only raises the max_tokens ceiling; only generated tokens are billed. Budget-controlled hybrid models (Sonnet 4.x, Opus 4.6 and earlier) keep the 8192 default; that branch is correct for them. Neither constant changed, only which models reach them. Both fixed parts of this for Opus 5.5 only, per provider: * Zoo-Code-Org#1854 (closing Zoo-Code-Org#1853) fixed direct Anthropic, by dropping supportsReasoningBudget from that one entry. * Zoo-Code-Org#1903 made Bedrock Opus 5.5 user-adjustable via supportsMaxTokens, but explicitly kept the request defaults; the entry still declares supportsReasoningBudget, so it still defaults to 16_384 / 8192. Upstream (as of v3.88.0), Vertex Opus 5.5 still defaults to 16_384 / 8192 and is not adjustable at all, and every sibling adaptive-thinking model still gets the hybrid defaults on Bedrock and Vertex. Keying on supportsReasoningBinary instead fixes all of them without touching each entry's supportsReasoningBudget flag. * Bedrock: claude-fable-5 maxTokens 8192 -> 128_000, matching its anthropicModels twin and the other entries. claude-opus-4-7 and claude-opus-4-8 gain supportsReasoningBinary: isAdaptiveThinkingModel already treats them as adaptive-thinking by model ID, so without the flag they bypassed the new branch. * Vertex: claude-sonnet-5, claude-opus-4-7, claude-opus-4-8, claude-opus-5 and claude-fable-5 maxTokens 8192 -> 128_000. * Both: supportsMaxTokens on every adaptive-thinking entry still missing it (Bedrock: opus-4-7, opus-4-8, fable-5, fable-5-1; Vertex: the five above plus opus-5-5 and fable-5-1), so the output budget is user-adjustable everywhere, as on Anthropic direct. bedrock-models.test.ts pins this with generic sweeps over each registry (every supportsReasoningBinary entry shares the 128K ceiling and is adjustable) plus per-model tables, so a new copy-pasted entry with the old 8192 default fails a test. Also updates the spec assertions that pinned the old 8192 / 16_384, including Bedrock and Anthropic Opus 5.5's resolved budget.
1 parent 46768bb commit baea820

11 files changed

Lines changed: 532 additions & 197 deletions

File tree

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
import type { ModelInfo } from "../model.js"
2+
import { bedrockModels } from "../providers/bedrock.js"
3+
import { vertexModels } from "../providers/vertex.js"
4+
import { anthropicModels } from "../providers/anthropic.js"
5+
6+
// The Bedrock and Vertex registries are hand-maintained, so entries are copy-pasted and the
7+
// pre-adaptive-thinking `maxTokens: 8192` default silently drifts onto new Claude models.
8+
const REFERENCE_MODEL_ID = "anthropic.claude-fable-5-1"
9+
const VERTEX_REFERENCE_MODEL_ID = "claude-fable-5-1"
10+
const ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS = 128_000
11+
12+
describe("bedrockModels adaptive-thinking output ceilings", () => {
13+
it("keeps the reference model at the documented 128K ceiling", () => {
14+
expect(bedrockModels[REFERENCE_MODEL_ID].maxTokens).toBe(ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS)
15+
})
16+
17+
it.each([
18+
["anthropic.claude-opus-5", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
19+
["anthropic.claude-opus-5-5", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
20+
["anthropic.claude-sonnet-5", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
21+
["anthropic.claude-fable-5", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
22+
["anthropic.claude-fable-5-1", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
23+
["anthropic.claude-opus-4-7", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
24+
["anthropic.claude-opus-4-8", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
25+
] as const)("declares maxTokens %s = %i", (modelId, expected) => {
26+
expect(bedrockModels[modelId].maxTokens).toBe(expected)
27+
})
28+
29+
it("gives every adaptive-thinking model the same ceiling as the reference model", () => {
30+
// Widening to ModelInfo: the const literal's inferred type only carries
31+
// `supportsReasoningBinary` on the members that declare it.
32+
const entries = Object.entries(bedrockModels as Record<string, ModelInfo>)
33+
const adaptiveThinkingEntries = entries.filter(([, info]) => info.supportsReasoningBinary === true)
34+
35+
// If the flag is ever renamed, an empty list must not pass silently.
36+
expect(adaptiveThinkingEntries.length).toBeGreaterThanOrEqual(4)
37+
38+
for (const [modelId, info] of adaptiveThinkingEntries) {
39+
expect(info.maxTokens, `${modelId} must share the adaptive-thinking output ceiling`).toBe(
40+
ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS,
41+
)
42+
}
43+
})
44+
45+
it.each([
46+
["anthropic.claude-opus-5", "claude-opus-5"],
47+
["anthropic.claude-opus-5-5", "claude-opus-5-5"],
48+
["anthropic.claude-sonnet-5", "claude-sonnet-5"],
49+
["anthropic.claude-fable-5", "claude-fable-5"],
50+
["anthropic.claude-fable-5-1", "claude-fable-5-1"],
51+
["anthropic.claude-opus-4-7", "claude-opus-4-7"],
52+
["anthropic.claude-opus-4-8", "claude-opus-4-8"],
53+
] as const)("matches the Anthropic direct-API ceiling for %s", (bedrockId, anthropicId) => {
54+
expect(bedrockModels[bedrockId].maxTokens).toBe(anthropicModels[anthropicId].maxTokens)
55+
})
56+
57+
it.each([
58+
["anthropic.claude-opus-4-7"],
59+
["anthropic.claude-opus-4-8"],
60+
["anthropic.claude-opus-5"],
61+
["anthropic.claude-opus-5-5"],
62+
["anthropic.claude-sonnet-5"],
63+
["anthropic.claude-fable-5"],
64+
["anthropic.claude-fable-5-1"],
65+
] as const)("marks %s as binary-reasoning", (modelId) => {
66+
expect(bedrockModels[modelId].supportsReasoningBinary).toBe(true)
67+
})
68+
69+
it("lets the user override the output budget on every adaptive-thinking model", () => {
70+
const entries = Object.entries(bedrockModels as Record<string, ModelInfo>)
71+
const adaptiveThinkingEntries = entries.filter(([, info]) => info.supportsReasoningBinary === true)
72+
73+
expect(adaptiveThinkingEntries.length).toBeGreaterThanOrEqual(4)
74+
75+
for (const [modelId, info] of adaptiveThinkingEntries) {
76+
expect(info.supportsMaxTokens, `${modelId} must expose a configurable output budget`).toBe(true)
77+
}
78+
})
79+
})
80+
81+
// Vertex is a second hand-maintained registry that drifts independently of Bedrock.
82+
describe("vertexModels adaptive-thinking output ceilings", () => {
83+
it("keeps the reference model at the documented 128K ceiling", () => {
84+
expect(vertexModels[VERTEX_REFERENCE_MODEL_ID].maxTokens).toBe(ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS)
85+
})
86+
87+
it.each([
88+
["claude-opus-5", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
89+
["claude-opus-5-5", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
90+
["claude-sonnet-5", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
91+
["claude-fable-5", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
92+
["claude-fable-5-1", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
93+
["claude-opus-4-7", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
94+
["claude-opus-4-8", ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS],
95+
] as const)("declares maxTokens %s = %i", (modelId, expected) => {
96+
expect(vertexModels[modelId].maxTokens).toBe(expected)
97+
})
98+
99+
it("gives every adaptive-thinking model the same ceiling as the reference model", () => {
100+
const entries = Object.entries(vertexModels as Record<string, ModelInfo>)
101+
const adaptiveThinkingEntries = entries.filter(([, info]) => info.supportsReasoningBinary === true)
102+
103+
expect(adaptiveThinkingEntries.length).toBeGreaterThanOrEqual(4)
104+
105+
for (const [modelId, info] of adaptiveThinkingEntries) {
106+
expect(info.maxTokens, `${modelId} must share the adaptive-thinking output ceiling`).toBe(
107+
ADAPTIVE_THINKING_MAX_OUTPUT_TOKENS,
108+
)
109+
}
110+
})
111+
112+
it.each([
113+
["claude-opus-5"],
114+
["claude-opus-5-5"],
115+
["claude-sonnet-5"],
116+
["claude-fable-5"],
117+
["claude-fable-5-1"],
118+
["claude-opus-4-7"],
119+
["claude-opus-4-8"],
120+
] as const)("matches the Anthropic direct-API ceiling for %s", (modelId) => {
121+
expect(vertexModels[modelId].maxTokens).toBe(anthropicModels[modelId].maxTokens)
122+
})
123+
124+
it("lets the user override the output budget on every adaptive-thinking model", () => {
125+
const entries = Object.entries(vertexModels as Record<string, ModelInfo>)
126+
const adaptiveThinkingEntries = entries.filter(([, info]) => info.supportsReasoningBinary === true)
127+
128+
expect(adaptiveThinkingEntries.length).toBeGreaterThanOrEqual(4)
129+
130+
for (const [modelId, info] of adaptiveThinkingEntries) {
131+
expect(info.supportsMaxTokens, `${modelId} must expose a configurable output budget`).toBe(true)
132+
}
133+
})
134+
})

‎packages/types/src/providers/bedrock.ts‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,8 @@ export const bedrockModels = {
192192
supportsImages: true,
193193
supportsPromptCache: true,
194194
supportsReasoningBudget: true,
195+
supportsReasoningBinary: true,
196+
supportsMaxTokens: true,
195197
inputPrice: 5.0, // $5 per million input tokens (≤200K context) — verify against Bedrock console
196198
outputPrice: 25.0, // $25 per million output tokens (≤200K context) — verify against Bedrock console
197199
cacheWritesPrice: 6.25, // $6.25 per million tokens
@@ -216,6 +218,8 @@ export const bedrockModels = {
216218
supportsImages: true,
217219
supportsPromptCache: true,
218220
supportsReasoningBudget: true,
221+
supportsReasoningBinary: true,
222+
supportsMaxTokens: true,
219223
inputPrice: 5.0, // $5 per million input tokens (≤200K context) — verify against Bedrock console
220224
outputPrice: 25.0, // $25 per million output tokens (≤200K context) — verify against Bedrock console
221225
cacheWritesPrice: 6.25, // $6.25 per million tokens
@@ -284,6 +288,7 @@ export const bedrockModels = {
284288
supportsPromptCache: true,
285289
supportsReasoningBudget: true,
286290
supportsReasoningBinary: true,
291+
supportsMaxTokens: true,
287292
supportsTemperature: false,
288293
inputPrice: 10.0,
289294
outputPrice: 50.0,
@@ -296,12 +301,13 @@ export const bedrockModels = {
296301
"Claude Fable 5.1 extends Fable 5 with stronger long-running agentic coding, multistep research, and document work.",
297302
},
298303
"anthropic.claude-fable-5": {
299-
maxTokens: 8192,
304+
maxTokens: 128_000,
300305
contextWindow: 1_000_000,
301306
supportsImages: true,
302307
supportsPromptCache: true,
303308
supportsReasoningBudget: true,
304309
supportsReasoningBinary: true,
310+
supportsMaxTokens: true,
305311
supportsTemperature: false,
306312
inputPrice: 10.0,
307313
outputPrice: 50.0,

‎packages/types/src/providers/vertex.ts‎

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -382,7 +382,7 @@ export const vertexModels = {
382382
],
383383
},
384384
"claude-sonnet-5": {
385-
maxTokens: 8192,
385+
maxTokens: 128_000,
386386
contextWindow: 1_000_000, // 1M context window native (no beta header required)
387387
supportsImages: true,
388388
supportsPromptCache: true,
@@ -392,6 +392,7 @@ export const vertexModels = {
392392
cacheReadsPrice: 0.2, // $0.20 per million tokens (introductory pricing through Aug 31, 2026)
393393
supportsReasoningBudget: true,
394394
supportsReasoningBinary: true,
395+
supportsMaxTokens: true,
395396
supportsTemperature: false,
396397
description:
397398
"Claude Sonnet 5 is the best combination of speed and intelligence, optimized for coding, tool use, and agentic workflows.",
@@ -429,7 +430,7 @@ export const vertexModels = {
429430
],
430431
},
431432
"claude-opus-4-7": {
432-
maxTokens: 8192,
433+
maxTokens: 128_000,
433434
contextWindow: 200_000, // Default 200K, extendable to 1M with beta flag 'context-1m-2025-08-07'
434435
supportsImages: true,
435436
supportsPromptCache: true,
@@ -439,6 +440,7 @@ export const vertexModels = {
439440
cacheReadsPrice: 0.5, // $0.50 per million tokens
440441
supportsReasoningBudget: true,
441442
supportsReasoningBinary: true,
443+
supportsMaxTokens: true,
442444
supportsTemperature: false,
443445
// Tiered pricing for extended context (requires beta flag 'context-1m-2025-08-07')
444446
tiers: [
@@ -452,7 +454,7 @@ export const vertexModels = {
452454
],
453455
},
454456
"claude-opus-4-8": {
455-
maxTokens: 8192,
457+
maxTokens: 128_000,
456458
contextWindow: 200_000, // Default 200K, extendable to 1M with beta flag 'context-1m-2025-08-07'
457459
supportsImages: true,
458460
supportsPromptCache: true,
@@ -462,6 +464,7 @@ export const vertexModels = {
462464
cacheReadsPrice: 0.5, // $0.50 per million tokens
463465
supportsReasoningBudget: true,
464466
supportsReasoningBinary: true,
467+
supportsMaxTokens: true,
465468
supportsTemperature: false,
466469
// 4.8 inherits the same Vertex pricing structure as 4.7 — no breaking changes.
467470
// Tiered pricing for extended context (requires beta flag 'context-1m-2025-08-07')
@@ -476,7 +479,7 @@ export const vertexModels = {
476479
],
477480
},
478481
"claude-opus-5": {
479-
maxTokens: 8192,
482+
maxTokens: 128_000,
480483
contextWindow: 1_000_000, // 1M context window native (no beta header required)
481484
supportsImages: true,
482485
supportsPromptCache: true,
@@ -486,6 +489,7 @@ export const vertexModels = {
486489
cacheReadsPrice: 0.5, // $0.50 per million tokens
487490
supportsReasoningBudget: true,
488491
supportsReasoningBinary: true,
492+
supportsMaxTokens: true,
489493
supportsTemperature: false,
490494
description: "Claude Opus 5 is Anthropic's most capable model for complex agentic coding and enterprise work.",
491495
},
@@ -500,6 +504,7 @@ export const vertexModels = {
500504
cacheReadsPrice: 0.2, // $0.20 per million tokens
501505
supportsReasoningBudget: true,
502506
supportsReasoningBinary: true,
507+
supportsMaxTokens: true,
503508
supportsTemperature: false,
504509
description:
505510
"Claude Opus 5.5 is Anthropic's most capable model for complex agentic coding and enterprise work.",
@@ -515,12 +520,13 @@ export const vertexModels = {
515520
cacheReadsPrice: 0.25,
516521
supportsReasoningBudget: true,
517522
supportsReasoningBinary: true,
523+
supportsMaxTokens: true,
518524
supportsTemperature: false,
519525
description:
520526
"Claude Fable 5.1 extends Fable 5 with stronger long-running agentic coding, multistep research, and document work.",
521527
},
522528
"claude-fable-5": {
523-
maxTokens: 8192,
529+
maxTokens: 128_000,
524530
contextWindow: 1_000_000,
525531
supportsImages: true,
526532
supportsPromptCache: true,
@@ -530,6 +536,7 @@ export const vertexModels = {
530536
cacheReadsPrice: 1.0,
531537
supportsReasoningBudget: true,
532538
supportsReasoningBinary: true,
539+
supportsMaxTokens: true,
533540
supportsTemperature: false,
534541
description:
535542
"Claude Fable 5 is Anthropic's most capable widely released model for the most demanding reasoning and long-horizon agentic work.",

‎src/api/providers/__tests__/anthropic-vertex.spec.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -984,7 +984,7 @@ describe("VertexHandler", () => {
984984

985985
const model = handler.getModel()
986986
expect(model.id).toBe("claude-fable-5")
987-
expect(model.info.maxTokens).toBe(8192)
987+
expect(model.info.maxTokens).toBe(128_000)
988988
expect(model.info.contextWindow).toBe(1_000_000)
989989
expect(model.info.supportsReasoningBinary).toBe(true)
990990
expect(model.info.supportsReasoningBudget).toBe(true)
@@ -1019,7 +1019,7 @@ describe("VertexHandler", () => {
10191019

10201020
const model = handler.getModel()
10211021
expect(model.id).toBe("claude-sonnet-5")
1022-
expect(model.info.maxTokens).toBe(8192)
1022+
expect(model.info.maxTokens).toBe(128_000)
10231023
expect(model.info.contextWindow).toBe(1_000_000)
10241024
expect(model.info.supportsReasoningBinary).toBe(true)
10251025
expect(model.info.supportsReasoningBudget).toBe(true)
@@ -1036,7 +1036,7 @@ describe("VertexHandler", () => {
10361036

10371037
const model = handler.getModel()
10381038
expect(model.id).toBe("claude-opus-5")
1039-
expect(model.info.maxTokens).toBe(8192)
1039+
expect(model.info.maxTokens).toBe(128_000)
10401040
expect(model.info.contextWindow).toBe(1_000_000)
10411041
expect(model.info.supportsReasoningBinary).toBe(true)
10421042
expect(model.info.supportsReasoningBudget).toBe(true)

0 commit comments

Comments
 (0)