Skip to content

Commit f5aa3e1

Browse files
committed
fix(bedrock): fail closed on unlisted models and unpriced 1h tiers
What: - bedrock: clear `promptCacheTtlSupport` in the `guessModelInfoFromId` fallback. The fallback spreads `bedrockModels[bedrockDefaultModelId]`, which is 1h-capable, and then overlays the guess, so an id that resolves to no catalog row inherited `["5m","1h"]` and could emit `ttl: "1h"` for a model we cannot prove accepts it. An unlisted id is now 5m-only (ADR-0006 I2). - bedrock-cache-ttl.spec: add the regression case for that path, and make the tier-rate invariant fail when a tier reprices `inputPrice` without a `cacheWritesPrice1h` instead of skipping it. Why: CodeRabbit round 1 on PR #2270, threads KJYW (Major) and KJYK (Minor). The KJYW premise is real: a custom inference-profile ARN naming an unlisted model (for example `us.anthropic.claude-3-5-sonnet-20241022-v1:0`, which has no catalog row) produced `ttlSupport: ["5m","1h"]` on head. Bedrock answers an unsupported ttl with a ValidationException that kills the turn, so this was a turn-breaking path rather than a pricing nit. Impact: No change for the 48 catalog rows or for any listed ARN, inference profile, or prompt-router path: only the unresolvable-id fallback loses capability. The tier guard is a test-only tightening that currently passes on all four tiers. Red-on-head: with the production hunk reverted and the new test kept, `bedrock-cache-ttl.spec.ts` fails exactly one case ("emits no ttl for an unlisted model that the heuristic fallback guesses") and passes the other 24. With the tier guard reverted and a tier 1h rate temporarily removed from the catalog, the catalog invariant fails on `anthropic.claude-sonnet-4-6 tier 0`.
1 parent 281c22a commit f5aa3e1

2 files changed

Lines changed: 39 additions & 9 deletions

File tree

‎src/api/providers/__tests__/bedrock-cache-ttl.spec.ts‎

Lines changed: 31 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -314,6 +314,26 @@ describe("Bedrock cache-point TTL emission", () => {
314314
expect(containsTtlKey(input)).toBe(false)
315315
})
316316

317+
it("emits no ttl for an unlisted model that the heuristic fallback guesses", async () => {
318+
// The fallback spreads the default row (which is 1h-capable) and then overlays
319+
// the guess, so the capability is inherited unless it is cleared. An unlisted
320+
// id is exactly the case where we cannot prove the model accepts 1h, and Bedrock
321+
// answers an unsupported ttl with a ValidationException that kills the turn.
322+
const handler = createHandler({
323+
awsCustomArn:
324+
"arn:aws:bedrock:us-east-1:123456789012:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v1:0",
325+
promptCacheTtl: "1h",
326+
})
327+
handler["client"].send = vi.fn().mockResolvedValue({ stream: asyncStreamFrom([]) })
328+
329+
const input = await captureConverseInput(handler)
330+
331+
// The guess still reports caching support, so cache points exist and the
332+
// assertion is about the capability gate rather than caching being off.
333+
expect(collectCachePoints(input).length).toBeGreaterThan(0)
334+
expect(containsTtlKey(input)).toBe(false)
335+
})
336+
317337
it("emits 1h for a foundation-model ARN that names a listed model", async () => {
318338
const handler = createHandler({
319339
awsCustomArn: "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-sonnet-4-5-20250929-v1:0",
@@ -538,9 +558,17 @@ describe("Bedrock cache TTL capability catalog", () => {
538558
}
539559

540560
for (const [index, tier] of (entry.tiers ?? []).entries()) {
541-
if (tier.cacheWritesPrice1h === undefined) continue
542-
expect(tier.inputPrice, `${id} tier ${index}`).toBeDefined()
543-
expect(tier.cacheWritesPrice1h, `${id} tier ${index}`).toBeCloseTo((tier.inputPrice as number) * 2, 9)
561+
// A tier that reprices input without repricing the 1h write is exactly the
562+
// shadow this test exists to catch, so it must fail rather than be skipped.
563+
if (tier.inputPrice === undefined) {
564+
expect(
565+
tier.cacheWritesPrice1h,
566+
`${id} tier ${index} prices a 1h write without an input price`,
567+
).toBeUndefined()
568+
continue
569+
}
570+
expect(tier.cacheWritesPrice1h, `${id} tier ${index} reprices input but not the 1h write`).toBeDefined()
571+
expect(tier.cacheWritesPrice1h, `${id} tier ${index}`).toBeCloseTo(tier.inputPrice * 2, 9)
544572
}
545573
}
546574
})

‎src/api/providers/bedrock.ts‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1247,13 +1247,15 @@ export class AwsBedrockHandler extends BaseProvider implements SingleCompletionH
12471247
} else {
12481248
// Use heuristics for model info, then allow overrides from ProviderSettings
12491249
const guessed = this.guessModelInfoFromId(modelId)
1250-
model = {
1251-
id: bedrockDefaultModelId,
1252-
info: {
1253-
...JSON.parse(JSON.stringify(bedrockModels[bedrockDefaultModelId])),
1254-
...guessed,
1255-
},
1250+
const info: ModelInfo = {
1251+
...JSON.parse(JSON.stringify(bedrockModels[bedrockDefaultModelId])),
1252+
...guessed,
12561253
}
1254+
// The default row is 1h-capable, but an id we cannot resolve is not. Inheriting
1255+
// the capability would send `ttl: "1h"` to a model that may reject it with a
1256+
// ValidationException that kills the turn, so an unlisted id fails closed (I2).
1257+
delete info.promptCacheTtlSupport
1258+
model = { id: bedrockDefaultModelId, info }
12571259
}
12581260

12591261
// Always allow user to override detected/guessed maxTokens and contextWindow

0 commit comments

Comments
 (0)