Repository navigation
fix(ai): honor Anthropic cache format on Bedrock Converse - #55942
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
The change looks good. A model whose ID is an opaque application inference profile ARN now gets cache points when it declares compatibility.cacheFormat: "anthropic", and models detected by ID behave as before. I checked it by running the new tests on the base branch: the two ARN cases fail there (no cachePoint blocks) and pass with this change. All Bedrock suites in packages/ai pass (245), and typecheck passes. I couldn't run a full CLI round-trip against a mock Bedrock server in this sandbox, so I checked it at the request-building level only.
Public API: unchanged. No new config keys; the existing compatibility.cacheFormat is now respected on Bedrock Converse.
No blocking concerns.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Related to #53756. This fixes the caching part only. The Claude reasoning-variant problem with inference profile ARNs is still open, so this PR does not close the issue.
Type of change
What does this PR do?
Bedrock Converse decides whether to emit
cachePointblocks only from the model ID (anthropic.claude-). An application inference profile ARN (arn:aws:bedrock:…:application-inference-profile/…) has no model name in it, so ARN-backed Claude models never cache. This was an intended trade-off in #47660, but it leaves no way to opt in.Models already have a
compatibility.cacheFormatoverride. The cache policy reads it and places the hints, butBedrockCache.blockdropped them all because the ID check failed. NowBedrockCache.breakpointstakes the model and treatscacheFormat: "anthropic"as supported. This follows the same pattern as thesupportsThinkingBlockBindingoverride inclaude-model.ts. Models detected by ID are unaffected, and"alibaba"is ignored on Bedrock.Users can opt in with:
Known limit: an opaque ARN can't be recognized as one of the 5-minute-only legacy Claude models, so 1h TTL stays allowed. The default policy sends no TTL, so this only matters if a caller explicitly asks for a TTL of 3600s or more on an ARN backed by an old model.
How did you verify your code works?
bedrock-converse-cache.test.ts:"alibaba"emits none.bun teston the Bedrock Converse, Mantle and cache-policy suites: 247 pass.bun typecheckpasses inpackages/ai.Screenshots / recordings
N/A
Checklist
Added by the opencode bot (AI-generated).
Fixes #55938
Related: #53756