Skip to content

fix(ai): honor Anthropic cache format on Bedrock Converse - #55942

Merged
rekram1-node merged 1 commit into
devfrom
bedrock-arn-variants
Oct 16, 2026
Merged

rekram1-node merged 1 commit into
devfrom
bedrock-arn-variants

Conversation

@rekram1-node

@rekram1-node rekram1-node commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Issue for this PR

Related to #53756. This fixes the caching part only. The Claude reasoning-variant problem with inference profile ARNs is still open, so this PR does not close the issue.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Bedrock Converse decides whether to emit cachePoint blocks only from the model ID (anthropic.claude-). An application inference profile ARN (arn:aws:bedrock:…:application-inference-profile/…) has no model name in it, so ARN-backed Claude models never cache. This was an intended trade-off in #47660, but it leaves no way to opt in.

Models already have a compatibility.cacheFormat override. The cache policy reads it and places the hints, but BedrockCache.block dropped them all because the ID check failed. Now BedrockCache.breakpoints takes the model and treats cacheFormat: "anthropic" as supported. This follows the same pattern as the supportsThinkingBlockBinding override in claude-model.ts. Models detected by ID are unaffected, and "alibaba" is ignored on Bedrock.

Users can opt in with:

"team-sonnet": {
  "modelID": "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123",
  "compatibility": { "cacheFormat": "anthropic" }
}

Known limit: an opaque ARN can't be recognized as one of the 5-minute-only legacy Claude models, so 1h TTL stays allowed. The default policy sends no TTL, so this only matters if a caller explicitly asks for a TTL of 3600s or more on an ARN backed by an old model.

How did you verify your code works?

  • New tests in bedrock-converse-cache.test.ts:
    • An ARN with the override gets cache points on tools, system and messages, with both the default and a 1h TTL.
    • "alibaba" emits none.
    • Both tests fail without the source change.
  • The existing tests still cover an ARN without the override and non-Claude IDs getting no cache points.
  • bun test on the Bedrock Converse, Mantle and cache-policy suites: 247 pass. bun typecheck passes in packages/ai.
  • Not tested against a real application inference profile.

Screenshots / recordings

N/A

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Added by the opencode bot (AI-generated).

Fixes #55938

Related: #53756

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks good. A model whose ID is an opaque application inference profile ARN now gets cache points when it declares compatibility.cacheFormat: "anthropic", and models detected by ID behave as before. I checked it by running the new tests on the base branch: the two ARN cases fail there (no cachePoint blocks) and pass with this change. All Bedrock suites in packages/ai pass (245), and typecheck passes. I couldn't run a full CLI round-trip against a mock Bedrock server in this sandbox, so I checked it at the request-building level only.

Public API: unchanged. No new config keys; the existing compatibility.cacheFormat is now respected on Bedrock Converse.

No blocking concerns.

Fixes #55938
Related: #53756

@rekram1-node
rekram1-node merged commit b0a8e25 into dev Oct 16, 2026
23 checks passed
@rekram1-node
rekram1-node deleted the bedrock-arn-variants branch October 16, 2026 04:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bedrock: prompt caching silently off when modelID is an application inference profile ARN

1 participant