Repository navigation
setCacheKey sends promptCacheKey (wrong) instead of cache_control on content blocks for openai-compatible Bedrock proxies (Bifrost, LiteLLM) #25984
Description
Description
When setCacheKey: true is configured on an @ai-sdk/openai-compatible provider that routes to AWS Bedrock Claude models (via proxies like Bifrost or LiteLLM), OpenCode sends a promptCacheKey request option — which these proxies ignore entirely. No prompt caching occurs.
Root cause in transform.ts → options() (~line 1008):
if (input.model.providerID === "openai" || input.providerOptions?.setCacheKey) {
result["promptCacheKey"] = input.sessionID
}promptCacheKey is an OpenAI-native caching hint. Bifrost and LiteLLM (when routing to Bedrock/Anthropic) require cache_control: { type: "ephemeral" } injected as a property on message content blocks — the Anthropic-style caching format that both proxies accept and translate to the native backend format.
applyCaching() already knows how to inject openaiCompatible: { cache_control: { type: "ephemeral" } } onto content blocks, but message() never calls it for @ai-sdk/openai-compatible providers. Additionally, string system messages fall through to message-level providerOptions injection (not content-block level), which the @ai-sdk/openai-compatible SDK spreads as a top-level message field rather than a content block property — Bifrost ignores this for caching.
Correct wire format (required by Bifrost and LiteLLM for Bedrock/Anthropic routing):
{
"messages": [
{
"role": "system",
"content": [
{ "type": "text", "text": "...", "cache_control": { "type": "ephemeral" } }
]
},
{
"role": "user",
"content": [
{ "type": "text", "text": "...", "cache_control": { "type": "ephemeral" } }
]
}
]
}Plugins
None
OpenCode version
Latest (dev branch)
Steps to reproduce
- Configure an
@ai-sdk/openai-compatibleprovider pointing to Bifrost or LiteLLM routing to Bedrock Claude:
{
"provider": {
"bifrost": {
"npm": "@ai-sdk/openai-compatible",
"options": {
"baseURL": "http://localhost:24242/v1",
"setCacheKey": true
},
"models": {
"claude-sonnet-4-6": {
"id": "bedrock/global.anthropic.claude-sonnet-4-6"
}
}
}
}
}- Start a session and send a message with a large system prompt.
- Inspect the outgoing request body (e.g., via Bifrost debug logs or a proxy like Wireshark/mitmproxy).
- Observe: request contains
promptCacheKeyat top level — nocache_controlon content blocks. - Expected:
cache_control: { type: "ephemeral" }on the last content block of the system message and targeted user messages.
Screenshot and/or share link
PR with fix: #25985
Operating System
macOS 15.4 (Darwin arm64)
Terminal
Ghostty
Activity
github-actions commentedon May 6, 2026
This issue doesn't fully meet our contributing guidelines.
What needs to be fixed:
[FEATURE]:), or Question. For this issue, the Bug Report template is appropriate — it requires a Description field, steps to reproduce, and system information.Please edit this issue to address the above within 2 hours, or it will be automatically closed.
This issue might be a duplicate of existing issues. Please check:
setCacheKeynot working properly for caching with@ai-sdk/anthropicprovider behind a proxy, using the samecache_controlmechanismtransform.tsIf you believe this was flagged incorrectly, please let a maintainer know.
[-]fix: setCacheKey sends promptCacheKey (wrong) instead of cache_control on content blocks for openai-compatible Bedrock proxies (Bifrost, LiteLLM)[/-][+]setCacheKey sends promptCacheKey (wrong) instead of cache_control on content blocks for openai-compatible Bedrock proxies (Bifrost, LiteLLM)[/+]github-actions commentedon May 6, 2026
Thanks for updating the issue! It now includes all the required information — clear description, root cause analysis, steps to reproduce, and system details. The
needs:compliancelabel has been removed.jairbubbles commentedon May 8, 2026
I'm investigating a similar when using Databricks, it also expects to see
"cache_control": { "type": "ephemeral" }inmessages.content.https://docs.databricks.com/aws/en/machine-learning/model-serving/score-foundation-models#prompt-cache
xenstar commentedon May 9, 2026
Same issue for Xiaomi Mimo direct API. It doesn't cache at all. And it burns the credits super fast. When using openrouter (xiaomi mimo 2.5), almost 95% messages are cached. It needs to be fixed.
github-actions commentedon Jul 9, 2026
To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.
NetBr3ak commentedon Aug 22, 2026
promptCacheKey tells the provider where to route the request. cache_control is
what marks the prefix as cacheable. Send the first where the second is expected
and every request comes back fine while nothing gets cached at all.
A read costs 0.1 of a normal input token. So on a stable prefix that mistake
means paying about ten times what you should, on every repeated token, until
somebody reads the raw request body.
Translation layers keep doing this. There are open issues right now with the
same shape against an Anthropic to Bedrock translator in one of the open source
gateways, and against a vendor orchestration layer in a big enterprise
suite.