Repository navigation
Description
Prompt caching for Mistral is silently disabled: @ai-sdk/mistral 3.x drops promptCacheKey, and 4.x requires a spec-v4 core
Summary
Mistral's API supports explicit prompt caching via a prompt_cache_key body parameter (cached tokens billed at 10% of the input price, hits reported in usage.prompt_tokens_details.cached_tokens). opencode already does the right thing on its side — packages/opencode/src/provider/transform.ts (~line 1321) sets result["promptCacheKey"] = input.sessionID for @ai-sdk/mistral — but the parameter never reaches the wire, and today no provider combination can enable caching for Mistral reasoning models.
This affects mistral-large-latest, mistral-medium-*, magistral-*, and the relayed GLM models (zai-glm-5, zai-glm-5-2, zai-glm-5-3, zai-glm-latest) — i.e. any workload where long conversation histories are resent every turn, which is exactly the workload prompt caching is meant for.
The blocker chain (verified)
-
@ai-sdk/mistral3.x silently dropspromptCacheKey. The chat language model parses per-call provider options strictly againstmistralLanguageModelOptions(safePrompt,structuredOutputs,strictJsonSchema,documentImageLimit,documentPageLimit,parallelToolCalls,reasoningEffort...).promptCacheKeyis not in that schema, so it is stripped byparseProviderOptionsandprompt_cache_keyis never included in the request body. opencode pins@ai-sdk/[email protected](bundled 3.0.51 in the current desktop app; fix(mistral): bump @ai-sdk/mistral to 3.0.64 to unbreak streamed tool calls #49689 bumps to 3.0.64 — same limitation). -
@ai-sdk/mistral4.x fixes it but cannot be loaded. As of@ai-sdk/[email protected],mistralLanguageModelOptionsincludespromptCacheKey: z.string().optional()andgetArgsmaps it toprompt_cache_keyin the body. However, 4.x declaresspecificationVersion = "v4", and opencode's[email protected]core rejects anything other than v2/v3 inresolveLanguageModel()withUnsupportedModelVersionError(the v2-compat shim only handles "v2" and "v3"). So pinning"npm": "@ai-sdk/[email protected]"via config fails at model resolution. -
@ai-sdk/openai-compatiblecan injectprompt_cache_keybut breaks on Mistral reasoning models. The unknown-key passthrough (providerOptions[providerOptionsName]entries not inopenaiCompatibleLanguageModelChatOptions.shapego verbatim into the request body) makes"options": { "prompt_cache_key": ... }work at the config level. But for reasoning models:- History: assistant
reasoningparts are serialized as a flatreasoning_contentfield on the message. Mistral's API rejects it with HTTP 422:{"detail":[{"type":"extra_forbidden","loc":["body","messages",1,"assistant","reasoning_content"],"msg":"Extra inputs are not permitted"}]}. This 422s every multi-turn session that contains reasoning. By contrast,@ai-sdk/mistralserializes history reasoning as native thinking content blocks ({"type":"thinking","thinking":[{"type":"text",...}],"closed":true}inside thecontentarray), which Mistral accepts. - Stream chunks: Mistral reasons GLM/Magistral through native
thinkingcontent blocks in streamed chunkcontentarrays;@ai-sdk/[email protected]declarescontent: z.string()and throwsAI_TypeValidationError: expected string, received array. Fixed upstream in 2.0.75+ (chunkcontentis nowunion(string, array<looseObject>)with thinking-block → reasoning conversion), see Bump bundled @ai-sdk/openai-compatible 2.0.41 -> 2.0.75 (fixes fragmented reasoning streams) #50290.
- History: assistant
Evidence (reproduced against https://api.eu.mistral.ai/v1, model zai-glm-5-3)
# prompt_cache_key works: two identical requests
# 1st: usage.prompt_tokens_details.cached_tokens = 0
# 2nd: usage.prompt_tokens_details.cached_tokens = 640 (of 751 prompt tokens)
# history reasoning as reasoning_content -> 422
{"detail":[{"type":"extra_forbidden","loc":["body","messages",1,"assistant","reasoning_content"],"msg":"Extra inputs are not permitted"}]}
# history reasoning as native thinking content blocks -> 200Ask
Upgrade the ai core to a spec-v4-capable release (AI SDK line that accepts LanguageModelV4) and bump @ai-sdk/mistral to the 4.x line (4.0.48 verified: sends prompt_cache_key = session ID, i.e. the per-conversation key Mistral's docs recommend). That would turn the existing promptCacheKey = input.sessionID in transform.ts into working, per-session prompt caching for all Mistral models, with no config changes required by users.
Related: #49692 / #49689 (GLM streamed tool-call fix, @ai-sdk/mistral 3.0.64), #50290 (bundled @ai-sdk/openai-compatible lag), #50338 (v2/v3 provider handling).
Activity
opencode-agent commentedon Oct 9, 2026
Thanks for the detailed write-up. Closing this one, because the bundled
@ai-sdk/mistral3.0.51 that OpenCode installs is patched and already sendsprompt_cache_key. The upstream 3.0.51 package does drop it, as described here, but that isn't the code OpenCode runs.v1
patches/@ai-sdk%[email protected]addspromptCacheKeyto the Mistral options and maps it toprompt_cache_keyin the request body. The patch is applied throughpackage.json. It also sends reasoning history back as nativethinkingcontent blocks, notreasoning_content. Both changes landed in fix(provider): preserve Mistral reasoning history #38453. fix(mistral): pass through reasoning effort #42164 later letreasoningEffortpass through as a string.transform.tssetspromptCacheKey = sessionIDfor@ai-sdk/mistral, so every session gets its own cache key.provider-mistral.test.tschecks thatprompt_cache_keyreaches the request body.v2
prompt_cache_keyunless caching is turned off. It also handles reasoning as native thinking content, both when sending history back and when parsing streamed chunks.The 422 and "expected string, received array" failures described here only happen when Mistral models go through
@ai-sdk/openai-compatible. Models configured with@ai-sdk/mistraldon't hit them. We also switched the OpenCode Zenmistral-large-4entry to the Mistral SDK in models.dev (anomalyco/models.dev#9200). The separate openai-compatible version bump is tracked in #50290.If caching still isn't working for you on the current version with a
@ai-sdk/mistralmodel, please reopen and include the request body orusage.prompt_tokens_details.blackcodefr commentedon Oct 9, 2026
At the time of writing it worked with GLM-5-2 but not GLM-5-3 on the same Mistral API endpoint. Will try again with the default mistral sdk and let you know.