Skip to content

Prompt caching for Mistral is silently disabled: @ai-sdk/mistral 3.x drops promptCacheKey, and 4.x requires a spec-v4 core #50748

Closed

Description

@blackcodefr

Prompt caching for Mistral is silently disabled: @ai-sdk/mistral 3.x drops promptCacheKey, and 4.x requires a spec-v4 core

Summary

Mistral's API supports explicit prompt caching via a prompt_cache_key body parameter (cached tokens billed at 10% of the input price, hits reported in usage.prompt_tokens_details.cached_tokens). opencode already does the right thing on its side — packages/opencode/src/provider/transform.ts (~line 1321) sets result["promptCacheKey"] = input.sessionID for @ai-sdk/mistral — but the parameter never reaches the wire, and today no provider combination can enable caching for Mistral reasoning models.

This affects mistral-large-latest, mistral-medium-*, magistral-*, and the relayed GLM models (zai-glm-5, zai-glm-5-2, zai-glm-5-3, zai-glm-latest) — i.e. any workload where long conversation histories are resent every turn, which is exactly the workload prompt caching is meant for.

The blocker chain (verified)

  1. @ai-sdk/mistral 3.x silently drops promptCacheKey. The chat language model parses per-call provider options strictly against mistralLanguageModelOptions (safePrompt, structuredOutputs, strictJsonSchema, documentImageLimit, documentPageLimit, parallelToolCalls, reasoningEffort...). promptCacheKey is not in that schema, so it is stripped by parseProviderOptions and prompt_cache_key is never included in the request body. opencode pins @ai-sdk/[email protected] (bundled 3.0.51 in the current desktop app; fix(mistral): bump @ai-sdk/mistral to 3.0.64 to unbreak streamed tool calls #49689 bumps to 3.0.64 — same limitation).

  2. @ai-sdk/mistral 4.x fixes it but cannot be loaded. As of @ai-sdk/[email protected], mistralLanguageModelOptions includes promptCacheKey: z.string().optional() and getArgs maps it to prompt_cache_key in the body. However, 4.x declares specificationVersion = "v4", and opencode's [email protected] core rejects anything other than v2/v3 in resolveLanguageModel() with UnsupportedModelVersionError (the v2-compat shim only handles "v2" and "v3"). So pinning "npm": "@ai-sdk/[email protected]" via config fails at model resolution.

  3. @ai-sdk/openai-compatible can inject prompt_cache_key but breaks on Mistral reasoning models. The unknown-key passthrough (providerOptions[providerOptionsName] entries not in openaiCompatibleLanguageModelChatOptions.shape go verbatim into the request body) makes "options": { "prompt_cache_key": ... } work at the config level. But for reasoning models:

    • History: assistant reasoning parts are serialized as a flat reasoning_content field on the message. Mistral's API rejects it with HTTP 422: {"detail":[{"type":"extra_forbidden","loc":["body","messages",1,"assistant","reasoning_content"],"msg":"Extra inputs are not permitted"}]}. This 422s every multi-turn session that contains reasoning. By contrast, @ai-sdk/mistral serializes history reasoning as native thinking content blocks ({"type":"thinking","thinking":[{"type":"text",...}],"closed":true} inside the content array), which Mistral accepts.
    • Stream chunks: Mistral reasons GLM/Magistral through native thinking content blocks in streamed chunk content arrays; @ai-sdk/[email protected] declares content: z.string() and throws AI_TypeValidationError: expected string, received array. Fixed upstream in 2.0.75+ (chunk content is now union(string, array<looseObject>) with thinking-block → reasoning conversion), see Bump bundled @ai-sdk/openai-compatible 2.0.41 -> 2.0.75 (fixes fragmented reasoning streams) #50290.

Evidence (reproduced against https://api.eu.mistral.ai/v1, model zai-glm-5-3)

# prompt_cache_key works: two identical requests
# 1st: usage.prompt_tokens_details.cached_tokens = 0
# 2nd: usage.prompt_tokens_details.cached_tokens = 640 (of 751 prompt tokens)

# history reasoning as reasoning_content -> 422
{"detail":[{"type":"extra_forbidden","loc":["body","messages",1,"assistant","reasoning_content"],"msg":"Extra inputs are not permitted"}]}

# history reasoning as native thinking content blocks -> 200

Ask

Upgrade the ai core to a spec-v4-capable release (AI SDK line that accepts LanguageModelV4) and bump @ai-sdk/mistral to the 4.x line (4.0.48 verified: sends prompt_cache_key = session ID, i.e. the per-conversation key Mistral's docs recommend). That would turn the existing promptCacheKey = input.sessionID in transform.ts into working, per-session prompt caching for all Mistral models, with no config changes required by users.

Related: #49692 / #49689 (GLM streamed tool-call fix, @ai-sdk/mistral 3.0.64), #50290 (bundled @ai-sdk/openai-compatible lag), #50338 (v2/v3 provider handling).

Activity

  1. opencode-agent commented on Oct 9, 2026

    @opencode-agent
    Contributor

    Thanks for the detailed write-up. Closing this one, because the bundled @ai-sdk/mistral 3.0.51 that OpenCode installs is patched and already sends prompt_cache_key. The upstream 3.0.51 package does drop it, as described here, but that isn't the code OpenCode runs.

    v1

    v2

    The 422 and "expected string, received array" failures described here only happen when Mistral models go through @ai-sdk/openai-compatible. Models configured with @ai-sdk/mistral don't hit them. We also switched the OpenCode Zen mistral-large-4 entry to the Mistral SDK in models.dev (anomalyco/models.dev#9200). The separate openai-compatible version bump is tracked in #50290.

    If caching still isn't working for you on the current version with a @ai-sdk/mistral model, please reopen and include the request body or usage.prompt_tokens_details.

  2. blackcodefr commented on Oct 9, 2026

    @blackcodefr
    Author

    At the time of writing it worked with GLM-5-2 but not GLM-5-3 on the same Mistral API endpoint. Will try again with the default mistral sdk and let you know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions