OpenCode Go: no prompt caching on deepseek-v4-flash and qwen3.8-max (glm-5.2 caches fine) #48116

Activity

1. github-actions commented on Sep 9, 2026

   

   github-actionsBot

   on Sep 9, 2026 – with GitHub Actions

   Contributor

   More actions

   This issue might be a duplicate of existing issues. Please check:

   • OpenCode Go endpoint: no prompt caching on DeepSeek (cache_hit always 0 even on repeated identical prompts) #41125: OpenCode Go endpoint: no prompt caching on DeepSeek (same symptom — cache_hit always 0 on repeated identical prompts via the Go endpoint; you already referenced this one)
   • OpenCode Go quota exhausted in ~20 minutes after DeepSeek V4 Flash cache reads suddenly dropped to 0 #42935: OpenCode Go quota exhausted in ~20 minutes after DeepSeek V4 Flash cache reads suddenly dropped to 0 (same root cause, quota impact)
   • Opencodego is integrated with Claude Code and Codex via ccSwitch, but during usage the cache hit rate is extremely low—already below 10%, and it keeps dropping with use, even trending towards nearly 0%!!! #43218: Opencodego cache hit rate extremely low (<10%) when used via ccSwitch with Claude Code / Codex (broader cache miss pattern on the Go endpoint)

   If your issue covers new models (qwen3.8-max) or new test evidence that the existing reports don't capture, please add a note to the oldest open duplicate so the team can track it in one place.
2. csalvarado commented on Sep 9, 2026

   

   csalvarado

   on Sep 9, 2026

   Author

   More actions

   Investigation notes (from repo walkthrough) to speed this up

   The Go gateway lives in this repo (packages/console/app/src/routes/zen/go/v1/chat/completions.ts, modelList: "lite"). The routing/cache-affinity flow is:

   • handler.ts:124,171 → stickyId = x-opencode-session (or workspaceID/ip).
   • handler.ts:172 → createStickyTracker(model.id, model.stickyProvider, stickyId) — only active if the model config sets stickyProvider (stickyProviderTracker.ts:9; no-op otherwise).
   • handler.ts:636-642 → provider pick by hashing the last 4 chars of stickyId over the weighted provider list; pinned in DB after first 200 (handler.ts:299).
   • provider/openai-compatible.ts:30-31 → upstream x-session-affinity: stickyId.
   • Usage forwarding is fine: openai-compatible.ts:61-70 maps cached_tokens/prompt_tokens_details.cached_tokens correctly (plus an optional adjustCacheUsage 90% estimate fallback at :68-70). So a reported cached_tokens: 0 means the upstream genuinely re-processed — the cache domain changed between calls.

   Likely root cause: the liteModels entries for deepseek-v4-flash and qwen3.8-max (loaded from env ZEN_MODELS_*, packages/console/core/src/model.ts:85-117) either lack stickyProvider and/or have multiple weighted providers/keys. selectProvider (handler.ts:614-627) drops providers mid-session via the TPM/TPS/budget filters, so the deterministic hash lands on a different upstream on the next call → cache-domain hop → full prefix reprocessed. glm-5.2 is pinned (single provider or stickyProvider set) and therefore caches. This matches the "gateway can leave the cache domain without the client changing anything" analysis in #42935 and the GLM warm/cold/warm in #35402.

   Suggested fixes:

   1. Config (fastest, no code): in the ZEN_MODELS_* env secrets for liteModels, set stickyProvider: "strict" on deepseek-v4-flash and qwen3.8-max (like GLM), or collapse each to a single provider/key, or enable adjustCacheUsage: true on those providers as an estimate fallback.
   2. Code hardening (optional): handler.ts:649-663 — the budget/TPS overrides (return provider at :663) apply even in "strict" mode; gate them to "prefer" only so "strict" really pins the provider.
3. csalvarado commented on Sep 16, 2026

   

   csalvarado

   on Sep 16, 2026

   Author

   More actions

   Live re-test after latest dev (2026-09-15, legacy sk- key, raw API)

   Ran the exact repro from the issue: N identical requests to https://opencode.ai/zen/go/v1/chat/completions, fixed ~3.3k-token prompt, x-opencode-session set consistently per session.

   Result: caching now engages, but sessions still hop between two upstream stacks and lose the cache when they do.

   Model	Session	Call 1	Call 2	Verdict
   deepseek-v4-flash	ses_probe_A	prompt=3338, hit=0, cached=0	hit=3200, cached=3200	caches
   deepseek-v4-flash	ses_probe_C	prompt=3338, hit=0, cached=0	hit=3200, cached=3200	caches
   deepseek-v4-flash	ses_probe_B	prompt=3338, hit=0, cached=0	prompt=3391, hit/miss/cached absent	hop → cache lost
   qwen3.8-max	ses_livecache_qw01	prompt=3360, cached=0	cached=3072	caches
   glm-5.2 (control)	ses_livecache_glm01	cached=10	cached=3320	caches

   Observations:

   1. The literal "cached_tokens always 0" symptom is no longer reproducible for deepseek-v4-flash or qwen3.8-max — repeated identical requests now return cached reads (3200/3072), so part of this is resolved.
   2. There are two distinct upstream shapes behind the same session:
      • Stack A (DeepSeek-native): prompt_tokens=3338, reports prompt_cache_hit_tokens/prompt_cache_miss_tokens.
      • Stack B: prompt_tokens=3391 (different tokenizer), reports only prompt_tokens_details.cached_tokens and sometimes hollows that too (same missing-usage-fields pattern as [Bug] Zen Go /v1/chat/completions strips usage.completion_tokens_details for deepseek-v4-flash - official DeepSeek API sends it #45606).
   3. When the session hops A→B mid-conversation, the prefix cache does not carry over and the next call is cold again (ses_probe_B above). 1 of 3 fresh sessions hopped in this sample. A 6-call run pinned to stack B cached consistently (cached=3328), so each stack caches well on its own — the loss is the hop.
   4. This matches the "cache randomly drops to 0" family (OpenCode Go quota exhausted in ~20 minutes after DeepSeek V4 Flash cache reads suddenly dropped to 0 #42935, GLM-5.1 prompt cache randomly drops to 0 on opencode-go, while DeepSeek V4 Flash works reliably #31348, [OpenCode Go] GLM-5.2 prompt cache randomly drops to ~500 tokens on opencode-go (opencode.ai/zen/go) #33998) and the code path where a non-pinned model's provider selection is re-evaluated per request (handler.ts selectProvider: the budget/TPS overrides can abandon the sticky provider even in "strict" mode). stickyProvider on the liteModels entries for these models would prevent the hop; adjustCacheUsage alone would only mask the accounting.

   Conclusion: partially resolved — caching works now, but it is not session-stable while the gateway can switch upstream stacks mid-session.
4. csalvarado commented on Oct 2, 2026

   

   csalvarado

   on Oct 2, 2026

   Author

   More actions

   Update: retested live today with a legacy sk- key and caching is now working on the Go endpoint for the models in this issue. Identical repeated requests on the same session now return cached reads:

   model	call 1 cached	call 2 cached
   deepseek-v4-flash (session A)	cold	3328
   deepseek-v4-flash (session B)	3328	3328
   deepseek-v4.1-flash	0	3328
   qwen3.8-max	0	3072

   prompt_cache_hit_tokens / prompt_cache_miss_tokens are still not forwarded for these upstreams (same hollow-usage shape as #45606), but prompt_tokens_details.cached_tokens reports correctly and the effective behavior matches the expected output.

   Closing this as resolved. If the upstream-stack hop described in the earlier comments reappears (cache dropping mid-session), reopen and we can pick it up from there.
5. csalvarado

   closed this as completedon Oct 2, 2026