Repository navigation
V2: summary compaction still reads almost nothing from the prompt cache, even right after a warm request #52761
Description
Description
This is the new report requested in the closing comment of #43249 ("If you're seeing repeated cold compactions on
current V2, please open a new report with the exact version, provider/model, and input/cache-read counts").
On V2, summary compactions read almost nothing from the prompt cache. This also happens when the
previous model call in the same session ran less than a minute earlier and read more than 80% of its input from the
cache. On Anthropic Messages routes, about 96% of each compaction's input is billed as a new cache write.
Selection: compactions in the local opencode.db that completed on 2.0.12 or later (version estimated from the
sessions created just before), with at least 50k input tokens.
In each case, the previous model call in the session ran at most 5 minutes earlier and read at least 80% of its input
from the cache. Only 1 of these 132 compactions read at least 50% of its input from the cache.
| Route | Compactions | Versions | Median input | Median cache read | Read share | Write share | Median gap since previous call |
|---|---|---|---|---|---|---|---|
Built-in openai provider (gpt-6-luna, gpt-6-sol, gpt-6-astra), auto |
113 | 2.0.16–2.0.21 | 253,321 | 20,992 | 7.4% | 0.0% | 0.68 min |
| Anthropic Messages (Anthropic API, Vertex AI, Azure AI Foundry; Claude models), mostly manual | 12 | 2.0.14–2.0.21 | 407,080 | 8,051 | 4.2% | 95.8% | 0.76 min |
Example rows (previous call input / previous call cache read → compaction input / cache read / cache write):
2.0.21 openai gpt-6-luna auto gap 0.54 min: 784,645 / 776,704 → 487,401 / 4,608 / 0
2.0.19 anthropic claude-sonnet-5-5 auto gap 0.31 min: 903,250 / 883,134 → 641,900 / 8,051 / 633,845
2.0.16 google-vertex claude-opus-5-5 manual gap 1.27 min: 837,966 / 836,224 → 672,165 / 0 / 672,161
2.0.22 google-vertex gemini-3.8-flash auto gap 0.57 min: 951,296 / 899,938 → 421,711 / 0 / 0
The constant read of about 21k tokens on OpenAI and about 8k tokens on Anthropic looks like the system prompt and tool
definitions only. The cache match appears to stop at the first conversation message.
Possible cause
In 2.0.22, summarize sends only the older part of the history, split.older, and keeps the recent part as text
outside the request:
- https://github.com/anomalyco/opencode/blob/v2.0.22/packages/core/src/session/compaction.ts#L263-L270
- https://github.com/anomalyco/opencode/blob/v2.0.22/packages/core/src/session/compaction.ts#L690-L701
For Anthropic, this would explain the result. The request's cache breakpoints sit at the end of the older part. The
cache entries that the main loop keeps fresh end at the end of the full history, so no live entry matches the shorter
prefix. For OpenAI, I would expect the shorter prefix to still match, so I could not explain the miss there. The
request may differ from the main-loop request in the first conversation message, or in transport or cache-key
routing. I did not confirm either of these.
The closing comment on #43249 cites a /compact test on 2.0.12 that read about 99% from the cache. The one hit in
our data (github-copilot, 2.0.20) read 113,792 of 114,744 input tokens. Its input was about the same size as the
previous call's input (113,815), which suggests that the older part covered almost the full history there.
Possible direction
One option is to send the main loop's full current request with the summary instruction appended, as
session.generate does, and apply the older/recent split to the stored result instead of to the request. #46369
proposes something in this direction as an opt-in. Related: #48542 (Anthropic compaction cache writes never read).
Plugins
A local plugin rewrites the cache_control TTL from 5m to 1h on Anthropic Messages requests. It does not change
OpenAI requests, so the OpenAI rows are unaffected by it.
OpenCode version
2.0.22 (data from 2.0.12 to 2.0.22)
Steps to reproduce
- Run a long session on the built-in
openaiprovider until auto compaction triggers, or run/compacton a
session with an Anthropic Messages model after more than one exchange. - Compare
tokens.cache.readof the compaction message (session_message.type = 'compaction') with the previous
assistant message in the same session.
Operating System
Ubuntu 26.04.1 LTS
Terminal
Not applicable (OpenCode server)
Activity
github-actions commentedon Oct 2, 2026
This issue might be a duplicate of an existing issue. Please check:
rekram1-node commentedon Oct 2, 2026
Gtocha yup
sergiopadure commentedon Oct 2, 2026
Cost data from the same setup, as a follow-up to the measurements above (OpenCode 2.0.12 to 2.0.22, 11 days):
So about 88% of the compaction spend on these routes is avoidable. A few rows recorded $0 although their token counts are non-zero, so the real figure is likely a bit higher.
One example on 2.0.22 (Claude Opus 5.5, 1M window): an auto compaction at about 897k tokens of context, 14 minutes after the previous call read 889,326 tokens from cache. The compaction read 41,774 tokens from cache and wrote 697,966 new cache tokens, for $3.57. The cache write is also never read afterwards, because the next request starts from the summary, so even sending this request without cache breakpoints would have been cheaper ($3.03 at the uncached input price).
On subscription routes the same compactions cost no money but use quota: 126 compactions on the ChatGPT-login provider sent 27.9 million uncached input tokens in the same period.
AnnoyingTechnology commentedon Oct 9, 2026
@rekram1-node I opened #54107 as a draft for the direction above, in case it saves you time. It sends the conversation through the latest assistant reply (the prefix the provider has already seen) rather than the whole request, and falls back to the current older-only path when that does not fit. The stored
recentis unchanged. Take it over, change it, or close it as you prefer.Patch by Claude Opus 5.5 (via Claude Code), for @AnnoyingTechnology. Reviewed by GPT 6.1 Sol (xhigh, via Codex).
sergiopadure commentedon Oct 9, 2026
Correction to my report: most of the cache misses I measured came from a plugin in my own setup, not from OpenCode.
The plugin was ponytail. It added its ruleset to the end of the system prompt in the
contexthook only. OpenCode runs the separatecompactionhook for the summary request, so that request had a shorter last system block than the main loop. The cache match then stopped after the tools and the first system blocks. That would explain the constant reads in the table above (about 8k tokens on Anthropic, about 21k on OpenAI). The same plugin was active on both routes, but I retested only Anthropic. The published ponytail 5.1.0 does the same, so other ponytail users are likely affected too; I reported it there as DietrichGebert/ponytail#1086.Test setup: OpenCode built from source in a clean container, Claude Haiku 5.5 over the Anthropic Messages API, 3 scripted turns with tool calls, then
/compactwithin a minute. A logging forwarder captured the request bodies.v2at 1aae67e (#54107 parent)contextonlycompactionOn
v2at 1aae67e, summary compaction already reads most of its input from the cache when the system prompt is the same. #54107 still helps: it raised the read share from about 85% to 97% in these runs. Please disregard the cost figures in my earlier comment. They mostly reflect the plugin, not OpenCode.I fixed the plugin on my side. A note in the plugin documentation would help other authors. A plugin that changes
event.systemincontextshould register the same handler oncompaction. Otherwise, summary compactions miss the prompt cache.bhagirathsinh-vaghela commentedon Oct 11, 2026
One more data point on the approach, from my V1 fork (Faber). Compaction there builds its request through the same path as a normal turn: the session's own agent, the same tools and system blocks, the summary instruction appended as the last user message, and tools blocked when they run instead of removed from the schema. Since that change, 96.7% of compaction input on Anthropic has been read from cache (67 compactions, about 9x cheaper than a cold summary). Your plugin finding fits the same rule: anything that makes the compaction request's tools or system blocks differ from the main loop's (a separate hook, a dedicated compaction agent, an emptied tool list) breaks the match right there. Method and numbers are here.
bornmw commentedon Oct 16, 2026
if you could try this PR/branch? #54107
it fixed the issue for me