[FEATURE]:Performance issue: Context cache invalidation when switching modes or during compaction

Feature hasn't been suggested before.

I have verified this feature I'm about to request hasn't been suggested before.

Describe the enhancement you want to request

I am experiencing significant performance issues when using OpenCode with local LLMs (inference engines like vLLM or Ollama). The context cache prefix is frequently invalidated, forcing the engine to reprocess the entire conversation history, which is very slow on local hardware.
According to technical analysis (e.g., tests conducted by the Nicefox YouTube channel), OpenCode breaks the cache for several reasons:

Tool Switching: When switching between "Plan" and "Build" modes, the list of available tools changes. Since tools are part of the request prefix, this modification invalidates the cache.
History Modification: Instead of only appending new messages, OpenCode sometimes modifies previous messages in the history (for example, adding/removing "system reminders" about the current mode). Any change in the middle of the message history forces a full reprocessing of the context.
Compaction: The conversation compaction process does not seem to preserve the cache prefix.

Expected Behavior: The tools and the system prompt should remain constant (or be managed incrementally) across different modes to allow the inference engine to reuse the cached prefix. Previous messages should not be altered once they are part of the context.
Comment reproduire le bug ?

Pour prouver que le cache est "cassé", vous pouvez suivre ce protocole expérimental décrit dans les sources :

Préparation : Utilisez un moteur d'inférence local (comme vLLM) et, si possible, un proxy (comme LiteLLM) qui permet de voir si le préfixe de cache est réutilisé ou si le texte est "re-processé".
Étape 1 (Mode Plan) : Démarrez une session en mode Plan et envoyez un message simple (par exemple : "Bonjour"). Notez le temps de réponse.
Étape 2 (Bascule de mode) : Passez en mode Build et renvoyez un message (par exemple : "Peux-tu m'aider ?").
    Observation : Vous remarquerez que le moteur d'IA doit analyser à nouveau toute la conversation car OpenCode a changé la liste des outils disponibles entre les deux modes, ce qui "invalide" le cache.
Étape 3 (Vérification de l'historique) : Regardez les requêtes envoyées. Vous constaterez qu'OpenCode modifie souvent le contenu de votre premier message (en y insérant des rappels système au lieu de les mettre dans un nouveau message), ce qui change le "hash" du préfixe et casse le cache.
Étape 4 (Compaction) : Continuez la conversation jusqu'à ce que l'outil déclenche automatiquement le compactage de l'historique. Vous verrez que le temps de traitement devient extrêmement long car l'intégralité du contexte est à nouveau recalculée.

This issue might be a duplicate of existing issues. Please check:

PR #25303 causes mass KV invalidation on model switch (Plan↔Build↔Compact) #32603: PR fix: bedrock reasoning issue #25303 causes mass KV invalidation on model switch (Plan↔Build↔Compact) — directly covers tool-list changes on mode switch causing vLLM/local inference prefix cache invalidation
<system-reminder> keeps moving, causing unnecessary prompt processing in llama.cpp #23595: <system-reminder> keeps moving, causing unnecessary prompt processing in llama.cpp — covers history modification (inserting mode reminders into previous messages) breaking the cache
reasoning_content stripped from assistant messages on replay, causing KV cache invalidation on local inference #19081: reasoning_content stripped from assistant messages on replay, causing KV cache invalidation on local inference — covers history modification invalidating the local KV cache
[FEATURE]: Modify system prompt to improve cache hit rate #29672: [FEATURE]: Modify system prompt to improve cache hit rate — covers keeping the system prompt stable to maximize cache hits
Switching from plan to build mode with DeepSeek V4 Pro/Flash causes context loss and cache misses #35708: Switching from plan to build mode with DeepSeek V4 Pro/Flash causes context loss and cache misses — same mode-switch cache invalidation symptom

I observed a related but broader performance issue in #37767 .

vLLM logs confirm that cache stability is important, but I also found substantial overhead from sequential single-file exploration: OpenCode repeatedly invokes the model to choose and read one file at a time, whereas another client first performs a repository-wide grep and then reads only the candidate files.

In my test, backend decode throughput differed by only about 20%, while total task duration differed by approximately 6x. This suggests that cache invalidation and excessive agent-loop iterations are separate but compounding problems.

Thanks for your reply.
Based on observations on local AI, Claude Code and Kilocode don't have this bug.

https://www.youtube.com/watch?v=WQ_lekP3Og0

I observed this bug today with Deepseek v4 Flash hosted on ik_llama.cpp (a llama.cpp fork optimized for CPU inference). Whenever I switched from plan to build mode, there was a full reprocess of the context taking a long time.

I’ve reproduced the compaction-specific part of this issue on current dev and have a focused fix prepared. I can see the issue is already assigned, so I wanted to check before opening a PR in case this overlaps with work already in progress.

The underlying problem is that compaction constructs a different provider request, changing the system, tool, and message prefix from the preceding normal turn. The proposed fix uses the ordinary prepared request as the prefix for same-model text compaction, then appends the summary instruction. Existing serialized fallback behavior remains for alternate models and structured output, and summary-time tool execution is prevented.

Verification on current dev:

core compaction tests: 6 passed
focused OpenCode compaction, prompt, and Copilot tests: 132 passed, with 2 existing skips
package and monorepo typechecks passed
formatting, clean-base patch application, and standalone arm64 build passed
on a local cache-observable backend, two automatic compactions reused the full eligible prefixes—31,600 and 32,451 tokens—retained the required details, performed no summary-tool execution, and completed a normal post-compaction tool turn

This overlaps with the direction explored in #25100, which was closed by automated inactivity cleanup rather than technical review. Would a new focused PR be welcome, or is the compaction portion already being handled?

A local prefix cache (vLLM, llama.cpp, ik_llama.cpp) only survives if the next request starts with the same bytes as the last one: tools, then system, then messages. Two things in this repo change those bytes on purpose.

Plan ↔ Build changes the tool list. Tools are built from registry.snapshot(agent.permissions). A blanket-deny permission drops the tool from the request entirely, it is not sent as “disabled”:

opencode/packages/core/src/tool.ts

Lines 205 to 214 in 9e22c40

    
           snapshot: Effect.fn("Tool.snapshot")((permissions) => 
        
             lock.withPermit( 
        
               Effect.gen(function* () { 
        
                 const active = new Map<string, Tool.Info>() 
        
                 const rules = permissions ?? [] 
        
                 for (const [name, entries] of local) { 
        
                   const tool = entries.at(-1)?.tool 
        
                   if (!tool) continue 
        
                   if (whollyDisabled(tool.options?.permission ?? name, rules)) continue 
        
                   active.set(name, tool)

The tool block is the start of the prefix, so a different list is a full prefill. That matches the Plan→Build stall on ik_llama.cpp and the same symptom in #32603 and #35708.

Claude Code does not do that. It keeps the tool list still and refuses writes at execution time. OpenCode already has that refusal for Plan mode — edit / write / patch die in execute.before without touching the wire list:

opencode/packages/core/src/plugin/plan.ts

Lines 33 to 38 in 9e22c40

    
           yield* ctx.tool.hook("execute.before", (event) => { 
        
             if (event.agent !== plan) return Effect.void 
        
             if (event.tool !== "edit" && event.tool !== "write" && event.tool !== "patch") return Effect.void 
        
             return new ToolFailure({ 
        
               message: `Cannot use ${event.tool} in Plan mode. You are in a read-only mode and must not modify files.`, 
        
             })

Advertising the same tools every turn, and keeping that hook, is the fix. Removing tools from the snapshot is what burns the cache.

The Plan reminder is a new synthetic message (enter / leave on agent switch). If the tool list and system prompt stay put, that is only a short tail. The long stall is the tool-list change. Rewriting an older <system-reminder> in place is a separate bust and is #23595.

Compaction sends a different request under the same session cache key. Same promptCacheKey as the live turn, but tools: [], one user message, no system part:

opencode/packages/core/src/session/compaction.ts

Lines 274 to 281 in 9e22c40

    
           .stream( 
        
             LLM.request({ 
        
               model: plan.model, 
        
               promptCacheKey: SessionPromptCacheKey.make(plan.session.id), 
        
               http: { headers: SessionModelHeaders.make(plan.session, dependencies.app) }, 
        
               messages: [Message.user(plan.prompt)], 
        
               tools: [], 
        
             }),

That cannot read the cache the session just built, and it writes a divergent prefix onto the key the next Build turn will try to use. On a local engine that is another full prefill.

The repair is to compact as a continuation of the ordinary request — same tools, same system, summarize instruction appended — so the history is a cache read. That is the direction in the comment above, and what #25100 asked for before it was closed for inactivity. #43249 is the same share on hosted prompt-cache.

Keep the tool list still (Claude Code). Compact on top of the live prefix, not beside it. Those two changes are what the local traces are asking for.

To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.

Navigation Menu

Repository navigation

[FEATURE]:Performance issue: Context cache invalidation when switching modes or during compaction #37489

Feature hasn't been suggested before.

Describe the enhancement you want to request

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	snapshot: Effect.fn("Tool.snapshot")((permissions) =>
	lock.withPermit(
	Effect.gen(function* () {
	const active = new Map<string, Tool.Info>()
	const rules = permissions ?? []
	for (const [name, entries] of local) {
	const tool = entries.at(-1)?.tool
	if (!tool) continue
	if (whollyDisabled(tool.options?.permission ?? name, rules)) continue
	active.set(name, tool)

	yield* ctx.tool.hook("execute.before", (event) => {
	if (event.agent !== plan) return Effect.void
	if (event.tool !== "edit" && event.tool !== "write" && event.tool !== "patch") return Effect.void
	return new ToolFailure({
	message: `Cannot use ${event.tool} in Plan mode. You are in a read-only mode and must not modify files.`,
	})

	.stream(
	LLM.request({
	model: plan.model,
	promptCacheKey: SessionPromptCacheKey.make(plan.session.id),
	http: { headers: SessionModelHeaders.make(plan.session, dependencies.app) },
	messages: [Message.user(plan.prompt)],
	tools: [],
	}),

[FEATURE]:Performance issue: Context cache invalidation when switching modes or during compaction #37489

Description

Feature hasn't been suggested before.

Describe the enhancement you want to request

Activity

github-actions commented on Jul 17, 2026

xexxo2002-arch commented on Jul 19, 2026

ducon43 commented on Jul 19, 2026

VanceVagell commented on Aug 1, 2026

Hajsas commented on Aug 14, 2026

Zaczero commented on Aug 18, 2026

github-actions commented on Oct 18, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions