fix(cache): delay history image pruning to preserve prompt cache prefix by bcherny · Pull Request #58038 · openclaw/openclaw

bcherny · 2026-03-31T01:11:39Z

Problem

pruneProcessedHistoryImages replaces image blocks with [image data removed - already processed by model] text for every user/toolResult message before the last assistant turn, on every run.

Turn N: user sends image → provider caches the image bytes in the prompt prefix
Turn N+1: prune runs, replaces image bytes with text marker → request bytes diverge at that message → cache miss from there onward

This defeats prompt caching for any conversation that includes images.

Fix

Only prune images older than 3 assistant turns (PRESERVE_RECENT_ASSISTANT_TURNS). Recent history stays byte-identical so the cached prefix survives across turns, while legacy sessions with persisted image payloads still get cleaned up once they age out.

The original purpose — per the doc comment — was "idempotent cleanup for legacy sessions", so aggressive per-turn pruning was never needed.

Verification

No downstream consumer depends on the PRUNED_HISTORY_IMAGE_MARKER appearing at turn N+1 (only referenced in this file + test)
Context-size overflow is handled by compaction (applyPiAutoCompactionGuard), not by this prune — keeping ~3 turns of images is a small delta vs. the previous 0-1
Legacy session migration (state-migrations.ts) still exists, so this cleanup remains load-bearing for migrated sessions

Tests

6 tests pass including two new cases:

keeps image blocks within the last 3 assistant turns to preserve prompt cache
prunes only old images while preserving recent ones

🤖 Generated with Claude Code

pruneProcessedHistoryImages was stripping image blocks from every already-answered user turn on each run. Turn N sends image bytes → provider caches the prefix. Turn N+1 replaces image with text marker → bytes diverge at that message → cache miss from there onward. Now only prune images older than 3 assistant turns. Recent history stays byte-identical so the cached prefix survives, while legacy sessions with persisted image payloads still get cleaned up.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 922344f985

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

greptile-apps · 2026-03-31T01:16:00Z

Greptile Summary

This PR fixes prompt cache invalidation caused by pruneProcessedHistoryImages replacing image data too aggressively. By delaying pruning until images are older than 3 assistant turns (instead of 1), the recent conversation history stays byte-identical across requests, preserving the prompt-cache prefix.

Key changes:

history-image-prune.ts: Introduces PRESERVE_RECENT_ASSISTANT_TURNS = 3. The scan now counts assistant messages from the end and sets pruneBeforeIndex to the index of the 3rd-from-last assistant; only messages before that index are eligible for image pruning.
history-image-prune.test.ts: Tests updated to require 3 assistant turns before pruning fires, with two new cases covering the preservation window and the mixed old/recent image scenario.
attempt.ts: Comment updated to reflect the new semantics.

The boundary conditions are correct: assistantSeen >= PRESERVE_RECENT_ASSISTANT_TURNS trips on the 3rd assistant from the end, and the inner loop i < pruneBeforeIndex correctly excludes all messages in the preserved window. The PR also correctly scopes impact — PRUNED_HISTORY_IMAGE_MARKER is only referenced within the history-image-prune module and its test file, so no downstream consumer is broken by the delayed pruning.

Confidence Score: 5/5

Safe to merge — logic is correct, tests are comprehensive, and no downstream consumers are affected.
No P0 or P1 issues found. The reverse-scan algorithm correctly identifies the 3rd-from-last assistant turn and the pruning boundary is accurate. Tests cover the preserved-window, mixed old/recent, toolResult, and no-assistant-exists scenarios. The only change in attempt.ts is a comment update.
No files require special attention.

_{Reviews (1): Last reviewed commit: "fix(cache): delay history image pruning ..." | Re-trigger Greptile}

@bcherny

…ix (#58038) (thanks @bcherny)

odysseus0 · 2026-04-04T00:23:22Z

Landed via temp rebase onto main.

Gate: pnpm check ✅
Land commit: 2f8e89d
Merge commit: af81c43

Thanks @bcherny!

…ix (openclaw#58038) pruneProcessedHistoryImages was stripping image blocks from every already-answered user turn on each run. Turn N sends image bytes → provider caches the prefix. Turn N+1 replaces image with text marker → bytes diverge at that message → cache miss from there onward. Now only prune images older than 3 assistant turns. Recent history stays byte-identical so the cached prefix survives, while legacy sessions with persisted image payloads still get cleaned up.

arirahikkala · 2026-04-04T15:12:36Z

Granted that this is probably not worth spending human time on, since this whole path is only for cleaning up legacy sessions anyway, but: This makes things worse, right?

This moves the point where the inputs diverge from the longest cached prefix to earlier in the context. Since the pruning is idempotent, if you just prune the whole conversation's images, you have all of it in cache from then on, but this change instead means that at each new turn n, you may mutate the assistant input at turn n-3, and anything past that becomes uncached input. (until you get past the legacy image injections, anyway)

…enclaw#58037 openclaw#58038

(#60633) * docs: correct overstated prompt-cache comments from #58036 #58037 #58038 * docs: restore purpose context in MCP tool sort comment * docs: drop misleading 'legacy' framing from image-prune comments * docs: restore useful context stripped from image-prune comments * docs: restore 'deterministically' in MCP tool sort comment * docs: restore 'idempotent' at attempt.ts callsite * docs: restore 'provider prompt cache' in context-guard comment

…ix (openclaw#58038) pruneProcessedHistoryImages was stripping image blocks from every already-answered user turn on each run. Turn N sends image bytes → provider caches the prefix. Turn N+1 replaces image with text marker → bytes diverge at that message → cache miss from there onward. Now only prune images older than 3 assistant turns. Recent history stays byte-identical so the cached prefix survives, while legacy sessions with persisted image payloads still get cleaned up.

…enclaw#58037 openclaw#58038 (openclaw#60633) * docs: correct overstated prompt-cache comments from openclaw#58036 openclaw#58037 openclaw#58038 * docs: restore purpose context in MCP tool sort comment * docs: drop misleading 'legacy' framing from image-prune comments * docs: restore useful context stripped from image-prune comments * docs: restore 'deterministically' in MCP tool sort comment * docs: restore 'idempotent' at attempt.ts callsite * docs: restore 'provider prompt cache' in context-guard comment