Skip to content

fix: auto-compaction fires on fresh cached token counts (#66520)#2511

Open
BingqingLyu wants to merge 2 commits into
mainfrom
fork-pr-66716-fix-compaction-cached-tokens
Open

fix: auto-compaction fires on fresh cached token counts (#66520)#2511
BingqingLyu wants to merge 2 commits into
mainfrom
fork-pr-66716-fix-compaction-cached-tokens

Conversation

@BingqingLyu

@BingqingLyu BingqingLyu commented Apr 28, 2026

Copy link
Copy Markdown
Owner

Summary

  • Bug: runPreflightCompactionIfNeeded returned early when totalTokensFresh === true without checking the compaction threshold, so auto-compaction never triggered for sessions with fresh token counts — even at 153% of the context window
  • Root cause: The early-return optimization (skip transcript estimation when fresh data is available) accidentally bypassed the threshold comparison entirely
  • Fix: Restructure token resolution so fresh persisted totals (which include cacheRead from Anthropic prompt caching) are projected forward and checked against contextWindow - reserveTokens - softThreshold before deciding whether to compact

Details

When Anthropic's prompt cache absorbs nearly all tokens (100% hit rate), derivePromptTokens correctly computes input + cacheRead + cacheWrite (e.g. 99 + 305,000 + 0 = 305,099), and this is persisted as totalTokens with totalTokensFresh: true. However, runPreflightCompactionIfNeeded had this logic:

if (!shouldUseTranscriptFallback) {
    return entry;  // BUG: skips threshold check entirely
}

The fix moves the threshold check into both branches (fresh and stale), using the fresh persisted value directly when available.

Edge cases verified

  • 100% cache hit (Anthropic): totalTokens=305k, contextWindow=200k — compaction now fires
  • 0% cache hit: Stale tokens fall through to transcript estimation path — behavior unchanged
  • Partial cache: 180k total with 166k threshold — fires correctly
  • Below threshold: 50k total with 166k threshold — no compaction, as expected
  • Heartbeat/CLI: Still skipped regardless of token count
  • Non-Anthropic providers: No change — providers without caching have cacheRead=0, so totalTokens is just input + cacheWrite, same as before

Test plan

  • shouldRunPreflightCompaction unit tests: 100% cache hit triggers, below threshold skips, partial cache boundary correct
  • runPreflightCompactionIfNeeded integration tests: fresh tokens above threshold trigger compaction, below threshold skip, stale tokens use transcript fallback, heartbeat skips
  • All existing compaction tests pass unchanged
  • Session usage persistence tests pass (65 tests)
  • Followup runner tests pass (19 tests)
  • Preemptive compaction tests pass (9 tests)

Fixes openclaw#66520

🤖 Generated with Claude Code

kewang-pika and others added 2 commits April 14, 2026 12:48
…reshold

When a provider like Anthropic has a high prompt cache hit rate, totalTokens
on the session entry includes cached tokens (input + cacheRead + cacheWrite)
and is marked as fresh. Previously, runPreflightCompactionIfNeeded returned
early when totalTokensFresh was true WITHOUT checking the compaction threshold,
so auto-compaction never triggered — even at 153% of the context window.

This change restructures the token resolution so that fresh persisted totals
are checked against the compaction threshold (contextWindow - reserveTokens -
softThreshold) before deciding whether to compact. The stale/transcript
fallback path is preserved unchanged.

Fixes openclaw#66520

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The fresh-token branch in runPreflightCompactionIfNeeded ran when
shouldUseTranscriptFallback was false, but that predicate treated
totalTokensFresh: undefined as NOT needing fallback (undefined === false
evaluates to false). For legacy sessions where totalTokensFresh was
never set, this caused the fresh-token path to use potentially stale
persisted totals, triggering unnecessary compaction.

Three changes:

1. Replace shouldUseTranscriptFallback with hasFreshPersistedTokens
   that requires totalTokensFresh === true explicitly. When undefined
   (legacy) or false (known-stale), falls through to transcript
   estimation.

2. In the transcript fallback branch, always run transcript estimation
   instead of skipping when resolveFreshSessionTotalTokens returns a
   value (that function also treats undefined freshness as fresh).

3. Add bail-out before shouldRunPreflightCompaction when transcript
   estimation returns no count and freshness is unconfirmed, preventing
   the gate function from falling back to stale totals via
   resolveFreshSessionTotalTokens.

Resolves Codex P2 review on PR openclaw#66716.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@BingqingLyu BingqingLyu added conflicting-group-1 Conflicting PR group 1 — review as a batch conflicting-pr Shares at least one cross-PR dependency with other PRs labels May 26, 2026
@BingqingLyu

Copy link
Copy Markdown
Owner Author

⚠️ Cross-PR Conflict Detected

This PR shares modified code with #2272, #2499, #2523.

Shared functions:

Function File Also modified by
createReplyOperation src/auto-reply/reply/agent-runner-memory.test.ts #2272, #2523
runPreflightCompactionIfNeeded src/auto-reply/reply/agent-runner-memory.ts #2272, #2499

Recommendation: Coordinate with #2272, #2499, #2523 before merging.


Auto-detected by codegraph — a code graph analysis tool built on neug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conflicting-group-1 Conflicting PR group 1 — review as a batch conflicting-pr Shares at least one cross-PR dependency with other PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Auto-compaction never fires when Anthropic prompt cache hit rate is ~100%

2 participants