Skip to content

fix(tui): flag likely cache busts accurately - #38727

Merged
kitlangton merged 4 commits into
v2from
cache-reporting
Jul 24, 2026
Merged

kitlangton merged 4 commits into
v2from
cache-reporting

Conversation

@kitlangton

@kitlangton kitlangton commented Jul 24, 2026 •

Copy link
Copy Markdown
Contributor

What

Make the debug turn-token view warn about likely prompt-cache busts without flagging known cache resets or OpenAI accounting noise.

The TUI now:

  • renders unexplained comparable drops as ! Likely cache bust
  • uses warning styling rather than neutral or error styling
  • suppresses OpenAI movements between 1,024 and 2,048 tokens, which can reflect cache accounting quantization
  • compares cache usage only when provider, model, and variant match
  • starts a fresh comparison baseline after completed compaction

Before / After

Before

Every lower cached-token count was labeled a definitive Cache bust, including normal OpenAI quantization, comparisons across model switches, and expected cache resets after compaction. An intermediate revision suppressed those false positives but made every remaining drop a subdued neutral observation.

Cache reuse dropped by 62,464 tokens from the previous step

After

Known OpenAI bucket movements remain silent. Model switches and completed compactions establish a new baseline. A larger decrease between matching, uncompacted model steps is highlighted without claiming certainty about its cause.

! Likely cache bust: 62,464 fewer cached tokens than the previous step

How

  • packages/tui/src/routes/session/rows.ts carries model identity with cross-turn cache baselines, clears the baseline at completed compaction boundaries, and centralizes reuse-drop classification.
  • packages/tui/src/routes/session/index.tsx compares each step against the preceding comparable cache sample and renders unexplained drops with warning wording and color.
  • packages/tui/test/cli/tui/session-rows.test.ts covers OpenAI threshold boundaries, non-OpenAI drops, model/variant switches, cross-turn baseline projection, and completed compaction resets.

Scope

This changes debug-only reporting. It does not claim a proven cause, alter model requests, change cache policy, modify compaction, or change token accounting.

Testing

  • bun run test test/cli/tui/session-rows.test.ts: 15 passed, 0 failed
  • bun run test in packages/tui: 519 passed, 5 skipped, 0 failed
  • bun typecheck in packages/tui: passed
  • pre-push bun turbo typecheck --concurrency=3: 33 tasks passed
  • inspected real GPT-5.6 Sol session accounting: benign 1,024/2,048-token bucket movements are suppressed, while materially larger collapses remain reportable

@kitlangton kitlangton changed the title fix(tui): report cache reuse drops accurately fix(tui): flag likely cache busts accurately Jul 24, 2026
@kitlangton
kitlangton enabled auto-merge (squash) July 24, 2026 17:59
@kitlangton
kitlangton merged commit 3193f3a into v2 Jul 24, 2026
15 of 17 checks passed
@kitlangton
kitlangton deleted the cache-reporting branch July 24, 2026 18:35
jinhuang712 pushed a commit to jinhuang712/opencode that referenced this pull request Sep 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant