Skip to content

feat(session): chunk oversized compaction requests and escalate tail reduction - #48845

Closed
whitesm2000 wants to merge 2 commits into
anomalyco:devfrom
whitesm2000:fix/compaction-chunked-summaries
Closed

whitesm2000 wants to merge 2 commits into
anomalyco:devfrom
whitesm2000:fix/compaction-chunked-summaries

Conversation

@whitesm2000

@whitesm2000 whitesm2000 commented Sep 13, 2026 •

Copy link
Copy Markdown

Feat: chunk oversized compaction requests and escalate tail reduction

Fixes #48844
Depends on / includes #48828 — the first commit on this branch is that PR's loop guard; the escalation ladder builds on its streak machinery. If #48828 merges first, this PR's diff reduces to the second commit.

Problem

Compaction fails exactly when it is most needed. select() budgets only the retained tail; the serialized head is sent to the summarizer as a single unbudgeted request. A session that outgrows its model's window therefore produces a summarization request that doesn't fit, and compaction dies with Session too large to compact (observed live on a ~60k-token session with a 64k-context model). Two further issues compound it:

  • With a dedicated compaction model configured (agent.compaction.model), the retained-tail budget was derived from the compaction model's window instead of the session model's — e.g. a 1M-window compactor retains a 15k-token tail for a 64k-window session, which then permanently overflows.
  • A failed auto-compaction (post-compaction context still overflows) retried with the same tail budget, failing identically until the fix(session): stop looping auto-compaction when it cannot restore context headroom #48828 loop guard stopped the session.

Fix

Chunked (map-reduce) summarization. New reduceContexts step between serialization and the persisted summary call:

  • The serialized head is packed into chunks capped at 0.5 × usable window of the compaction model (estimate-based, with a 2× safety margin against the 4-chars/token heuristic).
  • Each chunk is summarized via an ephemeral, unpersisted LLM.stream call; partial summaries that still exceed the budget are packed and re-summarized (≤ 4 merge rounds, practically 1).
  • Only the final, fitting summary request goes through the persisted processor path — events, token accounting, and error handling for it are unchanged.
  • Sessions whose head fits take the previous single-request path unchanged (byte-identical prompt). Models with unknown limits (context: 0) skip chunking entirely, preserving current behavior.

Session-model tail budget with an escalation ladder. select() now sizes the retained tail from the session model (whose window the compacted session must fit), and repeated failed auto-compactions shrink it: full → half → none (retaining nothing means the compacted context is summary-only). The streak comes from the #48828 failedAutoCompactions machinery; the loop guard's limit becomes the ladder length (3 attempts, then the session stops with an actionable error).

Validation

  • Unit tests: chunkSerialized packing/truncation (4 cases), preserveRecentBudget escalation ladder incl. clamping and explicit-config scaling (4 cases).
  • Integration tests (real compaction pipeline against a fake LLM): oversized head is summarized in budget-fitting chunks (asserts every request, chunk and final, is under budget); bloated partial summaries trigger a merge round; first auto-compaction retains the tail; after two failed auto-compactions the retained tail is dropped (tail_start_id unset).
  • Full package suite: 3602 pass / 0 fail; tsgo --noEmit clean.
  • Repro scenario: with compaction model deepseek/deepseek-v4-flash (1M) and session model cerebras/qwen-3.8-27b (64k), a ~60k-token session that previously died with Session too large to compact now takes the single-request path (the head fits the compactor's 1M window), and the session-model-derived, escalating tail budget lets the compacted session fit qwen's 33.5k usable window. The integration tests cover the chunked path with a 20k-context model.

Follow-ups (not in this PR)

  • Chunk summarization calls are ephemeral and their token usage is not tracked in session stats.
  • Chunk calls don't take the session abort signal; cancellation takes effect between chunk requests rather than mid-request.
  • The core native-runtime compaction engine (@opencode-ai/core/session/compaction) fails safe (return false) on oversized prompts but doesn't chunk; it could adopt the same map-reduce helper.

…text headroom

When a session's irreducible context (system prompt plus retained tail)
exceeds the model's usable window, every step finishes over the overflow
threshold and the loop auto-compacts after every single step forever,
burning provider quota (and tripping rate limits) without making progress.

Add a no-progress guard: count consecutive automatic compactions whose
first finished successor step still overflows the model's usable window.
Once that streak reaches MAX_FAILED_AUTO_COMPACTIONS (2), stop the loop
and surface a ContextOverflowError telling the user to start a new
session or switch to a larger-context model, instead of compacting again.

A compaction is judged by its first finished non-summary successor step,
so legitimate flows that overflow again only after pulling in new large
tool results still get compacted normally. Real user turns and manual
compactions reset the streak.
…reduction

The summarization request sent to the compaction model had no size
budget: select() capped only the retained tail, so a large session
handed the whole serialized head to the summarizer in one request.
When that request exceeded the model's context window, compaction
failed outright with "Session too large to compact".

- reduceContexts: pack the serialized head into chunk-budget-sized
  groups (half the compaction model's usable window), summarize each
  chunk, and re-summarize partial summaries until the final request
  fits (map-reduce). Single-chunk sessions take the previous code
  path unchanged.
- The retained-tail budget now derives from the session model (whose
  window the compacted session must fit), not the compaction model.
- Repeated failed auto-compactions escalate the tail budget (full,
  half, none) so a session whose floor exceeds the window sheds
  retained context before the loop guard stops it. The guard's limit
  becomes the ladder length (3).

Refs anomalyco#48827
@github-actions github-actions Bot added the needs:compliance This means the issue will auto-close after 2 hours. label Sep 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR doesn't fully meet our contributing guidelines and PR template.

What needs to be fixed:

  • PR description is missing required template sections. Please use the PR template.

Please edit this PR description to address the above within 2 hours, or it will be automatically closed.

If you believe this was flagged incorrectly, please let a maintainer know.

@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically closed because it was not updated to meet our contributing guidelines within the 2-hour window.

Feel free to open a new pull request that follows our guidelines.

@github-actions github-actions Bot removed the needs:compliance This means the issue will auto-close after 2 hours. label Sep 13, 2026
@github-actions github-actions Bot closed this Sep 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

"Session too large to compact" — the summarization request itself has no size budget

1 participant