Repository navigation
feat(session): chunk oversized compaction requests and escalate tail reduction - #48845
Closed
whitesm2000 wants to merge 2 commits into
Closed
whitesm2000 wants to merge 2 commits into
whitesm2000 wants to merge 2 commits into
Conversation
…text headroom When a session's irreducible context (system prompt plus retained tail) exceeds the model's usable window, every step finishes over the overflow threshold and the loop auto-compacts after every single step forever, burning provider quota (and tripping rate limits) without making progress. Add a no-progress guard: count consecutive automatic compactions whose first finished successor step still overflows the model's usable window. Once that streak reaches MAX_FAILED_AUTO_COMPACTIONS (2), stop the loop and surface a ContextOverflowError telling the user to start a new session or switch to a larger-context model, instead of compacting again. A compaction is judged by its first finished non-summary successor step, so legitimate flows that overflow again only after pulling in new large tool results still get compacted normally. Real user turns and manual compactions reset the streak.
…reduction The summarization request sent to the compaction model had no size budget: select() capped only the retained tail, so a large session handed the whole serialized head to the summarizer in one request. When that request exceeded the model's context window, compaction failed outright with "Session too large to compact". - reduceContexts: pack the serialized head into chunk-budget-sized groups (half the compaction model's usable window), summarize each chunk, and re-summarize partial summaries until the final request fits (map-reduce). Single-chunk sessions take the previous code path unchanged. - The retained-tail budget now derives from the session model (whose window the compacted session must fit), not the compaction model. - Repeated failed auto-compactions escalate the tail budget (full, half, none) so a session whose floor exceeds the window sheds retained context before the loop guard stops it. The guard's limit becomes the ladder length (3). Refs anomalyco#48827
Contributor
|
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
This was referenced Sep 13, 2026
Contributor
|
This pull request has been automatically closed because it was not updated to meet our contributing guidelines within the 2-hour window. Feel free to open a new pull request that follows our guidelines. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Feat: chunk oversized compaction requests and escalate tail reduction
Fixes #48844
Depends on / includes #48828 — the first commit on this branch is that PR's loop guard; the escalation ladder builds on its streak machinery. If #48828 merges first, this PR's diff reduces to the second commit.
Problem
Compaction fails exactly when it is most needed.
select()budgets only the retained tail; the serialized head is sent to the summarizer as a single unbudgeted request. A session that outgrows its model's window therefore produces a summarization request that doesn't fit, and compaction dies withSession too large to compact(observed live on a ~60k-token session with a 64k-context model). Two further issues compound it:agent.compaction.model), the retained-tail budget was derived from the compaction model's window instead of the session model's — e.g. a 1M-window compactor retains a 15k-token tail for a 64k-window session, which then permanently overflows.Fix
Chunked (map-reduce) summarization. New
reduceContextsstep between serialization and the persisted summary call:0.5 × usable windowof the compaction model (estimate-based, with a 2× safety margin against the 4-chars/token heuristic).LLM.streamcall; partial summaries that still exceed the budget are packed and re-summarized (≤ 4 merge rounds, practically 1).context: 0) skip chunking entirely, preserving current behavior.Session-model tail budget with an escalation ladder.
select()now sizes the retained tail from the session model (whose window the compacted session must fit), and repeated failed auto-compactions shrink it: full → half → none (retaining nothing means the compacted context is summary-only). The streak comes from the #48828failedAutoCompactionsmachinery; the loop guard's limit becomes the ladder length (3 attempts, then the session stops with an actionable error).Validation
chunkSerializedpacking/truncation (4 cases),preserveRecentBudgetescalation ladder incl. clamping and explicit-config scaling (4 cases).tail_start_idunset).tsgo --noEmitclean.deepseek/deepseek-v4-flash(1M) and session modelcerebras/qwen-3.8-27b(64k), a ~60k-token session that previously died withSession too large to compactnow takes the single-request path (the head fits the compactor's 1M window), and the session-model-derived, escalating tail budget lets the compacted session fit qwen's 33.5k usable window. The integration tests cover the chunked path with a 20k-context model.Follow-ups (not in this PR)
@opencode-ai/core/session/compaction) fails safe (return false) on oversized prompts but doesn't chunk; it could adopt the same map-reduce helper.