Repository navigation
fix(session): stop looping auto-compaction when it cannot restore context headroom - #48828
Closed
whitesm2000 wants to merge 1 commit into
Closed
whitesm2000 wants to merge 1 commit into
whitesm2000 wants to merge 1 commit into
Conversation
…text headroom When a session's irreducible context (system prompt plus retained tail) exceeds the model's usable window, every step finishes over the overflow threshold and the loop auto-compacts after every single step forever, burning provider quota (and tripping rate limits) without making progress. Add a no-progress guard: count consecutive automatic compactions whose first finished successor step still overflows the model's usable window. Once that streak reaches MAX_FAILED_AUTO_COMPACTIONS (2), stop the loop and surface a ContextOverflowError telling the user to start a new session or switch to a larger-context model, instead of compacting again. A compaction is judged by its first finished non-summary successor step, so legitimate flows that overflow again only after pulling in new large tool results still get compacted normally. Real user turns and manual compactions reset the streak.
Contributor
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
Contributor
|
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
Contributor
|
This pull request has been automatically closed because it was not updated to meet our contributing guidelines within the 2-hour window. Feel free to open a new pull request that follows our guidelines. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix: stop looping auto-compaction when it cannot restore context headroom
Fixes 48827
Problem
When a session's irreducible context (system prompt + retained recent tail) exceeds the model's usable window (
context - min(limit.output, OUTPUT_TOKEN_MAX)), auto-compaction can never bring the session back under the overflow threshold. The session loop then compacts after every step forever: compaction → synthetic "Continue if you have next steps…" → overflowing step → compaction → …Concretely observed with
cerebras/qwen-3.8-27b(context 65,536 → usable 33,536) on a session whose post-compaction floor was ~43k tokens: an unattended compaction↔build ping-pong running 20+ cycles per minute, each cycle issuing a full-context API call, until it tripped the provider's TPM rate limit. The user sees a spinner and no error; meanwhile the auto-continue keeps the agent executing tool calls autonomously on every cycle.Fix
New no-progress guard, judged by outcome rather than count:
SessionCompaction.failedAutoCompactions(msgs, over)— pure helper: walks the session history and counts the trailing run of automatic compactions whose first finished non-summary successor step still overflows the model's usable window. A real user turn, a manual compaction, or a successful compaction resets the run.SessionCompaction.failedStreak({ sessionID, model })— service method that applies the helper to the full (unfiltered, chronological) session history using the sameisOverflowinputs as the trigger (cfg+outputTokenMaxaware).Both auto-compaction trigger sites in
session/prompt.ts(the start-of-iterationisOverflowcheck and the processor's"compact"outcome) now consult the streak before creating another compaction. When it reachesMAX_FAILED_AUTO_COMPACTIONS(2), the loop stops and publishes aContextOverflowError:Judging by the first successor step is what avoids false positives: legitimate flows that overflow again only after pulling in a new large tool result have a healthy first post-compaction step, so they compact normally. Because the streak is computed from persisted history (not loop-local state), the guard still trips promptly even if the session loop was restarted in between (app relaunch).
Validation
failedAutoCompactions(consecutive failures, synthetic-continue messages don't reset, real user turns reset, manual compactions ignored, summary steps skipped as successors, no-successor edge, later-overflow-doesn't-mark-good-compaction) plus a DB-backedfailedStreaktest building a real ping-pong history. Full package suite: 3590 pass / 0 fail,tsgo --noEmitclean.