-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
compaction: absolute token thresholds break when switching models with different context windows #87136
Copy link
Copy link
Open
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.Auth, provider routing, model choice, or SecretRef resolution may break.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Description
Metadata
Metadata
Assignees
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.Auth, provider routing, model choice, or SecretRef resolution may break.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Type
Fields
Priority
None yet
Problem
Four compaction parameters use absolute token counts, not relative percentages of the context window:
softThresholdTokens(400K)reserveTokens(40K)keepRecentTokens(20K)reserveTokensFloor(30K)Meanwhile,
contextThreshold(0.75) andmaxHistoryShare(0.8) correctly use ratios that scale with the context window.Impact
When switching a session from a 1M-token model (e.g. deepseek-v4-pro) to a 200K-token model (e.g. zai/glm-5.1),
softThresholdTokens=400000exceeds the entire 200K context window. This causes immediate memory flush on every turn, preventing the context from ever building up — effectively rendering LCM unusable.Every model change with a different context window requires manually editing the JSON config and restarting the gateway. This is error-prone and creates a silent footgun.
Proposed Fix
Allow percentage strings for these parameters (similar to how
contextThresholdworks):{ "softThresholdTokens": "40%", "reserveTokens": "4%", "keepRecentTokens": "2%", "reserveTokensFloor": "3%" }Resolved at runtime against the active model's context window. Backward-compatible: integer values keep the current behavior.
Workaround (current)
Manual JSON edit + gateway restart per model switch. Not sustainable.
Environment