-
Notifications
You must be signed in to change notification settings - Fork 2
enhancement(context): Soft compaction tier rarely fires in practice — window too narrow for typical file reads #1828
Copy link
Copy link
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Observation
In live testing with context_budget_tokens=12000 and hard_compaction_threshold=0.80:
- Soft threshold: 70% × 12000 = 8400 tokens
- Hard threshold: 80% × 12000 = 9600 tokens
- Window between Soft and Hard: 1200 tokens (10% of budget)
Any tool output larger than 1200 tokens causes the context to jump directly from below-soft to above-hard in a single iteration, bypassing the Soft tier entirely. Most file reads in practice produce >1200 tokens of output (e.g., reading context_manager.rs at 142K jumps context from 7383 to 19310).
In two separate tiered compaction test sessions, Soft tier was never observed. The Hard tier was observed in both sessions.
Impact
Soft compaction was introduced in PR #1720 as a lightweight first response (prune tool outputs + apply deferred summaries, no LLM call). In practice it appears to be rarely if ever triggered because:
- The Tier-0 mechanisms (inline pruning + deferred summaries) keep context below the Soft threshold during normal growth
- When Tier-0 is insufficient (large file reads), the context jumps past both Soft AND Hard in a single check
Proposed fix
Consider one or more:
- Widen the window: set Soft=0.60, Hard=0.80 (20% window vs. current 10%)
- Soft tier on count pressure: trigger Soft tier when
pair_count > cutoff(independent of token pressure) — currently Tier-0 handles this, but making it explicit inmaybe_compact()would ensure Soft fires on count pressure too - Mid-iteration compaction check: run
maybe_compact()after each individual tool call result is received, not only between complete turns — this would catch large single-file reads before they exceed the Hard threshold
Evidence
# Session 1 (budget=8000, hard=0.75):
context budget check: None cached_tokens=315 soft_threshold=5600
[large file read]
context budget check: Hard tier cached_tokens=6457 hard_threshold=6000
# Session 2 (budget=12000, hard=0.80):
context budget check: None cached_tokens=7383 soft_threshold=8400
[reading mod.rs 142K]
context budget check: Hard tier cached_tokens=19310 hard_threshold=9600
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request