Skip to content

enhancement(context): Soft compaction tier rarely fires in practice — window too narrow for typical file reads #1828

@bug-ops

Description

@bug-ops

Observation

In live testing with context_budget_tokens=12000 and hard_compaction_threshold=0.80:

  • Soft threshold: 70% × 12000 = 8400 tokens
  • Hard threshold: 80% × 12000 = 9600 tokens
  • Window between Soft and Hard: 1200 tokens (10% of budget)

Any tool output larger than 1200 tokens causes the context to jump directly from below-soft to above-hard in a single iteration, bypassing the Soft tier entirely. Most file reads in practice produce >1200 tokens of output (e.g., reading context_manager.rs at 142K jumps context from 7383 to 19310).

In two separate tiered compaction test sessions, Soft tier was never observed. The Hard tier was observed in both sessions.

Impact

Soft compaction was introduced in PR #1720 as a lightweight first response (prune tool outputs + apply deferred summaries, no LLM call). In practice it appears to be rarely if ever triggered because:

  1. The Tier-0 mechanisms (inline pruning + deferred summaries) keep context below the Soft threshold during normal growth
  2. When Tier-0 is insufficient (large file reads), the context jumps past both Soft AND Hard in a single check

Proposed fix

Consider one or more:

  1. Widen the window: set Soft=0.60, Hard=0.80 (20% window vs. current 10%)
  2. Soft tier on count pressure: trigger Soft tier when pair_count > cutoff (independent of token pressure) — currently Tier-0 handles this, but making it explicit in maybe_compact() would ensure Soft fires on count pressure too
  3. Mid-iteration compaction check: run maybe_compact() after each individual tool call result is received, not only between complete turns — this would catch large single-file reads before they exceed the Hard threshold

Evidence

# Session 1 (budget=8000, hard=0.75):
context budget check: None cached_tokens=315 soft_threshold=5600
[large file read]
context budget check: Hard tier cached_tokens=6457 hard_threshold=6000

# Session 2 (budget=12000, hard=0.80):
context budget check: None cached_tokens=7383 soft_threshold=8400
[reading mod.rs 142K]
context budget check: Hard tier cached_tokens=19310 hard_threshold=9600

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions