Skip to content

[Feature]: Add compaction.model config to use faster model for summarization #43834

Description

@stwith

Summary

Add agents.defaults.compaction.model config option to specify a different (faster) model for compaction summarization.

Problem to solve

Compaction uses the session's primary model for summarization. When the session uses a slow model like Opus, compaction becomes extremely slow:

  • Each chunk summarization requires a full LLM API call
  • Large sessions (150k+ tokens) split into 2-3 chunks of ~80k tokens each
  • With qualityGuard retries, this can mean 4-6 Opus API calls
  • Each Opus call for 80k tokens takes 1-3 minutes
  • Total compaction time: 8-15+ minutes, hitting timeout limits

Real-world example:

  • Session: 4.5MB JSONL, ~160k tokens, claude-opus-4-5
  • Compaction timed out at 900s (15 min)
  • Even after increasing timeout to 1800s, large sessions remain at risk

Proposed solution

Add a compaction.model config option:

{
  "agents": {
    "defaults": {
      "compaction": {
        "model": "anthropic/claude-sonnet-4-5",  // or "google/gemini-2.0-flash"
        // ... other compaction settings
      }
    }
  }
}

Behavior:

  • If compaction.model is set, use that model for generateSummary() calls
  • If not set, fall back to session's model (current behavior)
  • The model should have sufficient context window for the chunk size

Alternatives considered

Approach Why weaker
Increase timeout Delays but doesn't fundamentally solve; 30+ min compaction is poor UX
Disable qualityGuard Reduces calls by half, but Opus is still slow
Increase reserveTokensFloor Triggers compaction earlier with smaller batches, but multiple compactions still accumulate
User manually runs /compact early Relies on user vigilance, doesn't scale

Impact

  • Affected: All users with Opus/slow models and long-running sessions
  • Severity: High — compaction timeout causes sessions to get permanently stuck in processing state
  • Frequency: Every session that exceeds ~150k tokens
  • Consequence: Stuck sessions, lost messages, requires manual session reset

Evidence/examples

From OpenClaw 2026.3.7 logs:

embedded run timeout: sessionId=985aa0c1-... timeoutMs=900000
using current snapshot: timed out during compaction

Session details:

  • 4.5MB JSONL, 940 lines, 3 prior compactions
  • Model: claude-opus-4-5
  • Timeout: 900s → increased to 1800s as workaround

Additional information

  • Sonnet is ~3-5x faster than Opus for summarization tasks
  • Flash/Gemini would be even faster
  • Summarization doesn't require Opus-level reasoning depth
  • This would also reduce API costs for compaction

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions