docs: compaction tuning guide with per-tier presets and cache economics by 100yenadmin · Pull Request #308 · Martian-Engineering/lossless-claw

100yenadmin · 2026-04-07T06:20:11Z

Summary

Comprehensive operator-facing documentation for tuning LCM compaction behavior — model selection, cache economics, per-tier presets, debugging, and the full guard evaluation flow. No code changes.

Part 3 of 3 from #289 split. Merge after #306 and #307.

Merge order: #306 (guards + config) → #307 (engine wiring + logging) → #308 (this docs PR). The config fields, guard logic, and log lines documented here are implemented in #306/#307.

Why This Documentation Matters

Compaction tuning is the highest-leverage configuration in LCM. The wrong settings cost real money:

Wrong compaction model: Using Opus ($0.16/call) instead of GPT-4o-mini ($0.004/call) costs 40x more per compaction pass. A full sweep with 10 passes: $1.60 vs $0.04.
Wrong skip thresholds: Compacting when the reduction is only 3% of context invalidates the prompt cache. On Opus with 150K cached, one unnecessary miss costs $0.68.
Wrong expansion model: Each lcm_expand_query spawns a 3-8 turn sub-agent. On Opus, that's $0.15-0.50 per call. Setting expansionModel to Haiku drops it to $0.03-0.10.
No tuning at all: A 200-turn Opus session costs ~$4 in input tokens without tuning. With recommended config + Sonnet compaction model: ~$1.80. Net savings: ~$1.80/session (45%).

None of this is discoverable from the code alone.

New Content

`docs/compaction-tuning.md` (356 lines, NEW)

The main guide, structured for both quick-start and deep understanding:

TLDR section — Copy-paste configs for 4 common scenarios:

Opus 1M coding (Haiku compaction, 0.02 skip threshold, 0.55 headroom)
Sonnet 1M general (defaults work)
Haiku quick tasks (0.10 skip threshold, 0.90 headroom)
Agent orchestration (Sonnet compaction, 0.02 skip, 0.60 headroom)

How it works — Full lifecycle with Mermaid diagrams:

Compaction lifecycle (ingest → leaf trigger → full sweep → assembly)
Summary hierarchy visualization (raw → leaf → condensed)
Why compaction invalidates the prompt cache (ordinal resequencing)
Timing: leaf is async/best-effort, full sweep is sync/blocking

Cache economics — Concrete cost tables:

Cache miss penalty per model tier (Opus: $0.68/miss, Sonnet: $0.41, Haiku: $0.14)
Break-even formula with worked example (150K cached, 10K saved = 13.5 turns)
Per-tier preset rationale (why Opus uses 0.02 threshold but Haiku uses 0.10)

Model selection — Comparison table of 10 compaction model options:

GPT-4o-mini ($0.004/call) through Opus ($0.160/call)
Latency, context window, caching behavior for each
"Never use" list with reasons (Opus, o3, thinking models)

Guard evaluation flow — Mermaid flowchart showing:

Raw threshold → headroom gate → budget pressure → cache-aware skip
Design principles: budget pressure always wins, cache skip is conservative, per-pass estimation

Debugging checklist — Three common problems:

"Compaction never fires" (headroom too high, chunk too large)
"Compaction fires every turn" (chunk too low, skip disabled)
"Session hangs during compaction" (wrong model, no timeout)

References — Links to the LCM paper, arXiv research, Anthropic/OpenAI caching docs.

Updated docs

File	Change
`docs/architecture.md`	+24 lines. New "Cache-aware compaction guards" section with Mermaid flowchart showing guard evaluation order. Explains headroom gate, budget pressure override, cache-aware skip, and prompt cache impact ($0.68/miss on Opus).
`docs/configuration.md`	+54 lines. New "Cache-aware compaction tuning" section with settings table, escape hatches (set to 0 to disable), model comparison table (10 models with cost/latency/context), and cache economics (break-even formula, miss costs). Clarified quick-start snippet is a recommended override, not defaults.
`skills/lossless-claw/references/config.md`	+18 lines. Config field reference for `leafSkipReductionThreshold` and `leafBudgetHeadroomFactor` with defaults, ranges, and env var names.

New comprehensive guide for operators tuning LCM compaction behavior: - docs/compaction-tuning.md (356 lines): TLDR, per-tier model presets (Opus, Sonnet, Haiku, GPT-4o-mini, Gemini Flash), cache economics break-even formula, debugging checklist, orchestration scenarios - docs/architecture.md: cache-aware guards section with Mermaid flowchart - docs/configuration.md: new settings reference, model comparison table - skills references: config field updates Split from Martian-Engineering#289 (Part 3 of 3). Independent of Martian-Engineering#306 and Martian-Engineering#307.

Copilot

Pull request overview

Adds operator-facing documentation for tuning Lossless Claw compaction behavior, including a new compaction tuning guide and updates to existing docs/reference materials.

Changes:

Add new docs/compaction-tuning.md guide with presets, cache economics, and troubleshooting.
Update docs/configuration.md and docs/architecture.md with new compaction tuning/guard explanations.
Update skill config reference with new compaction-related fields.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
`docs/compaction-tuning.md`	New tuning guide describing cache-aware compaction behavior, presets, and economics.
`docs/configuration.md`	Updates defaults and adds a new “cache-aware compaction tuning” section.
`docs/architecture.md`	Adds an architecture section describing cache-aware compaction guard ordering with a Mermaid diagram.
`skills/lossless-claw/references/config.md`	Documents new config fields related to cache-aware compaction skipping.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…ineering#308)

- Change "Never use your main model" to "Avoid using your main model when it's expensive or slow" — consistent with later recommending Haiku/Sonnet which may be someone's main model - Add clarifying note that quick-start snippet values are recommended overrides, not defaults (freshTailCount=32 vs default 64)

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- TLDR example: "Sonnet" → "Haiku" to match the copy-paste config - Debug section: make log line guidance more resilient to wording changes - Architecture flowchart: "raw tokens" → "raw tokens outside fresh tail"

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Add LCM_LEAF_MIN_FANOUT, LCM_CONDENSED_MIN_FANOUT, and LCM_CONDENSED_MIN_FANOUT_HARD to the settings table - Replace hard-coded "5%" in flowcharts with "leafSkipReductionThreshold (default 5%)" in both compaction-tuning.md and architecture.md

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…n comparison table - README env var table: LCM_LEAF_TARGET_TOKENS default was 1200, config resolver uses 2400. Fixed to match code. - Model comparison table: add note that names are marketing names, use provider-qualified IDs for config.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Adversarial audit findings: - Remove unverifiable "$4 for 200-turn session" claim, replace with paper-backed reduction ranges (20-39% avg, up to 86%) - Fix gpt-4.1-mini pricing: $0.40/$1.60 → $0.20/$0.80 - Fix mistral-small pricing: $0.15/$0.60 → $0.20/$0.60 - Add gpt-4.1-nano ($0.10/$0.40) — cheapest available option - Add config IDs next to marketing names in model table - Remove MiniMax-Text-01 (unverifiable pricing) - Add cache write premium note to break-even formula (1.25x for 5-min TTL adds ~0.7 turns to break-even) - Sort table by cost/call ascending for easier comparison

…maryModel

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…ineering#308)

jaspmf · 2026-04-08T04:37:17Z

This is pure gold, thank you.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+| Model | Cost/call | Context | Notes |
+|-------|-----------|---------|-------|
+| `gpt-4o-mini` | ~$0.004 | 128K | Cheapest, auto caching |
+| `mistral-small-4` | ~$0.004 | 256K | Same price, bigger context |


+   ~2400 tokens         ~2400 tokens        ├── fresh tail ──┤
+
+After condensation (depth 1):
+  [condensed₁: leafs 1-3] [leaf₄] [leaf₅] [msg₄₁] ... [msg₅₀]


+When compaction runs on the main agent session, it stalls all connected sessions sharing that thread. To prevent this:
+
+1. **Isolate sub-agent sessions** — Configure `ignoreSessionPatterns` or `statelessSessionPatterns` to prevent sub-agents from triggering compaction
+2. **Use shorter timeouts** — Set `summaryTimeoutMs` to 30000 (30s) so failed compaction releases quickly
+3. **Choose fast models** — A 0.5s Haiku call is invisible even without isolation
+
+```json
+{
+  "summaryModel": "claude-haiku-4-5",
+  "summaryProvider": "anthropic",
+  "summaryTimeoutMs": 30000,
+  "ignoreSessionPatterns": ["agent:*:cron:**"],
+  "statelessSessionPatterns": ["agent:*:subagent:**"]


+**Fix:** Set `expansionModel` to a cheaper model:
+
+```json
+{
+  "expansionModel": "claude-haiku-4-5",
+  "expansionProvider": "anthropic"
+}


+2. Receives ~600-2400 output tokens (the summary)
+3. Blocks until complete (full sweep is synchronous)
+
+**Compaction model comparison** (cost per call = 20K input + 2.4K output). Model names below are marketing names — use the provider-qualified config ID (e.g., `gpt-4o-mini`, `claude-haiku-4-5`) when setting `summaryModel`:


Copilot AI review requested due to automatic review settings April 7, 2026 06:20

100yenadmin mentioned this pull request Apr 7, 2026

feat: cache-aware compaction guards with budget-pressure priority and per-tier tuning #289

Closed

Copilot started reviewing on behalf of 100yenadmin April 7, 2026 06:20 View session