Repository navigation
Description
What
compaction.buffer accepts only an absolute token count (NonNegativeInt). Its default is proportional — limit.input - max(15% of limit.input, 16_000) — but the moment buffer is set explicitly that proportional behaviour is discarded and one absolute number applies to every model in the config.
Why that's a problem
A single role commonly mixes model windows of very different sizes (e.g. a 131K local model and a 1M cloud model in the same opencode.jsonc). One flat reserve cannot serve both:
| model | usable input | buffer: 30000 |
reserve as share of input |
|---|---|---|---|
| local 131K | 98,304 | 68,304 | 30.5% |
| cloud 1M | 980,000 | 950,000 | 3.1% |
A buffer large enough to protect a slow local model's summarization request pushes that model down to ~52% of its window; a buffer small enough to be negligible for the cloud model leaves the local summarizer no headroom. There is no value that is correct for both.
Evidence from a real deployment
Raising buffer 10,000 → 30,000 fixed timeouts on one single-GPU local model. Because the key is global, it silently moved every model's trigger on that role: the local 131K model dropped from ~84% of its usable input to ~52% of its window. Nothing in the config said 52% — the flat buffer did. Diagnosing it required reading calculateCeiling() and the config backup history.
Related: a flat buffer also interacts badly with a per-model limit.input pin, because the trigger is limit.input - buffer while the UI gauge divides by limit.context. Two models with identical buffer values show completely different "percent used" at the moment they compact.
Proposal
Any one of these would remove the footgun:
- Accept a percentage, applied per model against
limit.input:"compaction": { "buffer": "10%" }
- Per-model override, mirroring the existing
limitblock:"provider": { "x": { "models": { "m": { "limit": { ... }, "compaction": { "buffer": 10000 } } } } }
- At minimum, warn when
bufferis set on a config whose models span very different windows. Today an unknown key undercompactionis silently dropped, so a typo'd percentage key looks configured and does nothing.
Repro
{
"compaction": { "auto": true, "buffer": 30000, "keep": { "tokens": 15000 } },
"provider": {
"local": { "models": { "small": { "limit": { "context": 131072, "input": 98304, "output": 32768 } } } },
"cloud": { "models": { "large": { "limit": { "context": 1000000, "input": 980000, "output": 131072 } } } }
}
}Compaction then fires at 68,304 for small (52% of its window) and 950,000 for large (95% of its window).
Version
v2.1.4 (also present on dev): packages/schema/src/config/compaction.ts declares buffer: NonNegativeInt; RESERVE_SHARE = 0.15 and RESERVE_MIN = 16_000 are module constants in packages/core/src/session/compaction.ts, not config keys
Activity
opencodejr commentedon Oct 28, 2026
Thanks for the detailed write-up and the worked numbers!
I think this is a duplicate of #43703, which asks for the same thing: a compaction buffer per model, so models with very different windows don't have to share one global value. Possible duplicates and related items, ordered from most to least likely:
reserved/buffer). It describes the same problem of mixing a small local model and a large cloud model.bufferto a model's or provider'ssettings.compaction(providers.<p>.models.<m>.settings.compaction.buffer), falling back to the top-levelcompaction.buffer. This is essentially your option 2.The warning you suggest for unknown keys under
compactionbeing dropped silently (option 3) isn't covered by these. If you'd like it tracked, a separate, focused issue would be best.The bot will close this issue as a duplicate in 1 day unless you reply explaining how your problem is different.
opencodejr commentedon Oct 29, 2026
Closing this as a duplicate of #43703, since there's been no reply in the past day. #43703 covers the same problem: a single global
compaction.buffercan't serve models with very different context windows. Open PR #56443 adds a per-model and per-providerbufferfor it. Please follow those for updates.If your case turns out to be different, or you'd like the warning about unknown
compactionkeys being dropped tracked on its own, please comment here or open a focused issue. Thanks again for the detailed report!