Repository navigation
[FEATURE]: per-model compaction threshold (reserved/buffer) #43703
Description
Feature hasn't been suggested before.
- I have verified this feature I'm about to request hasn't been suggested before.
Describe the enhancement you want to request
Summary
Allow configuring compaction thresholds per model instead of a single global value.
Currently the top-level compaction block (auto, prune, tail_turns, preserve_recent_tokens, reserved) is global — one value applied to every model/agent. There is no per-model or per-agent compaction field in the schema.
Use case
Users switch between models with very different context windows (e.g. a 32k local model and a 128k+ frontier model). A single reserved token buffer means the effective trigger point scales linearly with the model's context limit, which is not what we want: we want to compact a small model earlier (relative to its window) and a large model later, or vice-versa — independent reserved/buffer per model.
Current workaround (and its limitation)
The effective trigger is approximately model.limit.context − reserved. By setting a per-model limit.context in provider.<name>.models.<id>, the trigger point can be shifted per model. However, reserved (the headroom) stays a single global value, so you cannot set an independent buffer per model.
Proposed API
Something like a per-model override, e.g.:
{
"provider": {
"my-provider": {
"models": {
"small-model": {
"limit": { "context": 32000, "output": 4096 },
"compaction": { "auto": true, "reserved": 4000 }
},
"big-model": {
"limit": { "context": 128000, "output": 8192 },
"compaction": { "auto": true, "reserved": 24000 }
}
}
}
}
}or a hook that receives the model and returns the compaction settings, so plugins can compute a threshold at runtime.
Benefits
- Predictable, model-aware compaction behavior for multi-model setups.
- No wasted context on large models or over-aggressive compaction on small models.
- Enables provider-level defaults with per-model overrides, consistent with how
limitalready works.
Activity
github-actions commentedon Aug 20, 2026
This issue might be a duplicate of or closely related to existing issues. Please check:
compaction.reservedsilently ignored for models withoutlimit.input— directly related toreservednot working as expected per-modelIf those don't cover your use case, this issue is distinct enough to stand on its own — the specific ask for per-model
compactionoverrides insideprovider.<name>.models.<id>doesn't appear to have an exact match.zce commentedon Aug 30, 2026
Related context from #44094: if per-model compaction thresholds become a thing, the compaction model's own context window needs to participate in the preflight ceiling too — otherwise a smaller-window compaction model overflows exactly when compaction is needed most (see my comment on #44094 and #46164).
PzaThief commentedon Oct 17, 2026
Opened #56443 with a V2 implementation.
It adds an optional
bufferto the existing provider/modelsettings.compactionpolicy. The effective value is model setting, then provider setting, then the top-levelcompaction.buffer.bufferstays an absolute token count.Feedback on the config shape is welcome.
opencodejr commentedon Oct 28, 2026
Thanks for the clear write-up and the example config!
You're asking for the compaction buffer (
reservedin older configs,compaction.bufferin v2) to be set per model or provider, so a small local model and a large frontier model can each have their own headroom instead of sharing one global value. Right now v2 has a single top-levelcompaction.bufferfor every model. The per-modelsettings.compactiondoesn't have a buffer field yet. This can't be done with a plugin today either, because the plugin compaction settings are global and plugins aren't told which model is active.The maintainers will consider this. Related items:
settings.compaction.buffer, falling back to the globalcompaction.buffer, and links to this issuepma1999 commentedon Nov 5, 2026
Real-world case for this: one global
buffer: "55%"keeps a 1M model at 450K but forces a 200K model to compact at 90K. Absolute per-model buffers would fix it cleanly: 200K -> 50K buffer (compact at 150K), 628K -> 345K (283K), 1M -> 550K (450K).Is the
settings.compaction.buffershape in #56443 (absolute tokens, model -> provider -> global fallback) the one you'd approve? If so, I can help test or refresh that branch.ksmeltzer commentedon Nov 10, 2026
We hit the same problem with 1M-context models where inflating
bufferhurts smaller models, so I opened #61728 with a complementary approach: a configurablecompaction.window(token ceiling for automatic compaction) rather than scalingbuffer.It can be set globally (
compaction.window), per provider/model/variant (settings.compaction.window), or per agent (agents.<id>.compaction.window), with precedenceagent -> model/provider -> global(capped at the model's actual limit, while leaving the summary request's own budget at the model's real limit). That lets you set a global ceiling like"window": 180000with a normal"buffer": 20000, or cap/opt-in specific models and agents directly.