Skip to content

Fail fast when fixed request overhead exceeds the model's usable context window #48847

Closed as not planned

Description

@whitesm2000

Fail fast when fixed request overhead exceeds the model's usable context window

Summary

Follow-up to #48827 (auto-compaction loop guard) and #48844 (chunked compaction + tail escalation, PR #48845). With both in place, a third failure mode remains in this family: when a model's usable window (limit.context − maxOutputTokens, session/overflow.ts) is smaller than the request's fixed portion — system prompt + tool/MCP schemas + project instructions — every request overflows regardless of conversation content. Compaction can only shrink conversation content, never the fixed portion, so the outcome is knowable before the first attempt — yet the session loop still runs the full ladder: auto-compact → resume → overflow → escalate retained tail (full → half → none) → repeat, until the loop guard stops the session at attempt 3.

Evidence (live, desktop v1.18.30 with #48845 applied)

Project fixed floor — provider-reported tokens on the first request of fresh sessions, before any conversation content: 34,474–34,771 tokens.

cerebras/qwen-3.8-27b: limit.context = 65,536, limit.output = 32,768 → usable = 65,536 − min(32,768, 32,000) = 33,536.

Floor (34.5k) > usable (33.5k) — even an empty session overflows on its first turn in this project.

Observed sequence on a ~106k-token session after switching to this model:

  1. Auto-compaction succeeded: the 105,817-token head was summarized in a single request (1M-window compaction model deepseek/deepseek-v4-flash), producing a 3,254-token summary — the feat(session): chunk oversized compaction requests and escalate tail reduction #48845 path working as designed.
  2. The resumed request registered 54,499 tokens > 33,536 → second auto-compaction (input 11,516 — only the new remainder; tail escalation engaged).
  3. The resumed request still overflowed; the user cancelled before the guard's third attempt.

Every component behaved as designed, but ~117k compaction input tokens and ~40s of "compacting" were spent reaching an outcome that was determined before attempt 1.

Proposed behavior

Detect the impossible fit and fail fast with a targeted error, e.g.:

This model's usable context window (33,536 tokens: 65,536 context − 32,000 output reservation) is smaller than this project's baseline request size (~34,500 tokens of system prompt, tool definitions, and project instructions). Compaction cannot help — choose a model with a larger context window, or reduce enabled tools/MCP servers and instruction files.

Detection sketch: at the auto-compaction trigger in session/prompt.ts (or inside SessionCompaction.select()), estimate the fixed portion — Token.estimate over system prompt + tool schemas + instruction files — and compare it against usable(model). If the floor exceeds usable (or usable minus a minimal summary allowance), skip compaction entirely and raise the error. The same check could later power a proactive warning when the user switches to an undersized model.

Related

Environment

  • OpenCode desktop v1.18.30 (macOS arm64), session model cerebras/qwen-3.8-27b, compaction model deepseek/deepseek-v4-flash, project with MCP servers and instruction files enabled.

Activity

  1. thdxr commented on Oct 11, 2026

    @thdxr
    Member

    Thanks for the report! OpenCode 2.0 is now out and development is focused on V2, so we're closing issues filed against OpenCode 1. Please upgrade (opencode upgrade --major, details: https://anoma.ly/notes/opencode-2-0), and if this is still relevant on 2.0, reopen this issue or open a new one with your 2.0 version.

    — from 𝕺𝖕𝖊𝖓𝕮𝖔𝖉𝖊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions