[Bug] litellm ContextWindowExceededError not detected as context overflow #44888

Activity

1. github-actions

   assigned

   nexxeln

   on Aug 25, 2026
2. github-actions commented on Aug 25, 2026

   

   github-actionsBot

   on Aug 25, 2026 – with GitHub Actions

   Contributor

   More actions

   This issue might be a duplicate of (or closely related to) an existing issue. Please check:

   • Image-count 400s (Azure: "Exceeded maximum number of images (50)") are not classified as context overflow, so media-strip compaction never triggers #39677: Same root cause — an error not matched by the patterns in packages/llm/src/provider-error.ts means compaction never triggers. That report covers Azure's image-count 400; this one covers litellm's ContextWindowExceededError phrasing. The fix would be in the same place, so these may be best addressed together.
3. fabian33-ops commented on Sep 24, 2026

   

   fabian33-ops

   on Sep 24, 2026

   Author

   More actions

   This is not stale. Please review.
4. bacigal commented on Sep 30, 2026

   

   bacigal

   on Sep 30, 2026

   More actions

   Another live repro with OpenCode v2.0.19 behind a LiteLLM proxy (@ai-sdk/openai-compatible, fronting Azure OpenAI and self-hosted vLLM deployments). It matches the OP's scenario exactly: LiteLLM's max_input_tokens guard fires before the upstream serving engine returns its own (classifiable) overflow error.

   Recorded on an assistant message (the raw error also embeds the proxy's fallback list, omitted here):

   litellm.ContextWindowExceededError: litellm.BadRequestError: litellm._pre_call_checks: Context Window exceeded for given call. No models have context window large enough for this call.
   Model=hosted_vllm/GLM-5.2-AWQ-4bit, Max Input Tokens=300000, Got=300300
   model=glm-5.2. context_window_fallbacks=None.
   

   What happens afterwards:

   • litellm.ContextWindowExceededError is surfaced as a generic API error. No session.compaction.* events result from it, and the documented one-shot recovery ("recover once when a provider rejects a request as too long") never engages. The session is only rescued by a manual /compact.
   • Long-term data point: across ~3 months of daily use on LiteLLM-backed models I have exactly 10 compaction records — 7 reason: manual, 3 reason: auto (Jul 15, Sep 18, Sep 23). Sessions routinely reach the ceiling and dead-end with these remote 400s until I conclude them by hand; the silent-stop / repeated-error experience described above matches the low auto rate.
   • When the threshold-based proactive path also misses (see compaction: auto-compaction never triggers on hy3 — token count plateaus below the isOverflow threshold, context pinned at 196608 raw input tokens #45168 — real input ceiling below the advertised window makes usage plateau below the trigger), LiteLLM-guard errors like this one are the only remaining safety net, and they are precisely the ones not classified.

   Suggestion for the shared matcher (packages/llm/src/provider-error.ts): classify the LiteLLM overflow wrapper too — e.g. normalize "context window exceeded" / ContextWindowExceededError, and/or the very characteristic Max Input Tokens=…, Got=… token pair, into is_overflow. That single phrase would convert dead sessions into automatic compact-and-retry for anyone running models behind LiteLLM.

   Related: #46564 (opaque bare 400s not classified — same underlying gap in the matcher), #45168 (auto-compact plateau below the isOverflow threshold).

   Happy to provide more data (usage series, clean logs) if useful.

   👍React with 👍1Reacted by fabian33-ops
5. joryirving commented on Oct 17, 2026

   

   joryirving

   on Oct 17, 2026

   More actions

   Confirming from another environment (also v2.0.22, macOS arm64, OpenCode via LiteLLM proxy with enable_pre_call_checks: true): the pre-call rejection dead-ends the session exactly as described — litellm.ContextWindowExceededError: litellm.BadRequestError: litellm._pre_call_checks: Context Window exceeded for given call... surfaces as a System Error with no compaction. I filed #56403 with details; closing as a dup of this one is right.

   Two artifacts from that investigation that may help review of #54608:

   1. The shipped v2.0.22 classifier list (extracted from the installed binary) confirms none of the current patterns match the litellm router phrasing. The full list, for reference against future litellm message variants:

   Jn = [
     /prompt is too long/i,
     /input is too long for requested model/i,
     /exceeded this model context window limit/i,
     /exceeds the context window/i,
     /exceeds (?:the )?(?:model'?s )?maximum context length(?: of [\d,]+ tokens?|\s*\([\d,]+\))/i,
     /input token count.*exceeds the maximum/i,
     /number of input tokens exceeds maximum length/i,
     /tokens in request more than max tokens allowed/i,
     /maximum prompt length is \d+/i,
     /reduce the length of the messages/i,
     /requested input length \d+ exceeds maximum input length/i,
     /maximum context length is \d+ tokens/i,
     /exceeds (?:the )?maximum allowed input length of [\d,]+ tokens?/i,
     /input(?: \(\d+ tokens\))? is longer than the model'?s context length/i,
     /exceeds the limit of \d+/i,
     /exceeds the available context size/i,
     /greater than the context length/i,
     /`inputs` tokens \+ `max_new_tokens` must be <= \d+/i,
     /context window exceeds limit/i,
     /exceeded model token limit/i,
     /context[_ ]length[_ ]exceeded/i,
     /context length is only \d+ tokens/i,
     /input length.*exceeds.*context length/i,
     /too large for model with \d+ maximum context length/i,
     /prompt has [\d,]+ tokens?, but the configured context size is [\d,]+ tokens?/i,
     /model_context_window_exceeded/i,
     /range of input length should be/i,
     /too many tokens/i,
     /token limit exceeded/i,
     /request_too_large/i,
   ]

   2. The contrast case that proves the mechanism: a rejection from the backend (vLLM: This model's maximum context length is 147456 tokens. However, you requested…) matches /maximum context length is \d+ tokens/i and OpenCode logs provider rejected the request as too long; compacting and recovers. Only router-generated rejections (which never reach the backend) fall through.

   The pattern in #54608 (/context window exceeded/i) matches the exact string from my logs, so it should close this gap.
6. opencodejr commented on Oct 28, 2026

   

   opencodejrBot

   on Oct 28, 2026

   Contributor

   More actions

   Thanks for the clear report. I reproduced this on v2.2.0 and it still happens on the latest dev (a882cf9).

   Setup: a mock OpenAI-compatible endpoint stands in for the LiteLLM proxy. It answers the first chat request with HTTP 400 and LiteLLM's exact message (litellm.ContextWindowExceededError: litellm.BadRequestError: litellm._pre_call_checks: Context Window exceeded for given call. ...). Then I ran opencode run against it through @ai-sdk/openai-compatible.

   • With the LiteLLM message, OpenCode shows it as a plain error and stops. It never sends a compaction request.
   • With a vLLM-style message (This model's maximum context length is 1000 tokens...), the same setup compacts and retries right away.

   So the recovery itself works; LiteLLM's wording just isn't recognized as a context overflow. The script below reproduces it, and CONTROL=1 runs the vLLM comparison. Open PR #54608 proposes adding this pattern. Until a fix lands, running /compact by hand still works.

   Related: #56403 (same LiteLLM error, closed as a duplicate of this issue) and #46564 (another kind of overflow error that isn't recognized).

   Reproduction script (repro.sh)

   #!/usr/bin/env bash
   # Reproduces #44888: a LiteLLM pre-call "Context Window exceeded" 400 is not
   # treated as context overflow, so OpenCode never compacts and retries.
   # Usage: ./repro.sh            (uses `opencode` from PATH)
   #        CONTROL=1 ./repro.sh   (control: vLLM wording, which IS recognized and compacts)
   #        OC_DEV=/path/to/opencode-checkout ./repro.sh   (runs from source)
   set -u
   DIR=$(mktemp -d /tmp/opencode/litellm-XXXX)
   PORT=${PORT:-4517}
   cat > "$DIR/mock.ts" <<'TS'
   // Mock LiteLLM proxy: first non-title chat request -> LiteLLM ContextWindowExceededError (400),
   // all later requests (e.g. compaction / retry) -> a normal streamed reply.
   let n = 0, chats = 0
   const msg = process.env.CONTROL ? "This model's maximum context length is 1000 tokens. However, you requested 1300 tokens." : "litellm.ContextWindowExceededError: litellm.BadRequestError: litellm._pre_call_checks: Context Window exceeded for given call. No models have context window large enough for this call.\nModel=hosted_vllm/test, Max Input Tokens=1000, Got=1300\nmodel=test. context_window_fallbacks=None."
   Bun.serve({ port: Number(process.env.PORT), async fetch(req) {
     const url = new URL(req.url)
     if (url.pathname.endsWith("/models")) return Response.json({ data: [{ id: "test", object: "model" }] })
     const body = await req.text(); n++
     const j = JSON.parse(body); const sys = JSON.stringify(j.messages?.[0]?.content ?? "").slice(0, 90); const last = JSON.stringify(j.messages?.at(-1)?.content ?? "").slice(0, 90)
     console.log(`REQUEST #${n} bytes=${body.length} tools=${(j.tools ?? []).length} system=${sys} last=${last}`)
     const isTitle = sys.includes("You name conversations")
     if (!isTitle) chats++
     if (!isTitle && chats === 1) return Response.json({ error: { message: msg, type: null, param: null, code: "400" } }, { status: 400 })
     const chunk = (d: any) => `data: ${JSON.stringify({ id: "x", object: "chat.completion.chunk", created: 0, model: "test", choices: [{ index: 0, ...d }] })}\n\n`
     return new Response(chunk({ delta: { role: "assistant", content: "summary/ok" } }) + chunk({ delta: {}, finish_reason: "stop" }) + "data: [DONE]\n\n", { headers: { "content-type": "text/event-stream" } })
   } })
   TS
   CONTROL=${CONTROL:-} PORT=$PORT bun "$DIR/mock.ts" > "$DIR/mock.log" 2>&1 &
   MOCK=$!; sleep 1
   mkdir -p "$DIR/proj"; cd "$DIR/proj"
   cat > opencode.json <<JSON
   { "\$schema": "https://opencode.ai/config.json",
     "provider": { "litellm": { "npm": "@ai-sdk/openai-compatible", "name": "LiteLLM",
       "options": { "baseURL": "http://127.0.0.1:$PORT/v1", "apiKey": "x" },
       "models": { "test": { "name": "test", "limit": { "context": 200000, "output": 1000 } } } } } }
   JSON
   export HOME="$DIR/home" XDG_CONFIG_HOME="$DIR/home/.config" XDG_DATA_HOME="$DIR/home/.local/share" XDG_STATE_HOME="$DIR/home/.local/state" XDG_CACHE_HOME="$DIR/home/.cache"
   if [ -n "${OC_DEV:-}" ]; then
     export BUN_OPTIONS="--conditions=browser --preload=$OC_DEV/packages/tui/node_modules/@opentui/solid/scripts/preload.js"
     OC=(bun --conditions=browser --preload="$OC_DEV/packages/tui/node_modules/@opentui/solid/scripts/preload.js" "$OC_DEV/packages/cli/src/index.ts")
   else OC=(opencode); fi
   echo "== version: $("${OC[@]}" --version)"
   timeout 120 "${OC[@]}" run --standalone --model litellm/test "hello" > "$DIR/run.log" 2>&1
   echo "== exit: $?"
   echo "== opencode output:"; cat "$DIR/run.log"
   echo "== mock requests:"; cat "$DIR/mock.log"
   kill $MOCK

   Run it: bash repro.sh
7. opencodejr

   added

   reproduced

   on Oct 28, 2026
8. rekram1-node

   closed this as completedin #60692on Oct 30, 2026