providers: opencode-go stream aborts on transient capacity errors instead of retrying #62306

Activity

1. opencodejr commented on Nov 6, 2026

   

   opencodejrBot

   on Nov 6, 2026

   Contributor

   More actions

   Thanks for the detailed report and the log counts, they helped a lot.

   I couldn't reproduce this end to end. It needs a real OpenCode Go account at the moment the model is overloaded, and that part of the system runs on the OpenCode Go service, not in this repository. I did test the client side on 2.2.5 with a mock endpoint that sends the exact error from your logs once, then a normal reply:

   • When the error comes as HTTP 429, 503, or as an error event inside the response stream, OpenCode retries and the turn finishes.
   • When it comes as HTTP 400, OpenCode stops at once with exactly the error you saw and makes no retry.

   The second case is deliberate: OpenCode treats most 4xx responses (other than 429) as a problem with the request itself, so it doesn't retry them. Your turns dying right away with no "retrying" state matches that case. So the Go gateway most likely sends these capacity rejections with a 4xx status. Someone on the team with access to the OpenCode Go service will need to check that.

   For now, resending the message or switching models remain the workarounds. Related, but not the same problem: #51530 (OpenCode Go errors hidden behind HTTP 400) and #60753.
2. chernya2010 commented on Nov 6, 2026

   

   chernya2010

   on Nov 6, 2026

   Author

   More actions

   Thanks — that matches what we see, with one caveat from the client side.

   The stream-error log entry does not include the HTTP status or a retryability flag, so a user report can't distinguish the 400 case from the 429 case:

   timestamp=2026-11-06T07:43:39.867Z level=ERROR run=<id> message="stream error" providerID=opencode-go modelID=deepseek-v4.1-flash session.id=<redacted> small=false agent=build mode=primary error.error="AI_APICallError: Upstream request failed: [prefill_cap_refused] the cell is at capacity; retry shortly (after 5 s)"
   

   No statusCode, no isRetryable, no response headers. Adding statusCode (and Retry-After when present) to this line would make the next report conclusive and would also help separate gateway bugs from client bugs.

   Numbers, corrected from my first post: 42 stream-error entries — 23 on 2026-10-27, 18 on 2026-10-28, 1 on 2026-11-06; 41 x ttft_admission_refused, 1 x prefill_cap_refused; the embedded wait ranges from 2 s to 59 s. All on opencode-go/deepseek-v4.1-flash.

   Both variants carry an explicit wait ("retry shortly (after N s)"), so the Go gateway could return 429/503 with Retry-After and stay inside the retry path you tested. If the OpenCode Go service is the right tracker, happy to move this there — just point me at the repo.
3. wujun2081 commented on Nov 7, 2026

   

   wujun2081

   on Nov 7, 2026

   More actions

   Same failure mode on another account, adding data since this appears in bursts rather than as an account-quota issue.

   Environment: OpenCode host on OpenMausBot, client version 1.18.31, macOS (Darwin), provider opencode-go, model deepseek-v4.1-flash, active paid Go subscription.

   10 upstream capacity refusals across Nov 6-7, 2026 (UTC):

   #	When	Suggested retry
   1	2026-11-06T01:28:43Z	5 s
   2	2026-11-06T06:15:05Z	5 s
   3	2026-11-06T06:57:57Z	5 s
   4	2026-11-06T07:01:00Z	5 s
   5	2026-11-06T07:07:36Z	5 s
   6	2026-11-06T07:10:10Z	5 s
   7	2026-11-06T07:29:46Z	5 s
   8	2026-11-06T09:13:17Z	30 s
   9	2026-11-07T06:18:54Z	30 s
   10	2026-11-07T06:23:10Z	30 s

   Error text: Internal error: Upstream request failed: [prefill_cap_refused] the cell is at capacity; retry shortly (after N s).

   Note this persisted across two separate days and after idle gaps, on a paid Go plan — so it is upstream of any per-account free-tier cap. +1 to auto-retrying transient capacity refusals with the caller-suggested backoff instead of failing the turn.

   👍React with 👍1Reacted by Vladis
4. Yue-bin commented on Nov 7, 2026

   

   Yue-bin

   on Nov 7, 2026

   More actions

   Adding measurements from the server side — direct HTTPS calls to the Go endpoint with no client in the loop — since the bot's analysis points at the OpenCode Go service rather than this repository.

   Setup: POST https://opencode.ai/inference/go/v1/chat/completions, model deepseek-v4.1-flash, paid Go account, max_tokens: 4, stream: false, varying only x-opencode-session. Sizes are approximate (English filler, ~4 chars/token); times are wall clock, request → full response.

   payload	status	wall clock
   ~20 tokens × 4 different session ids	200	3.5 – 5.7 s
   ~200K tokens	200	24.5 s
   ~500K tokens	200	89.6 s

   What this says

   1. There is no hard prefill ceiling near 500K. A ~500K-token cold request returns 200 today. That agrees with the 1,000,000 context advertised by /inference/go/v1/models and with the 585K-token session in go: deepseek-v4.1-flash image rejection bricks sessions ("image input is not available on this engine") #61827.
   2. Admitted prefills cost roughly 5–8K tokens/s: ~24 s for 200K, ~90 s for 500K. A large request therefore needs a cell to hold a prefill slot for one to two minutes.
   3. The refusal is late, not early. In a real client session the 429s arrived 9 s, 9 s and 13 s after the request was sent — so the gateway spends seconds attempting admission before it refuses. An attempt that was admitted (486,795 prompt tokens, cached_tokens: 640) took 111 s to produce its first message. The client pays twice: seconds to learn "no", then ~2 minutes if it ever gets a "yes".
   4. Retrying after 5 s cannot succeed while the cell is busy, and every attempt is a fresh full prefill — cached_tokens was 640 out of 486,795, i.e. essentially no reuse. That is why "retry shortly (after 5 s)" produces a burst of identical failures instead of recovery.
   5. In this sample there was no permanent per-session pin. The session id that failed consistently two days ago returns 200 today at both ~20 tokens and ~500K. Sticky routing alone doesn't explain the failures; capacity does.

   Asks for the Go service

   • Return a status code + Retry-After (429 / 503) together with a stable machine-readable code for prefill_cap_refused / ttft_admission_refused / overloaded. Today the only signal is English prose, and per the mock test above a non-429 4xx bypasses client retry entirely.
   • If admission is all-or-nothing, prefer queueing large prefills — or refusing immediately — instead of refusing after 9–13 s of admission work.
   • Document the budget. With 1M context advertised, users cannot tell "this request is too large" from "the cell is busy right now"; that ambiguity is what turns a capacity blip into a dead turn.
   • If a retry with the same x-opencode-session always lands on the same cell, consider failing over on admission failures, or expose a way to re-pin a conversation.

   Happy to extend the size sweep or attach raw response headers / x-opencode-log-id values if that helps.
5. muaaz9329

   added a commit that references this issue on Nov 8, 2026

   fix(routing): a failing worker model retries once in a forked session…

   ...

   d3bc6ed