Skip to content

Transient provider-busy 503 kills the turn with no retry - headless/ACP sessions wedge idle indefinitely #47610

Closed

Description

@taylorhou

Summary

When an OpenAI-compatible provider returns a transient 503 ("no eligible device" / model temporarily unavailable), opencode does not retry: the error kills the in-flight turn. In autonomous/headless operation (e.g. as a gascity ACP harness) there is no human to re-prompt, so the session sits idle indefinitely with its work item stuck in_progress. Fast-fail is reasonable for interactive throughput, but it is broken for liveness in unattended loops.

Environment

  • opencode 1.18.29 (ACP, subprocess provider)
  • Provider: self-hosted OpenAI-compatible endpoint with bounded concurrency (returns 503 immediately when all inference slots are busy, rather than queueing)

Repro

  1. Run opencode headless as a long-lived agent harness against a provider whose capacity is lower than the number of concurrent consumers.
  2. Two consumers hold the slots; a third's turn gets an immediate 503 "no eligible device".
  3. The turn dies. No retry, no backoff, no resume. The session remains listed as active but produces zero activity until an external supervisor kills and respawns it.
  4. Observed at scale: every lost slot race silently wedges one session; a 12-minute kill+respawn cycle is currently the only retry mechanism. Same wedge reproduced twice on a long-running research session within hours.

Suggestion

Treat provider-busy responses (503 "model unavailable"/"no eligible device", and ideally 429) as retryable within the turn: bounded exponential backoff honoring Retry-After, with a configured attempt budget (default a few minutes of coverage), before surfacing a terminal error. Interactive users can still Ctrl-C; headless loops survive transient capacity contention.

Related: #44044 (managed gateway 503s killing turns) is the same failure class on a different provider path; this report isolates the retry-semantics gap with a minimal self-hosted repro.

Activity

  1. github-actions commented on Sep 6, 2026

    @github-actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

    If your issue is specifically about the headless/ACP wedge-idle behavior (no human to re-prompt) as a distinct liveness concern on top of the retry gap, it may warrant staying open as its own tracking issue. Otherwise, consider whether one of the above covers your scenario.

  2. nexxeln commented on Oct 1, 2026

    @nexxeln
    Contributor

    hey! this should be fixed in v2 now 🙏 if you're still on 1.x, upgrading to v2 should sort it out. let us know if you still hit it after upgrading and we'll take another look!

  3. nexxeln commented on Oct 6, 2026

    @nexxeln
    Contributor

    Fixed on v2. Provider 503s are retried inside the turn with backoff that honors Retry-After, each retry is reported to ACP clients (#50752), and the prompt fails with an error once the retries run out, so a headless session no longer sits idle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions