Skip to content

Sessions fail on transient network errors instead of retrying #30611

Closed

Description

@literally-dan

Description

The session retry path only treats ECONNRESET as retryable. Other transient transport failures are classified as hard errors, so a brief network problem kills the assistant turn instead of being retried by the existing retry policy.

Cases that currently fail the session outright include system error codes such as ETIMEDOUT, ENOTFOUND, EAI_AGAIN, ECONNREFUSED, EHOSTUNREACH, ENETUNREACH, and EPIPE; request timeouts surfaced as a TimeoutError; and bare fetch/undici errors whose only signal is the message text, such as fetch failed, socket hang up, terminated, and other side closed.

On a flaky connection this shows up as the turn ending with an error that a retry would have recovered from.

This is a different layer from #21893 (provider/stream-layer stream_read_error and wrapped rate-limit errors) and from #20822 (a broader "retry UnknownError by default" strategy). This issue is specifically about transport/socket-level failures in MessageV2.fromError.

I have a fix that widens the retryable classification while keeping a real user cancel non-retryable.

Plugins

None

OpenCode version

1.15.13

Steps to reproduce

  1. Start an assistant turn that makes a provider request.
  2. Trigger a transient transport failure (drop the network briefly, or force a DNS failure / connection timeout) so the error surfaces as one of the system codes above, a TimeoutError, or a bare fetch failed / socket hang up.
  3. The session ends in error instead of entering retry, even though the existing retry policy would have recovered.

Screenshot and/or share link

No response

Operating System

Linux 6.17.0-29-generic

Terminal

foot

Activity

  1. github-actions commented on Jun 3, 2026

    @github-actions
    Contributor

    This issue doesn't fully meet our contributing guidelines.

    What needs to be fixed:

    • Missing required bug report template fields: OpenCode version, Operating System, Terminal, and Steps to Reproduce. Please use the full bug report template and include these details so maintainers can reproduce and triage the issue.

    Please edit this issue to address the above within 2 hours, or it will be automatically closed.


    This issue might be a duplicate of existing issues. Please check:

    If you believe this was flagged incorrectly, please let a maintainer know.

  2. literally-dan commented on Jun 3, 2026

    @literally-dan
    Author

    Closing this, I opened it before it was ready for review. I'll reopen once I've gone over it properly.

  3. github-actions commented on Jun 3, 2026

    @github-actions
    Contributor

    Thanks for updating the issue! Everything looks good now — the description is clear, steps to reproduce are included, and system info is provided. The needs:compliance label has been removed.

  4. haki203 commented on Jun 30, 2026

    @haki203

    Another data point from a user hitting this with a self-hosted OpenAI-compatible provider (@ai-sdk/openai-compatible) — see #34672 (closed as dup). The TimeoutError: The operation timed out. case reproduces reliably when the downstream model is overloaded. Config timeout/chunkTimeout only controls when the error fires; it does not make the error retryable. +1 to the widening fix. Happy to test a build if useful.

  5. literally-dan commented on Jul 22, 2026

    @literally-dan
    Author

    #30638 which was closed fixes this...

  6. zopenb commented on Aug 8, 2026

    @zopenb

    Another reproducible data point on OpenCode 1.18.15 with @ai-sdk/openai-compatible: a transient error delivered inside an already-established SSE stream terminates the turn without entering session retry.

    The provider emits an SSE error shaped like:

    {
      "error": {
        "message": "Upstream HTTP/2 stream failed",
        "type": "upstream_error",
        "code": "upstream_http2_stream_error"
      }
    }

    Observed OpenCode logs contain exactly one message=stream, followed by message="stream error", process error, and pruning. There is no retry status and no second message=stream. The upstream request is therefore attempted once, with zero retries.

    The apparent normalization path is:

    1. The OpenAI-compatible adapter forwards chunk.value.error as a plain object.
    2. MessageV2.fromError() calls ProviderError.parseStreamError().
    3. parseStreamError() requires body.type === "error", but the unwrapped object has type === "upstream_error", so parsing returns undefined.
    4. The object becomes UnknownError.
    5. Its serialized message does not match the current retry patterns (upstream connect, stream timeout, etc.), so SessionRetry.retryable() returns undefined.

    This is adjacent to, but not the same layer as, the transport/socket errors described in this issue. It appears to be the same class as #21893, which is currently closed. Could a maintainer confirm whether #21893 should be reopened, or whether a narrowly scoped issue/PR for structured SSE errors would be preferred?

    A safe fix should avoid blindly replaying a stream after text or tool calls have already been emitted. Retrying only before any assistant output/tool call, with a small attempt cap, would avoid duplicate side effects.

  7. JavaGT commented on Aug 21, 2026

    @JavaGT

    Adding production data from a machine hitting this daily — with one wrinkle that may matter for the fix: even the error class this issue says is retryable (ECONNRESET) fails the session outright when it arrives wrapped by the AI SDK.

    Environment

    • opencode version: 0.0.0-beta-17793
    • OS: macOS (Darwin arm64)
    • Terminal: tmux / xterm-256color
    • Shell: /bin/zsh
    • Install/channel: beta
    • Active plugins: none configured

    What we see

    Server log, single day, all against https://opencode.ai/zen/v1/chat/completions, all surfacing as Failed to drain Session and killing the turn:

    AI.Error: RequestExecutor.read: ECONNRESET: The socket connection was closed unexpectedly. …   (32×)
    AI.Error: RequestExecutor.read: Decode error (200 POST https://opencode.ai/zen/v1/chat/completions)   (116×)
    

    Session records show no retry attempts for any of these — the turn just ends. The ECONNRESET cases are notable because raw transport ECONNRESET is supposedly already in the retryable set; it's the RequestExecutor.read: wrapping (the same wrapping discussed in #42482) that appears to bypass classification. That matches the mechanism in #43791: errors that don't arrive as classifiable provider failures fall through to a terminal state.

    Request

    For sub-agent orchestration this is the single most expensive failure mode: a coordinator dispatches N concurrent sub-agents, one stream dies mid-turn, no terminal event reaches the parent, and the coordinator can't distinguish "dead" from "stalled" — so the whole task gets re-run from scratch or a human manually nudges the session back to life.

    Minimum viable behavior from our side:

    1. Unwrap the AI SDK error chain (cause / RequestExecutor.read wrapper) before classification so mid-stream transport failures hit the existing retry policy (ai: classify plain AI SDK stream errors #43791 looks like the right place).
    2. Cap the retries and surface a terminal state to clients when exhausted — currently failures are either silent-and-fatal (this issue) or silent-and-infinite (fix(session): SessionRetry.policy() retries forever with no max attempt count #21960), and both look identical from outside.
  8. RoySalisbury commented on Sep 15, 2026

    @RoySalisbury

    Additional deployment evidence for transient SSE failures terminating a turn without recovery.

    Versions and route

    • OpenCode 1.18.29 (confirmed from request User-Agent)
    • CLIProxyAPI v7.3.3, commit 7bbfeaf
    • OpenCode → /v1/chat/completions → CLIProxyAPI → Codex upstream

    Observed sequence

    1. Upstream returned HTTP 200.
    2. Upstream emitted response.created, response.in_progress, and many response.output_text.delta events. Partial text was forwarded to OpenCode.
    3. After approximately 25.3 seconds total request duration, the upstream stream ended without response.completed.
    4. CLIProxyAPI logged:
      stream error: stream disconnected before completion: stream closed before response.completed
      
    5. Downstream remained HTTP 200 and ended with this exact SSE payload:
      data: {"error":{"message":"stream error: stream disconnected before completion: stream closed before response.completed","type":"invalid_request_error"}}
      
    6. The user observed OpenCode aborting the turn rather than automatically recovering.

    Proxy configuration

    request-retry: 3
    max-retry-interval: 20
    streaming:
      keepalive-seconds: 15
    codex:
      stream-bootstrap-buffering: true

    This is a post-output failure, not a bootstrap-only failure. Increasing bootstrap retries alone cannot resolve this captured case.

    The proxy-side request/response log confirms the sequence and payload above. We have not yet captured OpenCode’s normalized internal error or a client-side retry trace, so the specific classification path and client attempt count remain unconfirmed. The underlying reason for the upstream disconnect is also not established.

    Would a regression test using this exact SSE error after several valid text chunks help? The desired behaviour is bounded recovery that preserves completed tool actions, rather than blindly replaying the entire turn. The proxy labels this disconnect invalid_request_error, which may be relevant to classification.

    Related upstream disconnect report: router-for-me/CLIProxyAPI#3055

    Full request logs are intentionally omitted because they contain conversation content and headers.

  9. RoySalisbury commented on Sep 15, 2026

    @RoySalisbury

    Cross-link to the proxy-side error-classification report for the reproduction above: router-for-me/CLIProxyAPI#3055 (comment). It asks whether this premature stream disconnect should carry a stable transient type/code instead of invalid_request_error; post-output recovery remains a separate client-side concern.

  10. RoySalisbury commented on Sep 15, 2026

    @RoySalisbury

    I'm also thinking that a plugin could be a temporary workaround for this. The plugin could look for specifc errors and retry the prompt with backoff.

  11. neriousy commented on Oct 1, 2026

    @neriousy
    Member

    In OpenCode v2, transient network failures are retried instead of ending the turn. Connection errors such as ECONNRESET, ECONNREFUSED, ETIMEDOUT, EPIPE, undici UND_ERR_* errors, "fetch failed" and "other side closed" are classified as transport errors and go through the normal retry policy.

    Fixed in #45840; the retry policy itself is from #49441.

    Closing as completed. If a network error still ends a session without retrying on v2, please open a new issue with the error message.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions