Skip to content

openai-completions: intermittent mid-stream gateway truncation hard-fails as "Stream ended without finish_reason" despite partial content (OpenCode Go) #8460

Closed as not planned

Description

@Zaldaryon

Summary

OpenAI-compatible gateways that intermittently truncate streams mid-generation get zero tolerance from the openai-completions client, because detectCompat() defaults supportsFinishReason: true. When such a gateway closes the connection after emitting substantial partial content but before any finish_reason chunk or [DONE], pi throws a fatal Error: Stream ended without finish_reason and discards the partial response, even though the same request usually succeeds.

This is a different failure mode from #7062 (providers that never send finish_reason) and #6226 (missing finish_reason on tool calls): here the provider is normally compliant and the truncation is transient — but just as disruptive, because it surfaces randomly during long generations.

Environment

  • pi: 0.84.2 (@earendil-works/pi-ai 0.84.2)
  • Provider: built-in opencode-go (https://opencode.ai/zen/go/v1, api openai-completions)
  • Model: ox-alpha-free
  • OS: Windows 11

Reproduction

Direct SSE capture against the gateway endpoint (POST {baseUrl}/chat/completions, stream: true), identical long-generation prompt repeated:

Run Model Outcome
1–2 ox-alpha-free proper finish_reason:"stop" chunk + [DONE]
3 ox-alpha-free stream closed after ~partial output: no finish_reason, no [DONE]
1–2 deepseek-v4-flash (same tier) proper termination

Short requests succeed consistently; long generations fail intermittently (~30% in my sample), which is why it appears as random session errors. Inside pi this manifests as Error: Stream ended without finish_reason after visible partial output.

The raw tail of a successful stream also shows the gateway appends a non-protocol trailing frame after [DONE]:

data: {"id":"...","choices":[{"index":0,"finish_reason":"stop",...}]}
data: [DONE]
data: {"choices":[],"cost":"0"}

A truncated stream ends with nothing after the last content delta.

Root cause

packages/ai/src/api/openai-completions.ts (dist inspected at 0.84.2):

if (!hasFinishReason && !compat.supportsFinishReason) {
    output.stopReason = output.content.some(b => b.type === "toolCall") ? "toolUse" : "stop";
}
...
if ((compat.supportsFinishReason && !hasFinishReason) || output.stopReason === "pending") {
    throw new Error("Stream ended without finish_reason");
}

Since detectCompat() returns supportsFinishReason: true for any unknown OpenAI-compatible host, the graceful-degradation branch never applies to providers that are usually compliant but occasionally truncate. The distinction "stream ended cleanly via [DONE] without finish_reason" vs "connection dropped mid-stream" is not considered — both land in the same throw when supportsFinishReason is true.

Workaround (verified)

Per-model override so the tolerance branch engages for this provider:

// ~/.pi/agent/models.json
{
  "providers": {
    "opencode-go": {
      "modelOverrides": {
        "ox-alpha-free": {
          "compat": { "supportsFinishReason": false }
        }
      }
    }
  }
}

Trade-off: this also disables the strict check for genuinely malformed streams from that provider, and truncated tool-call JSON would be parsed as complete — acceptable here given partial tool calls were previously discarded anyway.

Possible directions

  1. Treat "stream closed without [DONE] and without finish_reason but with non-empty content" as a distinct outcome — e.g. stopReason: "stop" + warning, or a retryable error classification that preserves/resends context — instead of a bare throw.
  2. Alternatively, surface a compat.truncatedStreamPolicy knob so per-provider behavior doesn't require lying about supportsFinishReason.

Happy to test patches against the opencode-go tier. Related: #7062, #6226.

Activity

  1. github-actions commented on Aug 21, 2026

    @github-actions
    Contributor

    This issue was auto-closed. All issues from new contributors are auto-closed by default.

    Maintainers review auto-closed issues daily and reopen worthwhile ones. Issues that do not meet the quality bar in CONTRIBUTING.md will not be reopened or receive a reply.

    If a maintainer replies lgtmi on one of your issues, your future issues will stay open. If a maintainer replies lgtm, your future issues and PRs will stay open. The command must be at the start of the reply (optionally after one or more @username mentions) or at the end.

    See CONTRIBUTING.md.

  2. added
    untriagedThis issue has been auto closed and has not been triaged
    on Aug 21, 2026
  3. added a commit that references this issue on Aug 23, 2026
    a470b12
  4. added
    no-actionThis issue has been rejected after triage
    and removed
    untriagedThis issue has been auto closed and has not been triaged
    on Aug 25, 2026
  5. added a commit that references this issue on Sep 11, 2026
    fbefad4
  6. added a commit that references this issue on Oct 1, 2026
    e53ab50
  7. added a commit that references this issue on Oct 8, 2026
    6909341
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    no-actionThis issue has been rejected after triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions