Skip to content

provider: openai-compatible stream aborts the whole run on non-object SSE payloads (data: null) #50314

Closed

Description

@moeskia

Summary

V2's openai-compatible-chat stream decoder aborts the entire run when an OpenAI-compatible endpoint emits a syntactically valid but non-object SSE payload such as data: null. A gateway that injects data: null mid-stream makes every tool-calling step fail with Invalid <provider>/openai-compatible-chat stream event. Upstream AI SDK SSE parsing tolerates unknown payloads; V2 hard-fails and loses the whole step.

Environment

  • opencode version: 2.0.11
  • OS: Windows_NT 10.0.26200 (win32 x64)
  • Terminal: xterm-256color (TERM_PROGRAM unset)
  • Shell: C:\WINDOWS\system32\cmd.exe (pwsh configured via shell in config)
  • Install/channel: global npm install of @opencode/cli 2.0.11 (latest)
  • Active plugins: file:///C:/Users/<user>/antigravity-auth/packages/opencode-v2 (local), plus a local http.response hook plugin

Reproduction

  1. Configure a custom provider using @opencode/ai/providers/openai-compatible whose endpoint is an OpenAI-compatible gateway that emits data: null frames in the middle of a Chat Completions stream (verify with curl -N first; the gateway requires a specific User-Agent).
  2. Start a session on that provider and send a prompt that makes the model call a tool.
  3. The step aborts at the first tool call.

Raw upstream SSE (curl, "stream": true, with a tools array) contains, mid-stream:

data: {"choices":[{"index":0,"delta":{"reasoning_content":"\n"},"logprobs":null,"finish_reason":null,"index":0}],"usage":null}

data: null

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_...","type":"function","function":{"name":"...","arguments":""}}]},"logprobs":null,"finish_reason":null,"index":0}],"usage":null}

Response headers: Content-Type: text/event-stream, Transfer-Encoding: chunked.
On the same endpoint, a plain text stream (no tools) contains zero non-object frames and succeeds. Tool-call streams emit exactly two data: null frames (before the first tool_calls delta and before the finish_reason delta).

Expected Behavior

A data: payload that is not a JSON object should be ignored (like [DONE]), or surfaced as a recoverable provider/parse error. The step should not be discarded.

Actual Behavior

AI.Error.InvalidProviderOutput: Invalid <provider>/openai-compatible-chat stream event
  [cause]: SchemaError: Expected object

Server log (~/.local/share/opencode/log/opencode.log):

level=ERROR message="Failed to drain Session" cause="AI.Error: Invalid <provider>/openai-compatible-chat stream event ...
  [cause]: AI.Error.InvalidProviderOutput: Invalid <provider>/openai-compatible-chat stream event {
    [cause]: SchemaError: Expected object

The event schema for the route appears to be Union(Literal("[DONE]"), Struct({ choices, usage, error })). null is valid JSON but matches neither member, so decoding fails and the run is aborted. The error message also does not include the offending payload, which makes this hard to diagnose from logs.

Additional Context

  • Frequency: consistently reproducible on that gateway as soon as the model uses a tool; ~3 consecutive failures within minutes before the workaround.
  • Workaround (verified): a local plugin registering ctx.session.hook("http.response", handler, { providerID }) that pipes a text/event-stream body through a TransformStream dropping data: null (and empty data:) lines. After that, multi-step tool-calling sessions complete normally.
  • Suggestion: treat undecodable stream events as ignorable (log at debug level) instead of failing the run, matching the tolerant behavior of the AI SDK's SSE decoder.
  • The endpoint is non-standard, but the failure mode — the entire run dying with an error that hides the offending frame — is worth hardening.

Activity

  1. argszero commented on Sep 21, 2026

    @argszero
    Contributor

    I'd like to work on this.

    Confirmed on dev (70a24697ea): the SSE framing keeps only "not empty and not [DONE]" (packages/llm/src/protocols/shared.ts:247), so a data: null frame reaches the protocol decoder verbatim, and Protocol.jsonEvent (packages/llm/src/route/protocol.ts:82, Schema.fromJsonString(...)) then fails the struct decode — packages/llm/src/route/client.ts:232-241 turns that into a hard Invalid <provider>/<route> stream event and the whole step dies. Same path is shared by openai-chat, openai-compatible-chat, openai-responses, anthropic-messages and gemini, so the gateway behavior breaks every SSE route, not just the compatible one.

    Plan: treat a data: payload that is not a JSON object the same way the framing layer already treats [DONE] — drop it as a non-event — while keeping the existing hard error for payloads that are objects but do not match the event schema (that distinction is what preserves real diagnostics). Plus a unit test next to packages/llm/test/provider/openai-compatible-chat.test.ts feeding a stream with data: null between two valid deltas.

    I'll open a PR against dev once it passes the package suite.

  2. argszero commented on Sep 22, 2026

    @argszero
    Contributor

    Opened #50508 for this: the framing now drops a data: payload that is valid JSON but not a plain object, right beside the existing empty-frame and [DONE] drops. Invalid JSON and object payloads that fail the event schema still reach the decoder, so the Invalid <provider>/<route> stream event diagnostic that names the offending frame is preserved.

    Covered by two cases in packages/llm/test/provider/openai-compatible-chat.test.ts (a data: null keep-alive interleaved between real deltas, and the negative case of an object frame that is not a stream event).

  3. rekram1-node commented on Sep 23, 2026

    @rekram1-node
    Collaborator

    @argszero your pr wasnt for the correct branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions