Repository navigation
gateway: a reply translated from a dying stream no longer reads complete - #78
Merged
Merged
Conversation
translate ignored readSSE's return, so an upstream that severed mid-reply left a buffered client holding the partial text under finish_reason "stop" and a streaming client with a synthetic stop chunk, final usage and [DONE]; the buffered path now answers 502 and the streamed path emits the client protocol's error event and skips the normal completion, covered by tests that hijack an httptest upstream and close it mid-stream plus a control that a properly finished stream still translates end to end
yetone
reviewed
Sep 25, 2026
yetone
left a comment
Owner
There was a problem hiding this comment.
Thanks — reviewed and verified locally.
go test ./internal/gateway -count=1passes; the three new tests fail on main (the severed stream ends withfinish_reason: stop+[DONE]) and pass with the change.- Checked the side effect of skipping
finish()oncefailedis set, which now also covers an upstream error event: every encoder already writes a terminal error onKError(Anthropicerror, Responsesresponse.failed, chat/Gemini error chunk), so previously a failed reply was followed bymessage_stop/response.completed/[DONE]— contradicting itself. Dropping that is a fix too. - A client that hangs up mid-stream also makes
readSSEreturn an error, but the caller already turns a canceled context into 499 "the agent canceled the request", so it isn't counted against the provider.
Merging. (The subscription path in serveSubscription still calls finish() after an error; I'll align it separately.)
yetone
added a commit
that referenced
this pull request
Sep 25, 2026
…de Code, Cursor, Grok and Devin runs still wrote message_stop / response.completed / [DONE] after the error event, so a client could take the half reply as complete; the same fix #78 made for translated streams
Owner
|
Released in v0.1.93, together with the matching fix for subscription runs (6da20f5). Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
Server.translatespeaks one provider API to the client while streaming another from the provider, reading the provider's SSE stream withreadSSE(internal/gateway/sse.go).readSSEreturns the stream's terminal error, but both call sites intranslateignore that return (internal/gateway/gateway.go).So when a provider's connection dies mid-reply — upstream reset, truncated body — the failure vanishes:
stream:false) gets HTTP 200 with whatever partial text had arrived, rendered withfinish_reason: "stop"— a fragment indistinguishable from a complete answer.[DONE]over a stream the provider never finished.Repro on current main (f298bb4): an httptest upstream that hijacks the connection, promises 100 more bytes than it sends, delivers one
message_startand one text delta, then closes. Buffered: 200 with"content":"hel"andfinish_reason:"stop". Streamed: afinish_reason:"stop"chunk, usage,[DONE], and no error anywhere.Fix
Check
readSSE's return in both branches:UP: unexpected EOF) instead of returning the partial answer. A partial answer is not an answer.{"error": ...}for Chat,response.failedfor Responses,event: errorfor Anthropic, an error chunk for Gemini, each encoder's existing KError rendering — and skips the normal completion instead of appending a synthetic one. A stream whose provider already signalled an error likewise no longer gets a completion event after it.Complete streams are untouched: a control test runs a full Anthropic stream through both branches and asserts the same 200, translated body and
[DONE]as before.Tests
TestTranslateBufferedSeveredUpstream— fails on main (200 with partial body), passes with the fix (502 naming the provider).TestTranslateStreamSeveredUpstream— fails on main (synthetic stop +[DONE]), passes with the fix (error event, no completion).TestTranslateStreamCompleteUpstream— control: a properly finished stream still translates end to end, streamed and buffered.go vet ./internal/gateway/clean;go test -race -count=1 ./internal/gateway/passes;go test ./internal/...passes exceptinternal/gui, which fails to compile in this Linux container for want of GTK4/WebKitGTK 6 development libraries — pre-existing, since the unmodified tree fails the same way, and CI installs GTK with its own tags.