Repository navigation
Non-streaming /v1/chat/completions requests receive an SSE-framed body; client parse fails and usage is lost #4733
Description
Problem
When a client omits the stream parameter (OpenAI's documented default is false), 9router returns a response body that mixes a valid JSON object with SSE framing:
{"id":"...","object":"chat.completion",...,"lastOne":false}data: [DONE]
A conforming client calling .json() on this body throws:
Extra data: line 1 column N (char N)
This is a spec violation. OpenAI's API returns a single JSON object when stream is omitted or explicitly false; the SSE data: [DONE] terminator belongs only in streaming responses.
Reproduction
Same model, same prompt, same provider connection — only the stream parameter differs:
Omitted (should default to non-streaming):
curl -X POST http://localhost:30128/v1/chat/completions -H "Authorization: Bearer ***" -d '{"model":"combo-free-9r-ctx-256k","messages":[...],"max_tokens":300}'
→ HTTP 200, 1502 bytes, `{"...","lastOne":false}data: [DONE]
`
→ JSON parse fails at char 1488
Explicit stream:false:
curl -X POST http://localhost:30128/v1/chat/completions -H "Authorization: Bearer ***" -d '{"model":"combo-free-9r-ctx-256k","messages":[...],"max_tokens":300,"stream":false}'
→ HTTP 200, 1228 bytes, {"...","lastOne":false}
→ JSON parses clean
The difference is exactly the trailing `data: [DONE]
— an upstream SSE frame field (lastOne` is from the provider's streaming format).
Consequences beyond the parse error
Affected calls log IN 0 · OUT 0 in the journal and never reach usageHistory. The usage record is lost — not just the client broken, but the metering layer blind to that call. Measured: 19 calls in the window from martin-loop's governed run (two attempts) all logged IN 0 · OUT 0; zero rows added to usageHistory.
Suspected locus
src/sse/handlers/chat.js:317 — if (result.success) return result.response; passes the upstream response through verbatim. The handler does not normalise non-streaming responses to strip SSE framing.
Proposed fix
Force non-streaming upstream when the client omits stream (or normalise away SSE framing before returning). Either:
- Inject
stream: falseinto the upstream request when the client omitsstream, or - Detect an SSE-framed body on success and return only the first JSON object
Both are small, self-contained changes in the fork.
Our context
We run two 9router instances (WSL + VPS) behind a single OpenAI-compatible endpoint. The bug affects any client that omits stream — martin-loop's openai-compatible adapter was the first to expose it, but it is a general spec violation. Fixing it in our fork closes the gap for all downstream clients.
Activity
fix: thinking by model list, account failover, cut streams, missing c…
kuehnberger commentedon Oct 17, 2026
Confirmation — still present, and the impact is wider than a parse error
Measured against our two-instance fleet (WSL + VPS, OpenAI-compatible endpoint):
streamreturns a body that mixes a valid JSON object with SSE framing:{"id":"...","object":"chat.completion",...,"lastOne":false}data: [DONE]\n\nA conforming client calling
.json()throwsExtra data: line 1 column N.stream: falsereturns a single clean JSON object. Same model, same prompt,only the parameter differs — so the default is the defect, not the provider.
IN 0 · OUT 0and never reachusageHistory. The meteringlayer goes blind to those calls, not merely the client.
Live downstream victim: a local mutation-capable fleet watcher probes combos every
30 s,
json.loads()the response, classifies the parse failure as a model death, and on3 consecutive failures calls
PUT /api/combos/<id>to remove a member. It reported0 healthy, 4 unhealthy; 3 of those 4 were this false signal, and only its 2-membercircuit breaker stopped it from stripping healthy members.
Fixed by #4753 and #4427 — thank you both. We tested #4753 end to end and endorse it as
the canonical fix (detail in our review comment below).
@decolua — please consider adopting #4753 as the canonical solution rather than
#4427. Rationale and an offer to help are in that comment.
kuehnberger commentedon Oct 22, 2026
Adding the code-level mechanism we traced, plus a correction: the suspected locus in the original report (
src/sse/handlers/chat.js:317) is not where the framing is decided — that line only returns the already-framed response.Root cause (traced on a self-hosted instance, fork at 0.5.91; re-verified against current
master, v0.5.104 @87a74e1b):The streaming decision is computed for the response pipeline, but is never written into the request sent upstream.
open-sse/handlers/chatCore.js:144—let stream = providerRequiresStreaming ? true : (body.stream !== false);→ an omittedstreambecomestrue.chatCore.js:215callstranslateRequest(sourceFormat, targetFormat, …, body, stream, …)with the decision — butopen-sse/translator/index.js:101short-circuits onif (sourceFormat !== targetFormat). For a same-format request (openai → openai, i.e. every OpenAI-compatible provider) no translator runs and the body is returned unchanged (translator/index.js:70;filterToOpenAIFormat()never touchesstreameither).open-sse/executors/base.js:79-80—transformRequest()returns the body untouched; the executor's only use ofstreamis theAcceptheader (base.js:71-73,executors/default.js:253).So with
streamomitted, the upstream is called withAccept: text/event-streamand nostreamfield. The upstream then answersapplication/json(measured:stream:true→text/event-stream,stream:false→application/json, absent →application/json), while the router is in STREAM mode →handleStreamingResponse()(chatCore.js:531). The passthrough stream forwards the raw JSON line verbatim (open-sse/utils/stream.js:256-259) andflush()appendsdata: [DONE]\n\n(stream.js:448-452) — hence the 14-byte trailer glued onto a valid JSON body.This also explains the metering half of the report: the hybrid is the path where the router frames the reply as SSE while the upstream body is a plain JSON object, so no usage trailer is ever seen. Measured on our instance: the hybrid call logs
IN 0 · OUT 0and writes nousageHistoryrow at all, while astream:falsecontrol call on the same model writes one (promptTokens 12 / completionTokens 2 / status ok). Any fix that normalises the response to JSON — or that makes the decision agree with the body — restores the usage row as a side effect.On the fix: #4753 remains the canonical candidate as we said — it changes the default decision to
clientRequestedStreamingand normalises an upstream SSE body viahandleForcedSSEToJson. For completeness, #4427 is the same one-line decision change, and #3878 implements it asclientRequestedStreaming()in a newopen-sse/handlers/chatCore/streamMode.jswith a dedicated unit test (tests/unit/stream-mode-default-3492.test.js) that also cites #3492 — that one closes both open framing issues.One invariant worth keeping once the default is fixed, so the mismatch cannot re-open from another direction: for a same-format request, write the decision into the body before dispatch (
translatedBody.stream = streamfor OpenAI-format targets). Today the body keeps whatever the client sent, so any future mode that is not literallybody.stream === true— e.g. one driven byAcceptorforceStream— re-opens the gap.kuehnberger commentedon Nov 3, 2026
For visibility: the canonical-fix reconciliation for #4733 is now posted on #5384, with a side-by-side of all five open competitors (#5384, #5320, #4753, #4427, #3878) and an explicit request for the maintainer to pick one. #5384 (comment)