Repository navigation
[Bug] GLM-5.1 streaming: truncated JSON in SSE chunks causes client-side parse failures #66
Description
Summary
GLM-5.1 (via z.ai API and vLLM deployments) intermittently produces truncated JSON in SSE streaming chunks, causing JSON Parse error: Expected '}' on the client side. This breaks streaming conversations in tools like opencode, killing the entire session.
Observed Behavior
During streaming inference, SSE chunks arrive with JSON objects that are truncated mid-field — the closing portion of the JSON object is missing, and the next SSE data: frame begins inline within the previous frame's content:
JSON parsing failed: Text: {"id":"20260430204055297b97348c8e426c","created":1777552855,"object":"chat.cdata: {"id":"2026043020420635c6abc56a2741ac",
"created":1777552926,"object":"chat.completion.chunk","model":"glm-5.1","choices":[{"index":0,"delta":{"role":"assistant","content":"11"}}]}
Error message: JSON Parse error: Expected '}'
Notice the first object ends at "object":"chat.c — this is the beginning of "chat.completion.chunk" that got cut off. Then the second SSE data: frame starts immediately, producing invalid JSON.
Two Distinct Failure Modes
Mode 1: Truncated SSE chunks (this report)
The SSE frame contains incomplete JSON — the server sends a chunk that is cut off before the closing }. The next chunk's data: prefix appears within the previous chunk's text.
Frequency: Intermittent, occurs during long multi-turn sessions with 50+ tool calls. Not every session, but frequent enough to be a reliability blocker.
Mode 2: Hallucinated SSE content in model output
Related but distinct: the model sometimes generates data: {"id":"...","choices":[...]} as literal text in its response content field. Z.AI's server doesn't escape the inner quotes, breaking the outer JSON structure.
Tracked in: anomalyco/opencode#23442
Environment
| Component | Version |
|---|---|
| Model | zai-coding-plan/glm-5.1 |
| Provider | Z.AI API (api.z.ai/api/coding/paas/v4) |
| Client | opencode 1.4.10 (darwin-arm64, homebrew) |
| Hardware | Apple M1 Air 8GB |
| Protocol | OpenAI-compatible SSE streaming |
| Session context | 50+ tool calls, ~100k tokens context |
Also reproduced by others via:
- vLLM local deployment (DGX-B300) with
tool_call_parser: glm47— https://huggingface.co/zai-org/GLM-5/discussions/59 - NVIDIA NIM deployment — [Bug] Malformed tool-call JSON via NVIDIA NIM (GLM-5) causes parser failures in OpenCode #15
Root Cause Analysis
The truncation pattern ("object":"chat.c followed immediately by data: {) suggests one of:
-
Server-side SSE framing bug: The z.ai API server (or vLLM with
glm47parser) generates SSEdata:frames that contain incomplete JSON objects — perhaps due to a buffer flush boundary that splits mid-JSON. -
Streaming serialization race: The JSON serializer writes partial output and flushes before the object is complete, producing a truncated frame.
-
Token-level chunking mismatch: The model generates tokens that cross a JSON structural boundary, and the server doesn't buffer until the boundary is complete before emitting the SSE frame.
The fact that this reproduces across multiple deployment backends (z.ai hosted, vLLM, NVIDIA NIM) suggests it may be model-level — the tokenizer or generation logic produces output that confuses the server-side SSE formatter.
Reproduction
- Configure any OpenAI-compatible client with GLM-5.1
- Engage in a multi-turn conversation with 50+ tool calls
- Observe intermittent JSON parse errors in SSE chunks
- The client-side stream dies, killing the conversation
Minimal curl reproduction (will not trigger every time due to intermittency):
curl -N https://api.z.ai/api/coding/paas/v4/chat/completions \
-H "Authorization: Bearer $ZAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "glm-5.1",
"stream": true,
"messages": [{"role": "user", "content": "Write a complex Python class with 20 methods"}]
}' 2>&1 | grep -c '}'
# Count should equal number of data: lines; mismatches indicate truncationExpected Behavior
Every SSE data: frame should contain a complete, parseable JSON object. The server should buffer partial JSON until a complete object is ready before emitting the frame.
Actual Behavior
SSE frames contain truncated JSON objects, with the next frame's prefix appearing inline.
Impact
- Session death: The entire conversation dies when a malformed chunk hits the JSON parser
- No recovery: Current clients (opencode, Vercel AI SDK) treat parse errors as fatal stream errors
- Reliability blocker: Cannot use GLM-5.1 for long autonomous sessions where reliability matters
Suggested Server-Side Fix
Buffer the JSON output token-by-token and only emit an SSE data: frame when the JSON object is complete (balanced braces). This ensures every frame is parseable:
# Pseudocode for SSE emission
buffer = ""
brace_depth = 0
for token in model_stream():
buffer += token
brace_depth += token.count('{') - token.count('}')
if brace_depth == 0 and buffer.strip():
yield f"data: {buffer}\n\n"
buffer = ""Suggested Client-Side Mitigation (for opencode/others)
Skip malformed chunks and continue the stream instead of dying:
// In parseJsonEventStream TransformStream:
try {
controller.enqueue(await safeParseJSON({ text: data, schema }));
} catch (err) {
console.warn("SSE chunk JSON parse failed, skipping:", data.slice(0, 200));
// Continue to next chunk — real content is in adjacent valid frames
}Related Issues
- SSE JSON parse failure with GLM-5.1 (Z.AI) — malformed JSON in content field kills stream anomalyco/opencode#23442 — Same failure mode, opencode-side tracking
- https://huggingface.co/zai-org/GLM-5/discussions/59 — vLLM deployment, tool calling JSON parse
- [Bug] Malformed tool-call JSON via NVIDIA NIM (GLM-5) causes parser failures in OpenCode #15 — NVIDIA NIM, malformed tool-call JSON
- GLM 4.7 AI_JSONParseError anomalyco/opencode#8431 — GLM 4.7 AI_JSONParseError
- AI_JSONParseError during SSE streaming is not retried (falls to NamedError.Unknown) anomalyco/opencode#13579 — AI_JSONParseError not retried
Why This Matters
We run GLM-5.1 as a coding agent inside the MultiAgency framework — an autonomous system where reliability is critical. A single truncated SSE chunk kills an entire session that may have hours of accumulated context. We want to use GLM-5.1 as a first-class model in our rotation, but this issue makes it unreliable for long autonomous sessions.
We're happy to help test fixes or provide more detailed logs from our sessions.
Activity
JaredforReal commentedon May 6, 2026
@darkhipo Thanks for your feedback! But I didn't success to repro the issue. Here is the script I used test_sse_issue.py
Correct me if I did not get things right, and glad to have more information from ur side
BTW, vllm-project/vllm#39253, we have fixed a issue related to streaming tool call parser in vllm like a month ago, just for your information
wong-hao commentedon May 7, 2026
I have the same problem with GLM5.1 when tool calling. NousResearch/hermes-agent#13042
darkhipo commentedon May 9, 2026
Thanks for the test script, @JaredforReal — it's well-structured. A few notes on why it likely didn't reproduce:
The bug requires sustained multi-turn tool-calling sessions. Your script does single-turn completions in a loop, but each request starts fresh. The truncation we see only triggers after 50+ accumulated tool calls in a single streaming session where context exceeds ~100k tokens. The pattern is:
"object":"chat.cfollowed immediately bydata: {— i.e., the server splits a JSON object mid-field across two SSE framesYour test script sends individual requests — it won't hit the accumulated-context degradation path. To reproduce, you'd need a single persistent connection with the full conversation history growing across turns (like a coding agent loop does).
However, the vllm PR you linked (vllm-project/vllm#39253) may have fixed the server-side root cause for self-hosted deployments. That PR rewrote the GLM tool parser from a buffer-based state machine to a stateless re-parse approach — exactly the kind of fix that would prevent mid-JSON frame splits. We'll test against a recent vllm build to confirm.
For the z.ai hosted API specifically, the issue still reproduces as of this week. Since we can't control the server-side build there, the question is whether z.ai's API backend includes the fix from that vllm PR.
@wong-hao — thanks for confirming. Your hermes-agent issue (NousResearch/hermes-agent#13042) describes the same degradation pattern: "works fine for short-context tool calls, malforms in long contexts." This matches our experience exactly.
We've added client-side mitigation on our end (skip malformed chunks, continue the stream), which reduces session deaths but doesn't fix the root cause. The real fix needs to be server-side — either the z.ai API backend picking up the vllm parser fix, or ensuring the SSE framing layer buffers complete JSON objects before emitting frames.
darkhipo commentedon May 9, 2026
New Reproduction Data — 2026-05-10
Hit the same truncation again today during an active coding session. This time the truncation point is slightly different — cutting the
idfield instead of theobjectfield.Raw Error
Breakdown
The first SSE frame is truncated at the
idfield:{"id":"2026051004Then
data:appears inline, followed by a complete second frame:{"id":"202605100452505383be940b9c458d","created":1778359970,"object":"chat.completion.chunk","model":"glm-5.1","choices":[{"index":0,"delta":{"role":"assistant","content":"Now"}}]}Pattern Analysis
Comparing across our three captured incidents:
"object":"chat.c"chat.completion.chunk""id":"2026051004The truncated
idvalue"2026051004"is exactly the first 12 characters of the second frame's complete id"202605100452505383be940b9c458d". This is consistent with a fixed buffer size (possibly 12 bytes?) that flushes mid-field when the JSON object exceeds the buffer boundary.Session Context
zai-coding-plan/glm-5.1Key Observation: Not Limited to Long Contexts
This incident occurred at ~60-80k tokens context, which is lower than our previous incidents at 100k+. This suggests the truncation is not purely a function of context length — it may be related to response generation timing, server load, or buffer management that can trigger at any session length.
Suggested Server-Side Debugging
The consistent truncation at fixed character boundaries (12 chars of the
idfield, 8 chars of"chat.c) suggests a buffer flush threshold. If the server uses a fixed-size output buffer for SSE framing, look for:idfield that's generated from a timestamp and may have variable length (longer IDs = more likely to cross the buffer boundary)If you can share any details about the SSE emission layer in the z.ai API server, we can provide more targeted analysis.
Diego404YT commentedon May 16, 2026
Confirming this on the chat.z.ai web frontend
I am experiencing the exact same truncation behavior when using the web interface at chat.z.ai.
Symptoms:
Stream suddenly cuts off mid-response.
Or the entire response "pops" in all at once instead of streaming token-by-token.
The thinking process is often interrupted or only displayed at the very end.
I ran a network debugger on the frontend and captured the SSE chunks. The backend streams the thinking phase correctly at first, but then a malformed/truncated chunk kills the stream parsing on the client side, causing the UI to crash the generation or dump the buffer all at once.
This confirms the bug isn't just limited to OpenCode/vLLM; it's breaking the official hosted web chat as well. Any server-side fix for the SSE framing will fix the web UI too.
fix(completions): emit valid JSON SSE error frame on stream failures (#…