Skip to content

Commit 5422d5a

Browse files
konardclaude
andcommitted
docs: redo case study analysis, file upstream issues #169
Corrected analysis: provider is OpenCode Zen via Kilo AI Gateway (not OpenRouter as previously stated). Added comprehensive comparison of 4 CLI agents (Codex, Gemini, Qwen, OpenCode) showing how each handles SSE stream parse errors. Key findings: - OpenAI Codex skips bad SSE events and continues (best practice) - Vercel AI SDK AI_JSONParseError has no isRetryable property - OpenCode upstream has the same gap (filed anomalyco/opencode#13579) - Kilo Gateway corrupts SSE chunks when proxying Kimi K2.5 Filed upstream issues: - vercel/ai#12595: AI_JSONParseError should support retry - Kilo-Org/kilocode#5875: SSE stream corruption with Kimi K2.5 - anomalyco/opencode#13579: AI_JSONParseError not retried Co-Authored-By: Claude Opus 4.6 <[email protected]>
1 parent ccaa5b7 commit 5422d5a

11 files changed

Lines changed: 451 additions & 208 deletions

‎docs/case-studies/issue-169/analysis.md‎

Lines changed: 178 additions & 110 deletions
Large diffs are not rendered by default.
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# CLI Agent Comparison: SSE Stream Parse Error Handling
2+
3+
Analysis of how major CLI agents handle SSE stream parse errors (like `AI_JSONParseError`),
4+
conducted as part of the investigation for [issue #169](https://github.com/link-assistant/agent/issues/169).
5+
6+
## Summary Table
7+
8+
| Feature | OpenAI Codex | Gemini CLI | Qwen Code | OpenCode | This Agent |
9+
|---------|-------------|------------|-----------|----------|------------|
10+
| **Language** | Rust | TypeScript | TypeScript | TypeScript | TypeScript |
11+
| **SSE Parser** | `eventsource_stream` crate | Custom (`readline`) | Delegates to SDK | `eventsource-parser` (via AI SDK) | `eventsource-parser` (via AI SDK) |
12+
| **JSON parse error in SSE** | Skip & continue | Not caught (gap) | Silent skip + warn log | Falls to UnknownError (gap) | StreamParseError + retry (fix) |
13+
| **Stream-level retry** | Yes (5 retries, exp backoff) | Yes (2 attempts for InvalidStreamError) | Yes (1 retry for InvalidStreamError, 10 for rate limits) | No | Yes (3 retries, exp backoff) |
14+
| **Connection retry** | Yes (4 retries, exp backoff) | Yes (3 retries, exp backoff + jitter) | Yes (7 retries, exp backoff + jitter) | Depends on AI SDK `maxRetries` | Depends on AI SDK `maxRetries` |
15+
| **Transport fallback** | WebSocket → HTTP fallback | No | No | No | No |
16+
| **Idle timeout** | 5 min (configurable) | No | No | No | No |
17+
| **AI SDK maxRetries** | N/A (own HTTP client) | N/A (own HTTP client) | N/A (own HTTP client) | 0 (disabled) | Default (2) |
18+
19+
## Detailed Analysis
20+
21+
### OpenAI Codex CLI — Best Practice
22+
23+
**Approach**: Skip-and-continue for individual bad events; stream-level retry for stream termination.
24+
25+
**Key design decisions**:
26+
1. Individual SSE events that fail JSON parsing are **skipped** (logged at debug level, `continue;`)
27+
2. SSE framing errors (protocol-level) terminate the stream → triggers stream retry
28+
3. Stream idle timeout (5 min default) → triggers stream retry
29+
4. Stream closed before `response.completed` → triggers stream retry
30+
5. `CodexErr::Json` (serde JSON errors) explicitly classified as **retryable**
31+
6. WebSocket-to-HTTP fallback when WebSocket retries exhausted
32+
7. Configurable: `stream_max_retries: 5`, `request_max_retries: 4`
33+
34+
**Files**: `codex-api/src/sse/responses.rs`, `codex-rs/core/src/codex.rs`, `codex-rs/core/src/error.rs`
35+
36+
### Gemini CLI
37+
38+
**Approach**: Two-layer retry architecture (connection + stream consumption).
39+
40+
**Key design decisions**:
41+
1. Layer 1: `retryWithBackoff` wraps HTTP request (3 attempts, 5s initial, 30s max)
42+
2. Layer 2: Inner loop retries `InvalidStreamError` (no finish reason, empty response, malformed function call) — 2 attempts, 500ms delay
43+
3. `SyntaxError` from custom SSE parser JSON.parse is **NOT caught** (gap)
44+
4. Rate limit handling: Classifies 429 as terminal vs retryable based on quota type
45+
5. Model fallback on persistent quota errors (Google auth only)
46+
6. Retry event signals UI to discard partial content
47+
48+
**Files**: `packages/core/src/utils/retry.ts`, `packages/core/src/core/geminiChat.ts`
49+
50+
### Qwen Code
51+
52+
**Approach**: Graceful skip in SDK; stream retry for content errors and rate limits.
53+
54+
**Key design decisions**:
55+
1. SDK `parseJsonLineSafe()` silently returns `null` on parse failure (logs warning)
56+
2. `StreamContentError` for rate limits (429/503/1302) retried with 60s delay, up to 10 times
57+
3. `InvalidStreamError` (no finish reason, empty response) retried once with 500ms delay
58+
4. Rate limit retries don't count against content retry limit
59+
5. Credential refresh on 401/403 (Qwen-specific)
60+
6. `error_finish` in SSE chunk data detected and thrown as `StreamContentError`
61+
62+
**Files**: `packages/sdk-typescript/src/utils/jsonLines.ts`, `packages/core/src/core/geminiChat.ts`
63+
64+
### OpenCode (sst/opencode) — This Project's Upstream
65+
66+
**Approach**: AI SDK error classification; only API errors with `isRetryable` are retried.
67+
68+
**Key findings**:
69+
1. `AI_JSONParseError` falls through to `NamedError.Unknown` — **NOT retried**
70+
2. `ECONNRESET` explicitly classified as retryable `APIError`
71+
3. Raw SSE error objects (non-Error instances) parsed via `parseStreamError()` — all `isRetryable: false`
72+
4. The `retryable()` function tries to JSON.parse the error message — returns any parseable JSON as retryable
73+
5. AI SDK `maxRetries: 0` (disabled) — no built-in retries
74+
6. Custom SDK SSE client has its own exponential backoff for connection failures
75+
76+
**Files**: `packages/opencode/src/session/message-v2.ts`, `packages/opencode/src/session/retry.ts`
77+
78+
## Recommendations for This Agent
79+
80+
Based on the comparison:
81+
82+
1. **Short-term** (implemented): Classify `AI_JSONParseError` as `StreamParseError` (retryable) in `message-v2.ts`
83+
2. **Medium-term**: Consider the Codex "skip-and-continue" approach — this is more resilient than retrying the entire stream
84+
3. **Long-term**: File upstream issues for Vercel AI SDK and Kilo Gateway
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Filed Issues for Case Study #169
2+
3+
Issues filed as part of the investigation into premature session termination
4+
due to SSE stream corruption and unhandled `AI_JSONParseError`.
5+
6+
## Filed Issues
7+
8+
1. **Vercel AI SDK** — [vercel/ai#12595](https://github.com/vercel/ai/issues/12595)
9+
- Title: AI_JSONParseError should support retry for mid-stream parse errors
10+
- Requests `isRetryable` property, `onStreamParseError` callback, or stream-level retry config
11+
12+
2. **Kilo AI Gateway** — [Kilo-Org/kilocode#5875](https://github.com/Kilo-Org/kilocode/issues/5875)
13+
- Title: SSE stream corruption when proxying Kimi K2.5 — chunks concatenated without delimiters
14+
- Reports the specific SSE violation observed in production logs
15+
16+
3. **OpenCode (upstream)** — [anomalyco/opencode#13579](https://github.com/anomalyco/opencode/issues/13579)
17+
- Title: AI_JSONParseError during SSE streaming is not retried (falls to NamedError.Unknown)
18+
- Reports the same bug in the upstream project with proposed fix
19+
20+
## Related Existing Issues
21+
22+
- [Kilo-Org/kilocode#5433](https://github.com/Kilo-Org/kilocode/issues/5433): Kimi K2.5 - Fails with Kilo Gateway
23+
- [anomalyco/opencode#7692](https://github.com/anomalyco/opencode/issues/7692): JSON Parse Error with GLM-4.7 stream chunks concatenated
24+
- [anomalyco/opencode#8431](https://github.com/anomalyco/opencode/issues/8431): GLM 4.7 AI_JSONParseError
25+
- [vercel/ai#4099](https://github.com/vercel/ai/issues/4099): streamText error handling
26+
- [vercel/ai#8577](https://github.com/vercel/ai/issues/8577): AI_JSONParseError with generateText
27+
- [sglang#8613](https://github.com/sgl-project/sglang/issues/8613): Kimi-K2 incomplete content during streaming
Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Bug Report: SSE stream corruption when proxying Kimi K2.5 — chunks concatenated without delimiters
2+
3+
## Description
4+
5+
When streaming responses from `moonshot/kimi-k2.5:free` via the Kilo AI Gateway (`api.kilo.ai`), we occasionally receive malformed SSE data where two SSE events are concatenated without proper `\n\ndata: ` delimiters. This causes downstream JSON parsing to fail.
6+
7+
## Evidence
8+
9+
The raw SSE data received by the client (captured via Vercel AI SDK error):
10+
11+
```json
12+
{
13+
"name": "AI_JSONParseError",
14+
"text": "{\"id\":\"chatcmpl-jQugNdata:{\"id\":\"chatcmpl-iU6vkr3fItZ0Y4rTCmIyAnXO\",\"object\":\"chat.completion.chunk\",\"created\":1771058051,\"model\":\"kimi-k2.5\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"finish_reason\":null}],\"system_fingerprint\":\"fpv0_f7e5c49a\"}"
15+
}
16+
```
17+
18+
### Analysis of corruption
19+
20+
The data shows two SSE events merged into one:
21+
22+
| Part | Content | What it should be |
23+
|------|---------|-------------------|
24+
| Chunk 1 (truncated) | `{"id":"chatcmpl-jQugN` | Complete first SSE event JSON |
25+
| SSE prefix (embedded) | `data:` | Should be on a new line after `\n\n` |
26+
| Chunk 2 (complete) | `{"id":"chatcmpl-iU6vk...}` | Complete second SSE event JSON |
27+
28+
### Expected SSE format
29+
30+
Per the [SSE specification](https://html.spec.whatwg.org/multipage/server-sent-events.html), each event should be separated by a blank line (`\n\n`):
31+
32+
```
33+
data: {"id":"chatcmpl-jQugN","object":"chat.completion.chunk",...}\n
34+
\n
35+
data: {"id":"chatcmpl-iU6vkr3fItZ0Y4rTCmIyAnXO","object":"chat.completion.chunk",...}\n
36+
\n
37+
```
38+
39+
## Environment
40+
41+
- **Gateway**: Kilo AI Gateway (`https://api.kilo.ai/api/gateway`)
42+
- **Model**: `moonshot/kimi-k2.5:free`
43+
- **Client SDK**: Vercel AI SDK (`@ai-sdk/openai-compatible`)
44+
- **SSE Parser**: `eventsource-parser` (used by AI SDK)
45+
- **Runtime**: Bun 1.3.x
46+
- **API Key**: `public` (free tier)
47+
- **Timestamp**: 2026-02-14T08:34:12Z UTC
48+
49+
## Impact
50+
51+
This causes `AI_JSONParseError` which terminates stream processing for the consumer. Since this is a transient infrastructure issue (not a model capability issue), it should not fail the user's request.
52+
53+
## Possibly Related
54+
55+
- [Kilo-Org/kilocode#5433](https://github.com/Kilo-Org/kilocode/issues/5433): Kimi K2.5 - Fails with Kilo Gateway
56+
- [sglang#8613](https://github.com/sgl-project/sglang/issues/8613): Kimi-K2 model outputs incomplete content during multi-turn streaming
57+
- [anomalyco/opencode#7692](https://github.com/anomalyco/opencode/issues/7692): Similar SSE concatenation issue with GLM-4.7
58+
- [anomalyco/opencode#10967](https://github.com/anomalyco/opencode/issues/10967): Error Writing Large Files with Kimi K2.5
59+
60+
## Workaround
61+
62+
We classify `AI_JSONParseError` as retryable in our client and retry the entire stream with exponential backoff (1s, 2s, 4s, up to 3 retries).
63+
64+
See: https://github.com/link-assistant/agent/issues/169

‎docs/case-studies/issue-169/openrouter-issue-draft.md‎

Lines changed: 0 additions & 50 deletions
This file was deleted.

‎docs/case-studies/issue-169/vercel-ai-issue-draft.md‎

Lines changed: 59 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,87 +1,109 @@
1-
# Feature Request: Make AI_JSONParseError Retryable for Streaming
1+
# Feature Request: `AI_JSONParseError` should support retry for mid-stream parse errors
22

33
## Description
44

5-
When using `streamText` with providers that return malformed SSE responses (e.g., OpenRouter proxying to Kimi K2.5), the AI SDK throws `AI_JSONParseError` which is marked as `isRetryable: false`. However, these errors are typically transient and caused by:
5+
When using `streamText` with OpenAI-compatible providers (via `@ai-sdk/openai-compatible`), the AI SDK may throw `AI_JSONParseError` during stream consumption when a provider or gateway returns malformed SSE data. Currently, this error:
66

7-
1. SSE chunks being concatenated incorrectly at proxy level
8-
2. Network issues corrupting stream data
9-
3. Provider returning invalid JSON temporarily
7+
1. Has **no `isRetryable` property** (only `APICallError` has it)
8+
2. Is **never retried** by the SDK's built-in `retryWithExponentialBackoff` mechanism
9+
3. Occurs **after** the HTTP request succeeds (HTTP 200), so the request-level retry is already bypassed
1010

11-
These errors should be retryable because they're transient infrastructure issues, not permanent failures.
11+
These errors are typically **transient** — caused by:
12+
- SSE chunks being concatenated incorrectly at proxy/gateway level
13+
- Network issues corrupting stream data mid-flight
14+
- Provider temporarily returning invalid JSON in stream
1215

1316
## Evidence from Production
1417

15-
We observed this error pattern in production logs:
18+
We observed this error when using Kimi K2.5 via the Kilo AI Gateway (`api.kilo.ai`):
1619

1720
```json
1821
{
1922
"name": "AI_JSONParseError",
2023
"cause": {},
21-
"text": "{\"id\":\"chatcmpl-jQugNdata:{\"id\":\"chatcmpl-iU6vkr3fItZ0Y4rTCmIyAnXO\",...}"
24+
"text": "{\"id\":\"chatcmpl-jQugNdata:{\"id\":\"chatcmpl-iU6vkr3fItZ0Y4rTCmIyAnXO\",\"object\":\"chat.completion.chunk\",...}"
2225
}
2326
```
2427

25-
Notice the corrupted data: `"chatcmpl-jQugNdata:{"` - two SSE chunks concatenated together without proper parsing.
28+
The `data:` SSE prefix of the second event was embedded inside the first event's JSON, indicating SSE chunking corruption at the gateway level.
2629

27-
The error currently has `isRetryable: false`, which prevents retry logic from working:
30+
This is a known class of issue — similar reports:
31+
- [ollama/ollama#5417](https://github.com/ollama/ollama/issues/5417): Cloudflare Tunnel + Vercel AI SDK = AI_JSONParseError
32+
- [anomalyco/opencode#7692](https://github.com/anomalyco/opencode/issues/7692): Stream chunks concatenated incorrectly with GLM-4.7
33+
- [vercel/ai#4099](https://github.com/vercel/ai/issues/4099): streamText error handling
2834

29-
```json
30-
{
31-
"isRetryable": false,
32-
"name": "AI_APICallError",
33-
...
35+
## Current Behavior
36+
37+
The AI SDK's parsing pipeline (`parseJsonEventStream`) uses `safeParseJSON` which returns `{ success: false, error: JSONParseError }` on parse failure. The OpenAI-compatible model handler then emits this as `{ type: 'error', error: ... }` in the stream. The default `onError` handler logs to console. The stream continues but the `finishReason` is set to `'error'`.
38+
39+
There is no way for consumers to:
40+
- Retry the stream automatically on parse errors
41+
- Skip bad events and continue processing (like OpenAI Codex does)
42+
- Distinguish transient parse errors from permanent ones
43+
44+
## How Other CLI Agents Handle This
45+
46+
**OpenAI Codex CLI**: Skips unparseable SSE events with `continue;` and keeps processing. JSON parse errors are logged at debug level. Stream-level errors trigger up to 5 retries with exponential backoff. `CodexErr::Json` is explicitly marked retryable.
47+
48+
**Qwen Code**: SDK's `parseJsonLineSafe()` silently skips failed lines with a warning log.
49+
50+
**Gemini CLI** and **OpenCode**: Same gap — JSON parse errors during SSE consumption are not caught/retried.
51+
52+
## Proposed Solutions
53+
54+
### Option A: Add `isRetryable` property to `AI_JSONParseError`
55+
56+
```typescript
57+
// In JSONParseError constructor, or a subclass for stream context
58+
class StreamJSONParseError extends JSONParseError {
59+
readonly isRetryable = true;
3460
}
3561
```
3662

37-
## Proposed Solution
63+
This would allow consumers to check `isRetryable` consistently across all error types.
3864

39-
Mark `AI_JSONParseError` as retryable when it occurs during stream consumption:
65+
### Option B: Add `onStreamParseError` callback to `streamText`
4066

4167
```typescript
42-
// Current behavior
43-
throw new JSONParseError({ text, cause, isRetryable: false });
44-
45-
// Proposed behavior for streaming errors
46-
throw new JSONParseError({ text, cause, isRetryable: true });
68+
const result = await streamText({
69+
model: myModel,
70+
onStreamParseError: ({ error, skip }) => {
71+
// Option to skip the bad event and continue
72+
skip();
73+
},
74+
});
4775
```
4876

49-
Alternatively, add a configuration option:
77+
### Option C: Allow configuring stream-level retry
5078

5179
```typescript
5280
const result = await streamText({
5381
model: myModel,
54-
retryParseErrors: true, // New option
55-
// ...
82+
streamRetries: 3, // Retry entire stream on mid-stream errors
5683
});
5784
```
5885

5986
## Environment
6087

61-
- AI SDK Version: Multiple (observed in v4.x and v5.x)
62-
- Providers: OpenRouter (proxying to various models)
88+
- AI SDK Version: Observed across v4.x and v5.x
89+
- Provider: `@ai-sdk/openai-compatible` (Kilo AI Gateway)
90+
- Model: Kimi K2.5 (moonshot/kimi-k2.5:free)
6391
- Runtime: Bun 1.3.x
92+
- Timestamp: 2026-02-14T08:34:12Z
6493

6594
## Workaround
6695

67-
We implemented a workaround in our project by detecting JSONParseError in our error handling and treating it as retryable:
96+
We implemented a workaround by detecting `AI_JSONParseError` in our error handler and classifying it as retryable:
6897

6998
```typescript
7099
const isStreamParseError =
71100
e.name === 'AI_JSONParseError' ||
72101
message.includes('AI_JSONParseError') ||
73-
message.includes('JSON parsing failed') ||
74-
message.includes('JSON Parse error');
102+
message.includes('JSON parsing failed');
75103

76104
if (isStreamParseError) {
77-
// Retry with exponential backoff
105+
// Classify as retryable, retry with exponential backoff
78106
}
79107
```
80108

81109
See: https://github.com/link-assistant/agent/issues/169
82-
83-
## Related Issues
84-
85-
- #8577 - AI_JSONParseError with generateText (mentions isRetryable: false)
86-
- #4099 - StreamText error handling
87-
- #5417 (ollama/ollama) - Cloudflare Tunnel causing JSONParseError

‎js/.changeset/stream-parse-error-retry.md‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,17 @@
44

55
fix: Retry on stream parse errors (AI_JSONParseError)
66

7-
Add StreamParseError as a retryable error type to handle malformed JSON in SSE streams
8-
from AI providers. This fixes premature retry failures when providers return corrupted
9-
streaming responses (e.g., concatenated SSE chunks, invalid JSON).
7+
Add StreamParseError as a retryable error type to handle AI_JSONParseError from the
8+
Vercel AI SDK. This error occurs when AI gateways (e.g. Kilo AI Gateway) corrupt SSE
9+
stream chunks when proxying provider responses (e.g. Kimi K2.5, GLM-4.7). The AI SDK's
10+
AI_JSONParseError has no isRetryable property and is never retried by the SDK's built-in
11+
retry mechanism.
1012

1113
- Detect AI_JSONParseError, JSON parsing failures, and malformed JSON errors
1214
- Retry stream parse errors with exponential backoff (1s, 2s, 4s up to 3 retries)
1315
- Add streamParseErrorDelay() function for consistent retry timing
1416
- Add comprehensive test coverage for StreamParseError detection
17+
- Add case study with comparison of 4 CLI agents (Codex, Gemini, Qwen, OpenCode)
18+
- Filed upstream issues: vercel/ai#12595, Kilo-Org/kilocode#5875, anomalyco/opencode#13579
1519

1620
Fixes #169

0 commit comments

Comments
 (0)