|
1 | | -# Feature Request: Make AI_JSONParseError Retryable for Streaming |
| 1 | +# Feature Request: `AI_JSONParseError` should support retry for mid-stream parse errors |
2 | 2 |
|
3 | 3 | ## Description |
4 | 4 |
|
5 | | -When using `streamText` with providers that return malformed SSE responses (e.g., OpenRouter proxying to Kimi K2.5), the AI SDK throws `AI_JSONParseError` which is marked as `isRetryable: false`. However, these errors are typically transient and caused by: |
| 5 | +When using `streamText` with OpenAI-compatible providers (via `@ai-sdk/openai-compatible`), the AI SDK may throw `AI_JSONParseError` during stream consumption when a provider or gateway returns malformed SSE data. Currently, this error: |
6 | 6 |
|
7 | | -1. SSE chunks being concatenated incorrectly at proxy level |
8 | | -2. Network issues corrupting stream data |
9 | | -3. Provider returning invalid JSON temporarily |
| 7 | +1. Has **no `isRetryable` property** (only `APICallError` has it) |
| 8 | +2. Is **never retried** by the SDK's built-in `retryWithExponentialBackoff` mechanism |
| 9 | +3. Occurs **after** the HTTP request succeeds (HTTP 200), so the request-level retry is already bypassed |
10 | 10 |
|
11 | | -These errors should be retryable because they're transient infrastructure issues, not permanent failures. |
| 11 | +These errors are typically **transient** — caused by: |
| 12 | +- SSE chunks being concatenated incorrectly at proxy/gateway level |
| 13 | +- Network issues corrupting stream data mid-flight |
| 14 | +- Provider temporarily returning invalid JSON in stream |
12 | 15 |
|
13 | 16 | ## Evidence from Production |
14 | 17 |
|
15 | | -We observed this error pattern in production logs: |
| 18 | +We observed this error when using Kimi K2.5 via the Kilo AI Gateway (`api.kilo.ai`): |
16 | 19 |
|
17 | 20 | ```json |
18 | 21 | { |
19 | 22 | "name": "AI_JSONParseError", |
20 | 23 | "cause": {}, |
21 | | - "text": "{\"id\":\"chatcmpl-jQugNdata:{\"id\":\"chatcmpl-iU6vkr3fItZ0Y4rTCmIyAnXO\",...}" |
| 24 | + "text": "{\"id\":\"chatcmpl-jQugNdata:{\"id\":\"chatcmpl-iU6vkr3fItZ0Y4rTCmIyAnXO\",\"object\":\"chat.completion.chunk\",...}" |
22 | 25 | } |
23 | 26 | ``` |
24 | 27 |
|
25 | | -Notice the corrupted data: `"chatcmpl-jQugNdata:{"` - two SSE chunks concatenated together without proper parsing. |
| 28 | +The `data:` SSE prefix of the second event was embedded inside the first event's JSON, indicating SSE chunking corruption at the gateway level. |
26 | 29 |
|
27 | | -The error currently has `isRetryable: false`, which prevents retry logic from working: |
| 30 | +This is a known class of issue — similar reports: |
| 31 | +- [ollama/ollama#5417](https://github.com/ollama/ollama/issues/5417): Cloudflare Tunnel + Vercel AI SDK = AI_JSONParseError |
| 32 | +- [anomalyco/opencode#7692](https://github.com/anomalyco/opencode/issues/7692): Stream chunks concatenated incorrectly with GLM-4.7 |
| 33 | +- [vercel/ai#4099](https://github.com/vercel/ai/issues/4099): streamText error handling |
28 | 34 |
|
29 | | -```json |
30 | | -{ |
31 | | - "isRetryable": false, |
32 | | - "name": "AI_APICallError", |
33 | | - ... |
| 35 | +## Current Behavior |
| 36 | + |
| 37 | +The AI SDK's parsing pipeline (`parseJsonEventStream`) uses `safeParseJSON` which returns `{ success: false, error: JSONParseError }` on parse failure. The OpenAI-compatible model handler then emits this as `{ type: 'error', error: ... }` in the stream. The default `onError` handler logs to console. The stream continues but the `finishReason` is set to `'error'`. |
| 38 | + |
| 39 | +There is no way for consumers to: |
| 40 | +- Retry the stream automatically on parse errors |
| 41 | +- Skip bad events and continue processing (like OpenAI Codex does) |
| 42 | +- Distinguish transient parse errors from permanent ones |
| 43 | + |
| 44 | +## How Other CLI Agents Handle This |
| 45 | + |
| 46 | +**OpenAI Codex CLI**: Skips unparseable SSE events with `continue;` and keeps processing. JSON parse errors are logged at debug level. Stream-level errors trigger up to 5 retries with exponential backoff. `CodexErr::Json` is explicitly marked retryable. |
| 47 | + |
| 48 | +**Qwen Code**: SDK's `parseJsonLineSafe()` silently skips failed lines with a warning log. |
| 49 | + |
| 50 | +**Gemini CLI** and **OpenCode**: Same gap — JSON parse errors during SSE consumption are not caught/retried. |
| 51 | + |
| 52 | +## Proposed Solutions |
| 53 | + |
| 54 | +### Option A: Add `isRetryable` property to `AI_JSONParseError` |
| 55 | + |
| 56 | +```typescript |
| 57 | +// In JSONParseError constructor, or a subclass for stream context |
| 58 | +class StreamJSONParseError extends JSONParseError { |
| 59 | + readonly isRetryable = true; |
34 | 60 | } |
35 | 61 | ``` |
36 | 62 |
|
37 | | -## Proposed Solution |
| 63 | +This would allow consumers to check `isRetryable` consistently across all error types. |
38 | 64 |
|
39 | | -Mark `AI_JSONParseError` as retryable when it occurs during stream consumption: |
| 65 | +### Option B: Add `onStreamParseError` callback to `streamText` |
40 | 66 |
|
41 | 67 | ```typescript |
42 | | -// Current behavior |
43 | | -throw new JSONParseError({ text, cause, isRetryable: false }); |
44 | | - |
45 | | -// Proposed behavior for streaming errors |
46 | | -throw new JSONParseError({ text, cause, isRetryable: true }); |
| 68 | +const result = await streamText({ |
| 69 | + model: myModel, |
| 70 | + onStreamParseError: ({ error, skip }) => { |
| 71 | + // Option to skip the bad event and continue |
| 72 | + skip(); |
| 73 | + }, |
| 74 | +}); |
47 | 75 | ``` |
48 | 76 |
|
49 | | -Alternatively, add a configuration option: |
| 77 | +### Option C: Allow configuring stream-level retry |
50 | 78 |
|
51 | 79 | ```typescript |
52 | 80 | const result = await streamText({ |
53 | 81 | model: myModel, |
54 | | - retryParseErrors: true, // New option |
55 | | - // ... |
| 82 | + streamRetries: 3, // Retry entire stream on mid-stream errors |
56 | 83 | }); |
57 | 84 | ``` |
58 | 85 |
|
59 | 86 | ## Environment |
60 | 87 |
|
61 | | -- AI SDK Version: Multiple (observed in v4.x and v5.x) |
62 | | -- Providers: OpenRouter (proxying to various models) |
| 88 | +- AI SDK Version: Observed across v4.x and v5.x |
| 89 | +- Provider: `@ai-sdk/openai-compatible` (Kilo AI Gateway) |
| 90 | +- Model: Kimi K2.5 (moonshot/kimi-k2.5:free) |
63 | 91 | - Runtime: Bun 1.3.x |
| 92 | +- Timestamp: 2026-02-14T08:34:12Z |
64 | 93 |
|
65 | 94 | ## Workaround |
66 | 95 |
|
67 | | -We implemented a workaround in our project by detecting JSONParseError in our error handling and treating it as retryable: |
| 96 | +We implemented a workaround by detecting `AI_JSONParseError` in our error handler and classifying it as retryable: |
68 | 97 |
|
69 | 98 | ```typescript |
70 | 99 | const isStreamParseError = |
71 | 100 | e.name === 'AI_JSONParseError' || |
72 | 101 | message.includes('AI_JSONParseError') || |
73 | | - message.includes('JSON parsing failed') || |
74 | | - message.includes('JSON Parse error'); |
| 102 | + message.includes('JSON parsing failed'); |
75 | 103 |
|
76 | 104 | if (isStreamParseError) { |
77 | | - // Retry with exponential backoff |
| 105 | + // Classify as retryable, retry with exponential backoff |
78 | 106 | } |
79 | 107 | ``` |
80 | 108 |
|
81 | 109 | See: https://github.com/link-assistant/agent/issues/169 |
82 | | - |
83 | | -## Related Issues |
84 | | - |
85 | | -- #8577 - AI_JSONParseError with generateText (mentions isRetryable: false) |
86 | | -- #4099 - StreamText error handling |
87 | | -- #5417 (ollama/ollama) - Cloudflare Tunnel causing JSONParseError |
|
0 commit comments