Repository navigation
feat(ai): add opt-in raw provider events to LLM streams - #59382
Conversation
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
No blocking issues; the feature works as described. Using a scratch OpenAI Chat stream with includeRaw: true, I checked that each chunk's raw event comes right before that chunk's own events, that the usage-only chunk and the [DONE] sentinel (as a string) both show up, and that generate is unaffected. The packages/ai suite and tsc type check pass.
Public API: adds includeRaw?: boolean to LLMRequest and the { type: "raw", data: unknown } variant to LLMEvent, with LLMEvent.raw and LLMEvent.is.raw. Both are additions and backwards compatible. Code that switches over every LLMEvent type with an exhaustive check will need a raw case to compile.
One small point inline: Bedrock raw frames include the transport's internal rawBody field.
| state = next | ||
| return output | ||
| return request.includeRaw && isProviderFrame(frame) | ||
| ? [LLMEvent.raw({ data: rawFrame(frame) }), ...output] |
There was a problem hiding this comment.
For Bedrock, frame is the event-stream record built in bedrock-event-stream.ts, so data will include the internal rawBody string next to the event, which repeats the payload. Since data is public, it may be worth stripping that field here (or letting the transport supply the raw value) before callers start depending on it.
There was a problem hiding this comment.
Addressed in 8453f6e. Framing definitions can now say what the raw value of a frame is (an optional raw(frame) next to body(frame)). Bedrock uses it to drop rawBody, so raw data is { <eventType>: payload } with padding removed. Exceptions keep { exception: { type, details } }. A Bedrock test covers it.
There was a problem hiding this comment.
The earlier Bedrock point is fixed: the framing's new raw hook strips rawBody, and the new test checks that raw data contains only the event payload. The packages/ai suite and tsc type check pass. Nothing new to raise.
Public API is the same as before: adds includeRaw?: boolean to LLMRequest and the raw variant to LLMEvent (LLMEvent.raw, LLMEvent.is.raw). Both are backwards compatible. The Framing.Definition.raw and TransportExecution.raw hooks are new and optional.
Adds an opt-in way to see every provider frame in an LLM stream.
Shape
includeRaw?: booleanonLLMRequest. It works throughLLM.request,LLM.streamand the Promise client (ai.llm.stream({ ..., includeRaw: true })) with no extra plumbing.LLMEventvariant:{ type: "raw", data: unknown }, withLLMEvent.raw(...)andLLMEvent.is.raw.includeRawis set, each decoded provider frame produces one raw event, in order, right before the events parsed from that frame. Frames the protocol ignores still show up, so callers can translate unknown events with a plain stream map.datais the parsed frame: JSON text frames are parsed (non-JSON sentinels like[DONE]stay strings), and already-decoded frames (Bedrock event-stream records, Gemini's parsed chunks) pass through as the protocol sees them. Transport-internal markers (Gemini's continuation boundary) are skipped.Where
Implemented once in the shared route stream in
packages/ai/src/route/client.ts, so every chat protocol gets it with no protocol changes. The terminal-event guard ignores raw events, so a frame that trailsfinish(for example an Anthropicping) does not fail the stream.LLMResponse.reduceskips raw events, sogeneratereturns the same response with or without the option.Prior art
The AI SDK's
includeRawChunksoption and its{ type: 'raw', rawValue }stream part.Scope
LLMEvent, so the client does not need regenerating.Tests
packages/ai/test/raw-events.test.ts: raw events come in order before each frame's own events, include a frame the protocol ignores and a frame afterfinish, and do not appear without the option;generategives the same result with the option on.packages/aisuite passes, recorded tests replay unchanged;bun run checkpasses.Added by the opencode bot (AI-generated).
Public API: adds
includeRawto LLMRequest and arawLLMEvent variant to @opencode/ai (backwards compatible)