-
Notifications
You must be signed in to change notification settings - Fork 2
feat(gemini): Phase 2 -- streaming support (SSE) #1594
Copy link
Copy link
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Part of #1592
Scope
Add SSE streaming support for Gemini's streamGenerateContent endpoint.
Files to Modify
crates/zeph-llm/src/gemini.rs-- implementchat_stream(), updatesupports_streaming()crates/zeph-llm/src/sse.rs-- addgemini_sse_to_stream()parser
Key Implementation Details
- Streaming endpoint:
POST /v1beta/models/{model}:streamGenerateContent?key={API_KEY}&alt=sse - Each SSE
data:event contains a partial JSON response with full candidate structure - Text is delivered incrementally in
candidates[0].content.parts[0].text - Gemini 2.5 models emit thinking content as parts with
thought: trueflag -- map toStreamChunk::Thinking - Need to handle both text-only and thinking+text streaming
Acceptance Criteria
-
supports_streaming()returnstrue -
chat_stream()produces correctStreamChunk::Contentitems - Thinking content emitted as
StreamChunk::Thinkingfor 2.5 models - SSE connection errors handled gracefully with retry
-
status_txevents emitted on retry - Unit tests with wiremock SSE mock responses
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request