Skip to content

feat(gemini): Phase 2 -- streaming support (SSE) #1594

@bug-ops

Description

@bug-ops

Part of #1592

Scope

Add SSE streaming support for Gemini's streamGenerateContent endpoint.

Files to Modify

  • crates/zeph-llm/src/gemini.rs -- implement chat_stream(), update supports_streaming()
  • crates/zeph-llm/src/sse.rs -- add gemini_sse_to_stream() parser

Key Implementation Details

  • Streaming endpoint: POST /v1beta/models/{model}:streamGenerateContent?key={API_KEY}&alt=sse
  • Each SSE data: event contains a partial JSON response with full candidate structure
  • Text is delivered incrementally in candidates[0].content.parts[0].text
  • Gemini 2.5 models emit thinking content as parts with thought: true flag -- map to StreamChunk::Thinking
  • Need to handle both text-only and thinking+text streaming

Acceptance Criteria

  • supports_streaming() returns true
  • chat_stream() produces correct StreamChunk::Content items
  • Thinking content emitted as StreamChunk::Thinking for 2.5 models
  • SSE connection errors handled gracefully with retry
  • status_tx events emitted on retry
  • Unit tests with wiremock SSE mock responses

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions