Repository navigation
Description
Feature Request
Add a bounded native Gemini Developer API path for generateContent and SSE streamGenerateContent through the existing AI Proxy Protocol/Provider/Transport architecture. The first delivery should preserve the native wire contract instead of requiring clients to use an OpenAI-compatible endpoint.
Current Baseline and Existing Work
At master 88883dc20fb1f57db6343ba811c1d1f3ee70a6bf, AI Proxy documents Chat/Responses support and excludes native Gemini transport. #7382 explicitly leaves Gemini to future work; #7871 provides provider-native protocol routing foundations. #7955 owns shared usage metering and requires actual supported routes before Gemini coverage can be claimed.
First Delivery
- Define an explicit Gemini protocol registration and a provider binding for the direct Developer API. Document the supported API profile, how a routed model maps to the model resource, and how the JSON versus streaming operation is selected.
- Support native text content with ordinary JSON and incremental SSE. Preserve candidate structure, finish/block outcomes and supported native fields. Specify unknown-field handling and reject unsupported multimodal/tool modes without silent loss.
- Bind credentials and endpoints to administrator-owned targets using existing proxy-key and fallback authorization. Safely encode model/path components; never accept an arbitrary client-selected destination or log credentials.
- Reuse WebClient transport, bounded framing, deadlines, backpressure and cancellation ownership. Distinguish protocol completion from HTTP EOF and from OpenAI framing conventions; follow the selected Gemini profile.
- Feed native usage into [Task] Integrate shared per-attempt LLM usage metering across supported protocols #7955/[Task][AI Proxy] Integrate AI Proxy attempt lifecycle with shared usage and invocation telemetry #8125 with source/certainty and attempt identity. Publish which existing Prompt/PII/Token stages can inspect the payload, with unknown/unsupported states rather than inferred policy coverage.
- Keep the existing Chat/Responses and provider contracts compatible. Native support does not automatically authorize lossy cross-protocol conversion.
Acceptance Criteria
- A local native-protocol fixture exercises both operations, model/endpoint/authentication binding, and text JSON/SSE preservation.
- Cover multiple candidates, finish/block outcomes, usage placement, fragmented UTF-8/SSE, malformed/truncated streams and non-success HTTP responses.
- Request/response/event budgets, deadlines and downstream cancellation work through the original upstream exchange; no full-stream aggregation or post-commit replay is introduced.
- Unsupported capability profiles are rejected before network execution; no duplicate metering/debit path is added.
- Document the tested profile, opt-in configuration and governance coverage. Separate real-service smoke evidence from local protocol fixtures.
Scope Boundaries
Limit the first delivery to text generation through the direct Developer API. Vertex authentication/resource naming, Live API/WebSocket, image/audio/video generation, tool execution and protocol conversion are separate increments. Coordinate future shared detection with #8371 instead of adding competing payload guesses.
Protocol Reference
Google documents the separate JSON and streaming operations, native content/candidate representation and usage metadata in Generating content. Pin the supported profile and fixtures during implementation; this link does not establish current ShenYu compatibility.
Activity
hengyuss commentedon Nov 10, 2026
/assign