Summary

Support streaming for pipeline operations on large context payloads.

Motivation

Large context optimization can take several seconds. Streaming provides better UX with progress updates and faster time-to-first-byte.

Components

• Server-sent events (SSE) endpoint - /v1/pipeline/stream
• Chunked transfer encoding - For HTTP/1.1 clients
• Progress callbacks in SDK - Python/Go SDK support

API Design

POST /v1/pipeline/stream
Accept: text/event-stream

Response:
event: progress
data: {"stage": "dedup", "progress": 0.5}

event: progress
data: {"stage": "dedup", "progress": 1.0, "stats": {...}}

event: progress
data: {"stage": "compress", "progress": 0.3}

event: complete
data: {"chunks": [...], "stats": {...}}

SDK Usage

# Python with progress callback
def on_progress(stage, progress, stats):
    print(f"{stage}: {progress*100:.0f}%")

result = client.pipeline(
    chunks=[...],
    stream=True,
    on_progress=on_progress
)

Acceptance Criteria

• First byte in < 100ms for large requests
• Progress updates every 500ms
• Graceful handling of client disconnect
• Works with curl, Python SDK, browser fetch