The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 18, 2026 3:41am

Merging this PR will not alter performance

✅ 16 untouched benchmarks

_{Comparing Sameerlite:Sameerlite/azure-ai-annotations (6514446) with main (cfeafbe)}

Greptile Summary

This PR fixes a bug where Azure AI Foundry Agents with Bing Search grounding were silently dropping citation annotations from their responses. The fix introduces an _transform_annotations helper that maps the Azure annotation format (where start_index/end_index live at the annotation root level) to the OpenAI-compatible format (where those indices live inside the url_citation sub-object), and threads annotations through both the non-streaming and streaming response-building paths.

Key changes:

• _extract_content_from_messages now returns (content, annotations) instead of just content; callers updated accordingly across sync, async, and streaming paths
• New _transform_annotations method handles url_citation type remapping and passes unknown annotation types through as-is (good forward-compatibility practice)
• _build_model_response conditionally attaches annotations to the Message object only when non-empty, matching OpenAI spec behavior (where Message.annotations is deleted when absent)
• Streaming path (_process_sse_stream) collects annotations from thread.message.completed SSE events across all text content items using extend, correctly accumulating multiple blocks before emitting them on the final [DONE] chunk
• 5 new mock-only unit tests cover annotation extraction, transformation, response building, streaming accumulation, and multi-block accumulation — all compatible with CI requirements

Notes:

• The non-streaming _extract_content_from_messages still exits on the first text content block (already discussed in a prior review thread), while the streaming path iterates all blocks correctly; in practice Azure returns a single text item per message, so this asymmetry is unlikely to affect real usage
• Both Delta and Message Pydantic types already had annotations: Optional[List[ChatCompletionAnnotation]] fields, so no type changes were needed in litellm/types/

Confidence Score: 4/5

• Safe to merge; the fix is narrow in scope and correctly preserves backward compatibility (annotations field is optional and omitted when empty, matching prior behavior)
• The implementation is logically sound: annotation transformation is correct, both paths (sync/async/streaming) are updated, type system compatibility is maintained (ChatCompletionAnnotation/Delta.annotations were pre-existing), and 5 mock-only unit tests validate the behaviour. Score is 4 (not 5) because the non-streaming path still returns early on the first text content block (flagged in a prior review thread but not yet resolved), and the streaming annotation test is missing start_index/end_index assertions inside url_citation (also flagged previously)
• No files require special attention; both changed files are self-contained within the Azure AI Agents handler and its test suite

Sequence Diagram

sequenceDiagram
    participant C as Caller
    participant H as AzureAIAgentsHandler
    participant A as Azure AI Agent API

    C->>H: completion() / acompletion()
    H->>A: POST /threads (create thread)
    A-->>H: {id: thread_id}
    H->>A: POST /threads/{id}/messages
    A-->>H: 200 OK
    H->>A: POST /threads/{id}/runs
    A-->>H: {id: run_id}
    loop Poll until completed
        H->>A: GET /threads/{id}/runs/{run_id}
        A-->>H: {status: "..."}
    end
    H->>A: GET /threads/{id}/messages
    A-->>H: {data: [{role: "assistant", content: [{type: "text", text: {value, annotations}}]}]}
    H->>H: _extract_content_from_messages() → (content, annotations)
    H->>H: _transform_annotations() maps Azure→OpenAI format
    H->>H: _build_model_response(annotations=annotations)
    H-->>C: ModelResponse with Message.annotations

    Note over H,A: Streaming path (SSE)
    C->>H: acompletion_stream()
    H->>A: POST /threads/runs (stream=True)
    loop SSE events
        A-->>H: event: thread.created / data: {id}
        A-->>H: event: thread.message.delta / data: {delta.content}
        H-->>C: ModelResponseStream chunk (content)
        A-->>H: event: thread.message.completed / data: {content[].text.annotations}
        H->>H: collect_annotations.extend(_transform_annotations(...))
        A-->>H: data: [DONE]
        H-->>C: Final chunk with Delta.annotations = collected_annotations
    end

_{Last reviewed commit: "Update litellm/llms/..."}