The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Error		Mar 16, 2026 11:40am

Merging this PR will not alter performance

✅ 16 untouched benchmarks

_{Comparing awais786:fix/tool-result-images-gemini (41f3282) with main (58e74a6)}

Greptile Summary

This PR fixes silent image data loss in convert_to_gemini_tool_call_result when routing Anthropic /v1/messages requests through Gemini. Two previously unhandled cases are now correctly converted to Gemini inline_data parts: (1) tool result content that is a bare data-URL string, and (2) Anthropic-native {"type": "image", "source": {...}} blocks in list content. The inline_data single-variable was also correctly refactored to an inline_data_list so multiple images in one tool result are all preserved.

Key changes:

• Data-URL strings (e.g. data:image/png;base64,...) in str content are detected, parsed, and promoted to inline_data entries; extra MIME parameters like ;charset=UTF-8 are stripped cleanly
• Anthropic "image" blocks with "source": {"type": "base64", ...} in list content are now converted rather than silently skipped
• Pre-existing input_image/image_url handling is also migrated from the old single-variable pattern to the list, preserving multiple images correctly
• Four focused unit tests are added covering single image, multiple images, plain data-URL, and data-URL with extra MIME params — all mock-only with no real network calls

Issues found:

• elif content_type == "image": only handles source.type == "base64"; blocks with source.type == "url" are silently discarded without logging a warning, which can cause confusing data loss

Confidence Score: 4/5

• PR is safe to merge with one minor gap to address: URL-sourced Anthropic image blocks are silently dropped.
• The core logic is correct and well-tested. The inline_data_list refactor is clean and backward-compatible — callers already handle both VertexPartType and List[VertexPartType] returns. The only concern is that the new "image" branch silently discards URL-typed source blocks without a warning, which could cause confusing behaviour for users passing Anthropic URL images through tool results. This is a minor gap that doesn't affect the primary bug fix.
• litellm/litellm_core_utils/prompt_templates/factory.py — the new elif content_type == "image": branch (lines 1530–1544) should log a warning for unhandled source types to avoid silent data loss.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[convert_to_gemini_tool_call_result] --> B{message content type?}
    B -->|str| C{starts with data: and has ;base64,?}
    C -->|Yes| D[Parse MIME type, strip extra params\nadd to inline_data_list\nclear content_str]
    C -->|No| E[Use as plain content_str]
    B -->|list| F[Iterate content blocks]
    F --> G{block type?}
    G -->|text| H[Append to content_str]
    G -->|image NEW| I{source.type == base64?}
    I -->|Yes| J[BlobType from source.data + source.media_type\nadd to inline_data_list]
    I -->|No URL/other| K[⚠️ Silently skipped — no warning logged]
    G -->|input_image / image_url| L[convert_to_anthropic_image_obj\nadd BlobType to inline_data_list]
    G -->|file / input_file| M[convert_to_anthropic_image_obj\nadd BlobType to inline_data_list]
    D --> N[Build function_response from content_str]
    E --> N
    H --> N
    J --> N
    L --> N
    M --> N
    N --> O{inline_data_list empty?}
    O -->|No| P[Return list: function_response part + one inline_data part per entry]
    O -->|Yes| Q[Return single function_response VertexPartType]

_{Last reviewed commit: 41f3282}