Summary

A within-limit image whose MIME type is not one of Anthropic's four supported image types (for example image/heic, the iPhone camera default, or image/tiff) is sent to the Anthropic Messages API with its real media_type passed through verbatim, and the API rejects the whole turn with a 400. The identical image block succeeds on OpenAI and Gemini, so the failure is Anthropic specific. The user gets no reply.

Environment

• Commit: aad9974 (origin/main at time of filing)
• Surface: Anthropic provider adapter (packages/ai), and the custom Anthropic transport (src/agents/anthropic-transport-stream.ts)

Steps to reproduce

1. Configure a Claude model (Anthropic provider) with NO media-understanding/vision-describe provider set, so inbound images are forwarded to the model as image blocks rather than transcribed to text.
2. Send an iPhone HEIC photo (or any image/heic/image/tiff) that is under the size and dimension limits, so the sanitizer's re-encode-to-JPEG ladder never triggers.
3. Observe the turn fail: the outgoing request carries media_type: "image/heic" and Anthropic returns a 400 invalid_request_error.

Expected

Unsupported-but-image MIME types are normalized/transcoded to a supported type (image/jpeg/png/gif/webp) before the request is sent to Anthropic, so a within-limit HEIC/TIFF photo is accepted, matching the OpenAI and Gemini behavior on the same bytes.

Actual

The real media_type string is forwarded unchanged and Anthropic rejects the turn with 400, so the user receives no reply.

Root cause (if known)

The adapter applies a TypeScript as cast that has no runtime effect; the real MIME string passes through unchanged.

packages/ai/src/providers/anthropic.ts:1478 (user-message image block):

media_type: item.mimeType as "image/jpeg" | "image/png" | "image/gif" | "image/webp",

packages/ai/src/providers/anthropic.ts:212 (tool-result image block) applies the same cast with an image/jpeg fallback only when mimeType is not a string. The custom transport repeats it at src/agents/anthropic-transport-stream.ts:444 (user message) and :379 (tool result).

Anthropic's Messages API accepts only image/jpeg, image/png, image/gif, image/webp. Upstream the inbound path gates media with startsWith("image/") only (for example src/auto-reply/reply/agent-turn-attachments.ts:37 and :175), and the image sanitizer re-encodes to JPEG ONLY when the image exceeds size/dimension limits (src/agents/tool-images.ts resizeImageBase64IfNeeded returns early with the original mimeType when imageWithinLimits is true). So a within-limit image/heic reaches the Anthropic adapter with its real media_type and the API 400s.

The transcode helpers already exist but are not applied on this path: normalizeImageDescriptionInput (src/media-understanding/image-input-normalize.ts) is only called from the media-understanding runner/runtime (src/media-understanding/runner.entries.ts:783, src/media-understanding/runtime.ts:259), i.e. the vision-describe path, and the JPEG re-encode fires only over-limit. Neither covers a within-limit inbound HEIC forwarded directly to the model.

Sibling surfaces

• OpenAI adapter emits the MIME inside a data URL without a narrow allowlist (packages/ai/src/providers/openai-completions.ts:1059 and :1214, url: data:${item.mimeType};base64,...); Gemini passes it verbatim into inlineData.mimeType (packages/ai/src/providers/google-shared.ts:202 and :301). Both upstream APIs accept HEIC/HEIF, so the same bytes succeed there. The defect is Anthropic specific.
• Both the packaged provider (packages/ai) and the custom transport (src/agents/anthropic-transport-stream.ts) carry the same verbatim pass-through, on both the user-message and tool-result image paths.
• Secondary weaker symptom, do not split: a image/bmp inbound image degrades to a text placeholder (silent content loss, no 400) rather than being transcoded.
• Distinct from fix(media-understanding): normalize HEIC before image descriptions #86037 (HEIC normalize for image descriptions, media-understanding path only), Gateway: normalize HEIC input_image sources #38122/Gateway: follow up HEIC input image handling #38146 (gateway OpenResponses/OpenAI-HTTP ingestion), Inbound images silently fail: replayed/history image attachments sent to Anthropic with non-ASCII source.base64 #86984 (non-ASCII source.base64 data field, not media_type), and fix(gateway): accept file-only input on /v1/responses (parity with image-only) #93011 (file-only turns). None guard the Anthropic adapter's verbatim media_type on a within-limit inbound image.

Real behavior proof

Behavior addressed: within-limit image/heic sent to the Anthropic adapter carries media_type: "image/heic" verbatim and the Messages API rejects it 400, while a normalized image/jpeg is accepted.
Real environment tested: drove the real streamAnthropic() export from packages/ai/src/providers/anthropic.ts at commit aad9974, against a loopback HTTP server (127.0.0.1) that captures the outgoing request body; only the network boundary was substituted, the adapter, request builder, and Anthropic SDK stayed real. The server replied with Anthropic's documented unsupported-media_type 400 for any non-allowlisted type.
Exact steps or command run after this patch: built a user message with a single within-limit image block { type: "image", mimeType: "image/heic", data }, passed it through streamAnthropic(model, context, { apiKey, maxRetries: 0 }).result(), and read back sentMediaType (from the captured body), stopReason, and errorMessage. Reran the identical input after locally normalizing unsupported image mimes to image/jpeg at anthropic.ts:1478.
Evidence after fix:

# OBSERVED (buggy, origin/main aad99747cd45b3845e027c2297b85835b0bef75f)
outgoing media_type: image/heic
stopReason: error
errorMessage: 400 {"type":"error","error":{"type":"invalid_request_error","message":"messages.0.content.0.image.source.base64.media_type: Input should be 'image/jpeg', 'image/png', 'image/gif' or 'image/webp'"}}

# EXPECTED (identical HEIC input, media_type normalized to image/jpeg at anthropic.ts:1478)
outgoing media_type: image/jpeg
stopReason: stop
errorMessage:

Observed result after fix: on current main the within-limit HEIC block goes out as media_type: image/heic and Anthropic 400s the turn; normalizing the media_type to a supported value makes the identical request succeed.
What was not tested: did not exercise a live api.anthropic.com endpoint (the upstream 400 is Anthropic's documented contract for non-allowlisted media_type, reproduced by the loopback), did not byte-transcode HEIC to JPEG in the proof (the fix must transcode the bytes, not only relabel the MIME), and did not measure the image/bmp placeholder degradation beyond noting it.