fix(agent): declare read_file GIF/WEBP images with their real media type - #5847

Closes #5846

Problem

read_file accepts .jpg, .jpeg, .png, .gif and .webp, and FileSystem.read_file_structured (browser_use/filesystem/file_system.py:817-827) base64-encodes the bytes without converting them. The prompt builder that turns those images into message content parts, however, recognised exactly one suffix and assumed JPEG for everything else:

if img_name.lower().endswith('.png'):
	media_type = 'image/png'
else:
	media_type = 'image/jpeg'

so chart.gif left the agent as data:image/jpeg;base64,R0lGODdh... with ImageURL.media_type='image/jpeg'. The declaration is honoured downstream rather than re-derived: llm/anthropic/serializer.py:56 accepts image/gif and image/webp among the types it parses from the data URL, and llm/google/serializer.py:107-109 uses part.image_url.media_type directly under the comment "which correctly identifies the image format". Both are told the payload is a JPEG when it is not.

Change

Two added branches in browser_use/agent/prompts.py, using the two media types SupportedImageMediaType (llm/messages.py:51) already allows. .jpg/.jpeg and unrecognised names keep the existing image/jpeg result, so the only behaviour that changes is GIF and WEBP:

-				if img_name.lower().endswith('.png'):
+				lower_name = img_name.lower()
+				if lower_name.endswith('.png'):
 					media_type = 'image/png'
+				elif lower_name.endswith('.gif'):
+					media_type = 'image/gif'
+				elif lower_name.endswith('.webp'):
+					media_type = 'image/webp'
 				else:
 					media_type = 'image/jpeg'

No re-encoding and no magic-byte sniffing is introduced: the payload stays exactly what it was, only the label it is announced with changes.

Measurements

Local, d8110c5f (main) as the base, one test added to tests/ci/test_file_system_llm_integration.py:

The two pre-existing tests that assert data:image/png and data:image/jpeg for a PNG and a JPG pass identically on both sides, so they are the control for "nothing else moved".

A standalone script (the one in the linked issue) printed the declared type per file name:

Tooling on the diff: ruff check and ruff format --check clean (measured on LF-normalised copies — this clone has core.autocrlf=true, so the working files are CRLF and a raw format check reports a whole-file rewrite). pyright 1.1.414 reports 0 errors on browser_use/agent/prompts.py. An earlier revision of this fix used a dict + next() lookup and pyright rejected the assignment (Type "str" is not assignable to declared type "SupportedImageMediaType"), which is why the final diff is the plain if/elif chain.

Not measured / scope

• I did not run the whole tests/ci directory locally, and I sent no request to a real provider — the verified claim stops at what the agent puts into the message. That the mislabelled declaration then reaches Anthropic/Bedrock/Google as-is is read from the serializer code cited above, not from an API round-trip.
• .webp here means the file the user put on disk; this change does not add validation that the bytes match the extension, and does not touch the screenshot path, which really is PNG.

CLA / provenance

I have signed the CLA as prompted by the bot. This contribution was prepared autonomously by my coding agent and the diff has not been reviewed line-by-line by a human, so the human-only checkboxes are left unticked deliberately; I am the accountable author and will answer review questions and re-verify anything a maintainer asks about.

Summary by cubic

Fixes #5846 so read_file GIF and WEBP images are declared with their real media type instead of being labeled image/jpeg. The prompt builder now adds .gif and .webp branches; PNG, JPEG, and unrecognized names keep existing behavior, and image bytes are unchanged. Adds a test verifying GIF and WEBP data URLs and media types are declared correctly.

^{Written for commit 1ebfc95. Summary will update on new commits.}