Repository navigation
Description
What happened, and what did you expect?
Calling the public API inspect_ai.model.messages_from_google() on a Google GenAI Content that carries an inline_data part with real binary image bytes raises:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte
(every PNG starts with \x89PNG, so this is the standard Gemini multimodal user turn). Dict input with a base64-string data hits the same path, because google-genai's Blob decodes the string to raw bytes first.
Root cause — content_from_google_parts() in src/inspect_ai/model/_google_convert.py:244-250:
elif part.inline_data and part.inline_data.mime_type and part.inline_data.data:
content.append(
ContentImage(
image=as_data_uri(
part.inline_data.mime_type,
part.inline_data.data.decode("utf-8"),
)
)
)google.genai.types.Blob.data is typed Optional[bytes] and documented "Required. The raw bytes of the data.", so .decode("utf-8") fails on any real image, while as_data_uri() formats data:{mime_type};base64,{data} — its payload must be base64 text, not decoded raw bytes.
Two secondary effects of the same branch:
- If the bytes happen to be UTF-8-decodable, the conversion succeeds but emits an invalid data URI (
data:image/png;base64,hello-not-base64!!) — the payload is not base64. - The truthiness guard silently drops a part whose
datais empty bytes or whosemime_typeis missing, instead of converting it or erroring.
Expected: the same handling as the sibling branch for the identical Part/Blob in completion_choice_from_candidate(), src/inspect_ai/model/_providers/google.py:1975-1976:
if blob.data is not None:
b64 = base64.b64encode(blob.data).decode()
data_uri = f"data:{blob.mime_type};base64,{b64}"i.e. base64-encode the raw bytes, guarding only on data is not None. That is also this module's own round-trip contract: chat_content_to_part() sends ContentImage back as Part.from_bytes(mime_type, raw_bytes).
Steps to reproduce
import asyncio
from google.genai import types as t
from inspect_ai.model import messages_from_google
PNG = b"\x89PNG\r\n\x1a\n" + b"\x00\x01\x02image-bytes\xff\xfe"
async def main():
contents = [
t.Content(role="user", parts=[
t.Part(inline_data=t.Blob(mime_type="image/png", data=PNG)),
t.Part(text="what is in this image?"),
]),
]
try:
msgs = await messages_from_google(contents)
print("converted:", msgs[0].content)
except Exception as ex:
print(f"RAISED {type(ex).__name__}: {ex}")
asyncio.run(main())Output (verbatim, from three independent runs — my own plus two verifier runs):
RAISED UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte
The verifier runs also covered the two secondary cases, verbatim:
dict Content, b64-str data: RAISED UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte
utf8-decodable bytes: uri payload NOT valid base64: 'data:image/png;base64,hello-not-base64!!'
empty data bytes -> [ContentText(..., text='kept', ...)] / missing mime_type -> [ContentText(..., text='kept', ...)]
(in the last two, the inline_data part disappears entirely — only the sibling text part survives).
Version and environment
- inspect-ai at commit
1276f419913248bdb4c7c41a8e2274a396e97083(currentmain; thesrc/inspect_ai/model/_google_convert.pymodule was added in Review tool results after execution and stop a sample on what a tool returned #5356 (c94ab26, 2026-09-11), so the regression is main-only so far, after v0.3.263) - Python 3.12.14, google-genai 2.24.0 (also verified with 2.22.0)
- macOS 26.6.2 (arm64)
tests/model/test_google_convert.py(19 tests) has noinline_datacoverage, which is why the suite stays green
Related: #3516 touched null inline_data handling in the provider path (_providers/google.py) — different file and defect. I searched open and closed issues and open PRs for messages_from_google / content_from_google_parts / inline_data and found nothing covering this.
Happy to be assigned — I can open a PR mirroring the provider branch (guard on part.inline_data is not None and part.inline_data.data is not None, then as_data_uri(mime_type, base64.b64encode(part.inline_data.data).decode()), with a default mime type when it is None) plus a regression test using real binary bytes.
AI disclosure: found and verified with an AI coding agent (Claude); the reproduction above was run by me and independently by two separate verifier runs, and I can explain and defend every line of the report.
Activity
BlueX888 commentedon Sep 20, 2026
Reproduced this. Root cause: src/inspect_ai/model/_google_convert.py:249 (content_from_google_parts) decodes google-genai Blob.data, which is raw media bytes, with .decode('utf-8') and hands the result to as_data_uri(), whose payload must be base64 text.
Minimal reproduction:
Observed output:
Fix is ready on
BlueX888:fix/prep-google-convert-inline-data-utf8-decode(commitdfcd9b1d). Mirrors the provider branch in _providers/google.py: base64-encode the raw bytes, guard on inline_data/data being not None, and default a missing mime_type to image/png instead of dropping the part.Verified with:
pytest tests/model/test_google_convert.py -q (22 passed; also 22 passed with --runtrio)Happy to open the PR once you confirm the approach or assign this to me.
dragonstyle commentedon Oct 16, 2026
Accepting the narrow fix: convert inline image bytes to a base64 data URI instead of decoding them as UTF-8, using the MIME type the response provides. An unknown binary type shouldn't be assumed to be PNG.
fix(google): base64-encode inline image bytes without assuming a PNG …