Repository navigation
Description
Description
GeminiTools.generate_video builds the returned Video artifact with base64 text instead of the raw video bytes (libs/agno/agno/tools/models/gemini.py:186):
content=base64.b64encode(generated_video.video_bytes).decode("utf-8"),agno.media.Video.content is declared as Optional[bytes] — "Raw video bytes (standardized to bytes)" (libs/agno/agno/media/media.py:530) — so Pydantic silently coerces that str into the UTF-8 bytes of the base64 text. The artifact therefore carries base64 text rather than the video, and everything downstream that calls Video.get_content_bytes() ("Get video content as raw bytes", libs/agno/agno/media/media.py:580, which returns self.content directly) gets base64 text: media offload uploads it as the video file (libs/agno/agno/utils/media_offload.py:226), and saving the artifact to disk writes a corrupt file.
Steps to Reproduce
- Install agno from source (verified at commit
44219f8532e2fe6ce936850455f4b9a2567e4194, agno 3.0.9) withgoogle-genaiinstalled. - Run the script below. It stubs the
google.genaiclient, so no API credentials or network calls are needed:
"""GeminiTools.generate_video: does Video.content hold raw bytes or base64 text?
agno.media.Video declares `content: Optional[bytes]` — "Raw video bytes
(standardized to bytes)". The sibling video tool (opencv.py) passes raw bytes.
"""
from agno.media import Video
from agno.tools.models.gemini import GeminiTools
# Fake MP4 payload
video_bytes = b"\x00\x00\x00\x18ftypmp42" + bytes(range(256))
fake_video = type(
"FakeVideo",
(),
{
"video": type("V", (), {"video_bytes": video_bytes, "mime_type": "video/mp4"})(),
},
)()
operation = type(
"Op",
(),
{
"done": True,
"result": type("R", (), {"generated_videos": [fake_video]})(),
},
)()
tools = GeminiTools(api_key="test")
tools.vertexai = True # video generation requires Vertex AI mode
tools.client = type(
"FakeClient",
(),
{
"models": type(
"FakeModels",
(),
{"generate_videos": lambda self, **kw: operation},
)(),
"operations": type("FakeOps", (), {"get": lambda self, operation: operation})(),
},
)()
result = tools.generate_video(agent=None, prompt="a cat")
v = result.videos[0]
print("Video.content type:", type(v.content).__name__)
print("Video.content == raw bytes? ", v.content == video_bytes)
import base64
print("Video.content == base64 text (utf-8)? ", v.content == base64.b64encode(video_bytes).decode("utf-8").encode("utf-8"))
print("get_content_bytes() == raw bytes? ", v.get_content_bytes() == video_bytes)
print("len(raw) =", len(video_bytes), "| len(content) =", len(v.content))
# What the type itself says a plain caller gets:
print("Video(content=<raw bytes>).content type:", type(Video(content=video_bytes).content).__name__)Agent Configuration (if applicable)
# Video generation on GeminiTools requires Vertex AI mode:
agent = Agent(
model=Gemini(...),
tools=[GeminiTools(vertexai=True)],
)When the model calls generate_video, the artifacts flow ToolResult.videos → function_execution_result.videos (libs/agno/agno/models/base.py:3042-3043) → run_response.videos, and media offload uploads get_content_bytes() as the video file (libs/agno/agno/utils/media_offload.py:226).
Expected Behavior
Video.content should hold the raw video bytes, per the class's own contract:
content: Optional[bytes] = None # Raw video bytes (standardized to bytes)—libs/agno/agno/media/media.py:530Video.get_content_bytes()is documented "Get video content as raw bytes" and returnsself.contentdirectly —libs/agno/agno/media/media.py:580-582Video.to_base64()(libs/agno/agno/media/media.py:634-642) base64-encodesget_content_bytes(), i.e.contentis expected to be raw; pre-encoded content would be double-encoded- The model input path passes
video.contentstraight toPart.from_bytes(data=video.content)(libs/agno/agno/models/google/gemini.py:1086), which expects raw bytes - The sibling
generate_imagebranch in the same file passes raw bytes, with the comment "Create ImageArtifact with raw bytes (not base64 encoded)" (libs/agno/agno/tools/models/gemini.py:111-117) - The sibling video toolkit passes raw bytes:
content=video_bytes(libs/agno/agno/tools/opencv.py:295-299); every other video toolkit passesurl=— fal (libs/agno/agno/tools/fal.py:108-111), minimax (libs/agno/agno/tools/minimax.py:131-134), models_labs (libs/agno/agno/tools/models_labs.py:129), replicate (libs/agno/agno/tools/replicate.py:111), wavespeed (libs/agno/agno/tools/wavespeed.py:103), lumalab (libs/agno/agno/tools/lumalab.py:112)
The Gemini video branch is the only one that base64-encodes.
Actual Behavior
Video.content holds the UTF-8 bytes of the base64 text. For a 268-byte MP4 payload the artifact carries 360 bytes of base64 text; get_content_bytes() returns that text, and a file written from the artifact is base64 text, not a video.
Screenshots or Logs (if applicable)
Verbatim output of the script above (run 1):
[B] content type: bytes
[B] content == raw video bytes? False
[B] content == utf-8(base64 text)? True
[B] get_content_bytes() == raw? False
[B] len(raw) = 268 | len(content) = 360
[C] file written from artifact == base64 text? True / first 16 bytes on disk: b'AAAAGGZ0eXBtcDQy'
An independent second run of the same script:
Video.content type: bytes
Video.content == raw bytes? False
Video.content == base64 text (utf-8)? True
get_content_bytes() == raw bytes? False
len(raw) = 268 | len(content) = 360
Environment
- OS: macOS (arm64)
- Agno Version: 3.0.9 (source checkout at commit
44219f8532e2fe6ce936850455f4b9a2567e4194) - External Dependency Versions: google-genai 2.25.0
- Additional Environment Details: Python 3.12.14; reproduced with a stubbed
google.genaiclient (no API calls)
Possible Solutions
Pass the raw bytes instead: content=generated_video.video_bytes (drop the base64.b64encode(...).decode("utf-8") wrapper), matching the generate_image branch in the same file and Video.content's declared contract. The mime_type/original_prompt handling stays as is. Happy to open a PR with this change.
Additional Context
- No existing issue or PR found for this (searched the tracker for "gemini generate_video" / "generate_video base64").
- The comment above the line (
libs/agno/agno/tools/models/gemini.py:183, "Create VideoArtifact with base64 encoded content") appears stale —libs/agno/agno/tools/opencv.py:294carries the same comment but passes raw bytes.
Activity
BlueX888 commentedon Sep 25, 2026
Reproduced this. Root cause: libs/agno/agno/tools/models/gemini.py:186 passes base64.b64encode(video_bytes).decode("utf-8") into Video.content, which is declared Optional[bytes], so pydantic coerces it to the UTF-8 bytes of the base64 text and get_content_bytes() consumers (media offload upload, saving to disk) receive base64 text instead of the video.
Minimal reproduction:
Observed output:
Fix is ready on
BlueX888:fix/prep-gemini-generate-video-stores-base64-text-as-video-content(commit5fc2700a). Passes the raw video_bytes to Video.content, dropping the base64 wrapper — matching the generate_image branch in the same toolkit and the declared raw-bytes contract.Verified with:
pytest tests/unit/tools/models/test_gemini.py (run from libs/agno; 11 passed, including the updated test_generate_video_success asserting content and get_content_bytes() equal the raw bytes)Happy to open the PR once you confirm the approach or assign this to me.
fix: store raw video bytes in GeminiTools.generate_video artifact (#1…
sannya-singal commentedon Sep 25, 2026
Resolved with #10554.