Summary

openclaw infer model run currently only accepts --prompt <text>. There is no way to pass image/audio files for multimodal inference (e.g., send images + text prompt to a vision model and get a text description back).

This is distinct from infer image describe, which routes through image-generation providers — not vision/understanding models.

Use case

We run a multi-agent system (11 agents on Mac mini M4 Pro) with a TikTok analysis pipeline that:

1. Extracts carousel images or video frames from TikTok posts
2. Sends them to Gemini 2.5 Flash with a text prompt ("describe what you see")
3. Gets a text summary back

Today this requires 50+ lines of raw urllib code managing API keys, base64 encoding, JSON payload construction, and error handling — all of which OpenClaw already manages internally for agents.

Proposed API

# Single image + prompt
openclaw infer model run \
  --model google/gemini-2.5-flash \
  --prompt "Describe this image in French, 150 words max" \
  --file /tmp/frame.jpg \
  --json

# Multiple images + prompt
openclaw infer model run \
  --model google/gemini-2.5-flash \
  --prompt "Describe each image" \
  --file /tmp/frame1.jpg \
  --file /tmp/frame2.jpg \
  --json

The --file flag would:

• Accept image files (jpg, png, webp) and audio files (wav, mp3)
• Base64-encode and attach as inline content parts
• Work with any multimodal-capable model (Gemini, Claude, GPT-4o, Ollama vision models)
• Be repeatable for multi-image prompts

Alternatives considered

• infer image describe: Tried it — routes through image-generation providers, not vision models. Returns "No description returned" even with Google/Gemini configured.
• infer image describe-many: Same issue — it's the generation pipeline, not vision.
• Raw API calls: What we do today. Works but defeats the purpose of having a unified inference hub.

Environment

• OpenClaw 2026.4.9
• macOS 26.3.1 (Apple Silicon M4 Pro)
• Providers: Ollama (local), Google (Gemini), OpenRouter

Filed from a Claude Code session investigating openclaw infer migration for our TikTok analysis pipeline.