Repository navigation
read: audio files are rejected as binary, so multimodal models never receive them #54491
Description
Description
The read tool cannot read audio files, so a multimodal model with audio input can never receive one.
packages/core/src/tool/read-filesystem.ts gates media on a fixed set:
export const MEDIA_MIMES = new Set(["image/png", "image/jpeg", "image/gif", "image/webp", "application/pdf"])Mime.detect (packages/core/src/mime.ts) sniffs magic bytes for PNG, JPEG, GIF, BMP, PDF, WEBP and AVIF only — there is no audio signature. An .mp3 (or .wav, .m4a, .ogg, .flac) therefore detects as application/octet-stream, misses MEDIA_MIMES, and falls through to the text path, where the first NUL byte rejects it:
if (first.bytes.includes(0)) return yield* new BinaryFileError({ resource })So read on an audio file fails with BinaryFileError.
The rest of the pipeline already supports audio. packages/core/src/session/model-request.ts classifies a file part's mime into a modality and checks it against the model's declared capabilities:
const mimeToModality = (mime: string) => {
if (mime.startsWith("image/")) return "image"
if (mime.startsWith("audio/")) return "audio"
if (mime.startsWith("video/")) return "video"
if (mime === "application/pdf") return "pdf"
}read is the only place that excludes audio, and the tool's own description says "Supports text files, images, and PDFs".
Two related gaps in the same set:
- Video has the same problem:
mimeToModalityhandlesvideo/, butMEDIA_MIMESandMime.detectdo not. Mime.detectreturnsimage/bmpandimage/avif, which are not inMEDIA_MIMESeither, so those are rejected as binary too.
Plugins
none
OpenCode version
2.0.26
Steps to reproduce
- Ask the agent to read an audio file, e.g.
read ./clip.mp3, with a model whosecapabilities.inputincludesaudio. - The tool fails with
BinaryFileErrorinstead of presenting the file to the model.
Screenshot and/or share link
n/a
Operating System
Windows 11
Terminal
n/a (desktop app)
Activity
opencode-agent commentedon Oct 11, 2026
Thanks for the detailed write-up! You're asking for the
readtool to attach audio files (and also video, BMP and AVIF) when the model'scapabilities.inputsupports them. Right nowreadonly attaches PNG, JPEG, GIF, WebP and PDF. Everything else is detected as binary and rejected, even though the rest of the request pipeline already handlesaudio/andvideo/parts.That matches the tool's current design, which supports text, images and PDFs, so we're treating this as a feature request. Maintainers will consider it.
Related items:
readto handle.mp4(the video part of this)readdropping audio files for MiMo V2.5read(not merged)Te-River commentedon Oct 11, 2026
Cross-referencing the siblings, since they are one change in
read:.mp4), open and assigned.So the gap is one set:
MEDIA_MIMESinpackages/core/src/tool/read-filesystem.tsand the magic-byte sniffing inpackages/core/src/mime.tsboth stop at images and PDF, whilemimeToModalityinpackages/core/src/session/model-request.tsalready routesaudio/andvideo/parts. Fixing one without the other leaves the same class of file rejected.