[FEATURE]: Native Multimodal Context Support (Video/Audio)

Feature hasn't been suggested before.

I have verified this feature I'm about to request hasn't been suggested before.

Describe the enhancement you want to request

Hi Team,

I recently moved to opencode from claude code and I'm liking it way of handling multiple models and working with it.

I am looking to bring the True Multimodal capabilities found in environments like Gemini AI Studio into the OpenCode workflow. Currently, handling video based tasks requires manual transcription or frame extraction, which adds significant friction. Also it can't understand audio in it.

Use Case example: A developer records a 30 second bug reproduction. They attach the video to OpenCode and ask: "Analyze this UI glitch and find the relevant CSS class in my project that might be causing the flickering."

This issue might be a duplicate of existing issues. Please check:

[DOCS] Missing documentation for modalities property in custom provider configuration #9897: [DOCS] Missing documentation for modalities property in custom provider configuration - the infrastructure for multimodal support already exists but lacks documentation
[FEATURE]: Speech-to-Text Voice Input for Lazy People in OpenCode #4695: [FEATURE]: Speech-to-Text Voice Input for Lazy People in OpenCode - addresses audio input component
My opencode app cannot read files #9356: My opencode app cannot read files - PDF file reading issues
Read tool throws error when reading pdf files #10294: Read tool throws error when reading pdf files - PDF support incomplete
[BUG] ai is saying that it can't read a pdf. but the model is support it. #9825: [BUG] ai is saying that it can't read a pdf - model supports it but OpenCode doesn't leverage it
Working with REST & media data? [correct approaches?] #6665: Working with REST & media data? - documents the friction with handling non-text content
Add n1n.ai API as a model provider #3178: Add n1n.ai API as a model provider - requests providers with native multimodal capabilities

Feel free to ignore if none of these address your specific case.

Also on my wish list of features. I'd like to read PDFs and docx that contain images without having to call and wait on a parsing API.

Also be nice to load in mp3 and mp4 files for audio transcription.

Workarounds exist to call external APIs and tools, but the innate capabilities of models cannot be fully used in OpenCode.

Gemini can transcribe audio, but not in OpenCode.

+1
Tool calling and image understanding are both intrinsic capabilities of Kimi 2.5. However, even after integrating Kimi 2.5, OpenCode still cannot proactively invoke its multimodal features.

+1 from me
Being able to upload videos would be a game changer for models that support it like kimi k2.5 which is really good as visual coding
Tho idk if the openai or anthropic compatiable API for kimi for coding supports uploading videos...

if its okey i would like to tackle this feature myself since based on the api docs of kimi they extended the openai schema to accept video input similar to how gemini does. I would like to experiment on this and make a patch as long as all agree on this. i would probably first either start with kimi or gemini tho gemini should be pretty easy

Update:
from the looks of it it seems pretty easy since kimi cli already implements native video so yeah ill just implement that

+1
Yeah,if we could use gemini 3.1 pro to understand videos and generate websites from them,that would be amazing.

+1 Tool calling and image understanding are both intrinsic capabilities of Kimi 2.5. However, even after integrating Kimi 2.5, OpenCode still cannot proactively invoke its multimodal features.

+1 here. would be great for video understanding

google-gemini/gemini-cli#1556 gemini CLI supports it

same thing with kimi-cli it also supports video: https://github.com/MoonshotAI/kimi-cli

To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.

Think this would be great to have...

+1 need this feature

"Closed as not planned" aight bro lol

Navigation Menu

Repository navigation

[FEATURE]: Native Multimodal Context Support (Video/Audio) #10531

Feature hasn't been suggested before.

Describe the enhancement you want to request

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[FEATURE]: Native Multimodal Context Support (Video/Audio) #10531

Description

Feature hasn't been suggested before.

Describe the enhancement you want to request

Activity

github-actions commented on Jan 25, 2026

NamedIdentity commented on Jan 30, 2026

LCorleone commented on Feb 6, 2026

HadiCherkaoui commented on Feb 9, 2026

HadiCherkaoui commented on Feb 9, 2026

pgd-LC2 commented on Feb 20, 2026

markwitt1 commented on Feb 26, 2026

markwitt1 commented on Feb 26, 2026

HadiCherkaoui commented on Feb 26, 2026

github-actions commented on Apr 28, 2026

shirschfield commented on Apr 29, 2026

cwyalpha commented on May 6, 2026

butterbrodskiy commented on Aug 20, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions