[FEATURE]: Transcribe images to text when the selected model has no vision support #48332

Activity

1. github-actions

   assigned

   kitlangton

   on Sep 10, 2026
2. github-actions commented on Sep 10, 2026

   

   github-actionsBot

   on Sep 10, 2026 – with GitHub Actions

   Contributor

   More actions

   This issue might be a duplicate of existing issues. Please check:

   • [FEATURE]: Auto-switch to vision model when images enter the agent loop (incl. tool-produced) #46794: [FEATURE]: Auto-switch to vision model when images enter the agent loop — same root problem (non-vision model receives an image), proposes switching the active model rather than transcribing
   • [FEATURE]: Add RapidOCR-json support — reading image text into the model when it can't see imagesI come from CHINA! I solution OPENCODE'S EYE problem, you can use RapidOCR-json, let bigpickle model (LLM) takes the picture's words!! #46785: [FEATURE]: Add RapidOCR-json support — reading image text when the model can't see images — proposes a local OCR fallback (RapidOCR) for the identical scenario
   • Attached images degrade to a bare text placeholder when the model lacks vision — agent has no way to read them #42758: Attached images degrade to a bare text placeholder when the model lacks vision — documents the same failure mode your proposal addresses

   Your issue is distinct in proposing a provider-constrained vision-model transcription step (same provider first, then local OCR), which is a more complete fallback strategy than either of the above individually. Linking these so the maintainers can decide whether to consolidate or treat yours as a separate proposal.
3. savasturkoglu1 commented on Sep 10, 2026

   

   savasturkoglu1

   on Sep 10, 2026

   Author

   More actions

   Thanks for linking these.

   • [FEATURE]: Auto-switch to vision model when images enter the agent loop (incl. tool-produced) #46794 proposes switching the active model. This issue deliberately keeps the selected model and only transcribes the image, so the model choice never changes.
   • The local OCR fallback is now split out into [FEATURE]: Local OCR fallback for images when no vision model is available #48333, so this issue is scoped to vision-model transcription only.
   • Attached images degrade to a bare text placeholder when the model lacks vision — agent has no way to read them #42758 is the same failure mode; this issue is the fix path for when a vision-capable model exists in the same provider.
4. morpheus9393 commented on Oct 10, 2026

   

   morpheus9393

   on Oct 10, 2026

   More actions

   Big +1 from us — we run text-only coding models (GLM-5.x / DeepSeek tiers) as daily drivers in production and this is the single biggest quality-of-life gap in OpenCode for that setup.

   Right now the practical workaround is an external plugin that proxies pasted images through a vision-capable model and swaps in the description. It works, but it has to be maintained against every plugin-API change, and the capability (knowing which models see images, and rewriting the message) is something the host already has all the information for — model capabilities are already tracked, the image pipeline is already normalized/hosted. Having this as a first-class option (e.g. a \�isionFallbackModel\ config, or a per-session toggle that keeps the selected model and only transcribes attachments) would remove an entire class of third-party plugin churn for us and, judging by the cluster of related issues here, for a lot of other developers running text-only models on coding plans too.

   For us this is not a nice-to-have: teammates regularly paste screenshots, mockups and error photos into chat, and the model just gets [Attached image/png: ...]. Anything the team can prioritize here would be genuinely appreciated — happy to test an implementation on real workloads.
5. opencodejr commented on Oct 28, 2026

   

   opencodejrBot

   on Oct 28, 2026

   Contributor

   More actions

   Thanks for the detailed proposal, @savasturkoglu1!

   To restate it: when the selected model can't read images, OpenCode would transcribe a pasted image to text with a vision-capable model and add that text to the request. That model would be experimental.vision_model if set, otherwise a vision model from the same provider, so the image never goes to a different provider. The selected model stays the same and no error is shown. Maintainers will look at it and decide.

   One note on scope: the error you quote (Cannot read "<file>" (this model does not support image input)) comes from the v2 session code (unsupportedParts() in packages/core/src/session/model-request.ts), and v2 is where new features are built now. A design or PR would need to target that path, not v1.

   It might also work as a plugin in v2. The session context hook lets plugins change the outgoing messages before unsupported images are replaced, so a plugin could swap image parts for transcribed text. That only works if the plugin can call a vision model itself.

   Related issues:

   • [FEATURE]: Auto-switch to vision model when images enter the agent loop (incl. tool-produced) #46794: auto-switching the active model to a vision model (broader than this)
   • Attached images degrade to a bare text placeholder when the model lacks vision — agent has no way to read them #42758: images turning into a placeholder for models without vision
   • [FEATURE]: Local OCR fallback for images when no vision model is available #48333: your local OCR follow-up