Support image input for text-only models via vision proxy middleware #36841

The problem

When you configure a text-only model like GLM-5.2 as the primary model, pasting an image into the chat just fails. The model can't handle it, the image gets rejected before any agent logic runs, and there's no fallback. You're stuck — either switch to a different model entirely, or tell the user "sorry, I can't see images."

But the thing is, the fix doesn't have to be complicated. A lot of text-only models (GLM-5.2, DeepSeek-R1, Qwen3, Kimi K2) are genuinely the best choice for the coding/reasoning task at hand. You don't want to ditch them just because a screenshot showed up.

What I'd like to see

A config option that lets you specify a "vision proxy" — a secondary model that handles image interpretation when the primary model is text-only. Something like:

{
  "provider": {
    "openrouter": {
      "options": {
        "apiKey": "..."
      },
      "models": {
        "z-ai/glm-5.2": {
          "name": "GLM-5.2"
        }
      },
      "vision_proxy": {
        "model": "google/gemini-2.5-flash",
        "prompt": "Describe this image in detail for a text-only AI model. Include all visible text, layout, colours, and any error messages."
      }
    }
  }
}

When the user pastes an image:

1. opencode intercepts it before sending to the primary model
2. Sends it to the vision proxy model instead (with the configured prompt)
3. Gets back a text description
4. Passes that text description to the primary model as part of the user's message
5. The primary model never sees the raw image — it just gets text, which it can handle

The user experience should be seamless — drop a screenshot in, get an answer back, no manual steps.

Why this matters

Text-only models are often picked on purpose — they're cheaper, faster, have longer context windows, or are better at specific tasks. Forcing a model switch just to handle an occasional screenshot is a poor trade-off. A vision proxy lets you keep your primary model and handle images when they come up, at a cost of a fraction of a cent per image (Gemini-2.5-flash is $0.075/$0.30 per 1M tokens).

What I've tested

I've verified the two-step pipeline manually:

1. Image to OpenRouter + GPT-4o to text description (works, ~239 input tokens)
2. Image to OpenRouter + Gemini-2.5-flash to text description (works, ~1049 input tokens, cheaper)
3. Text description to GLM-5.2 to analysis (works, GLM-5.2 reasons about the description correctly)

The pipeline works end to end. The gap is just that opencode doesn't do this automatically — it rejects the image at the input layer before the agent gets a chance to proxy it.

Alternatives considered

• Sub-agent pattern: the agent could spawn a sub-agent that calls the vision API via bash and returns the description. This works if the image is saved to a file, but the agent can't access images pasted directly into the chat — they're rejected before the agent sees them.
• Manual workflow: user saves the image to a file, tells the agent the path, agent calls the vision API. Works but is clunky and defeats the "drop it in the chat" UX.

Environment

• opencode v1.17.20
• Windows
• Primary model: z-ai/glm-5.2 (text-only, via OpenRouter)
• Tested vision models: openai/gpt-4o, google/gemini-2.5-flash (both via OpenRouter, both work)