Repository navigation
Description
Feature hasn't been suggested before.
- I have verified this feature I'm about to request hasn't been suggested before.
Describe the enhancement you want to request
Problem
When using an OpenAI-compatible provider (e.g., Manifest, LocalAI, LM Studio, AnyScale) with a vision-capable model (Kimi 2.5, GPT-4o, Claude), image inputs are silently stripped before the API call is made. The model receives a text string: "ERROR: Cannot read image (this model does not support image input). Inform the user." — which is fabricated by the framework's unsupportedParts() check. The actual image_url content never reaches the provider.
Root Cause
@ai-sdk/openai-compatible hardcodes input: ["text"] for models from /v1/models responses since the OpenAI-compatible schema doesn't include modality metadata. When a user sends a message with image content, the framework checks model.capabilities.input.image — finds false — and strips the image.
Workaround (Confirmed Working in v1.15.12)
The modalities property on model definitions tells the framework the model accepts image inputs:
{
"provider": {
"manifest": {
"npm": "@ai-sdk/openai-compatible",
"options": {
"baseURL": "http://localhost:2099/v1"
},
"models": {
"static-image": {
"name": "static-image",
"headers": { "x-manifest-tier": "StaticImage" },
"modalities": {
"input": ["text", "image"],
"output": ["text"]
}
}
}
}
}
}
After adding "modalities": { "input": ["text", "image"], ... }, the multimodal-looker subagent successfully reads and describes screenshots from Playwright.
Why This Needs Fixing
- Undocumented — The modalities property exists in packages/opencode/src/provider/models.ts but isn't in the config schema docs. Users have no way to discover it.
- Bad defaults — Auto-discovered OpenAI-compatible models default to input: ["text"], which silently breaks vision for any model that actually supports images.
- Others fixed this — KiloCode (PR Disable force dependency update while starting #8729) had the same issue and fixed it by defaulting input.image and attachment to true for auto-discovered models. OpenCode should do the same, or at minimum document the modalities escape hatch.
Suggested Fixes
- Default input: ["text", "image"] for auto-discovered OpenAI-compatible models — matches what KiloCode did. Most OpenAI-compatible providers expose at least one vision-capable model, and the worst case (model doesn't support images) is a graceful API error, not silent data loss.
- Document modalities in the provider config schema — add it to opencode.ai/config.json and the docs so users can opt in per-model.
- Allow /v1/models capabilities passthrough — if a provider's /v1/models response includes any capability fields, forward them to the AI SDK.
Activity
github-actions commentedon May 31, 2026
This issue might be a duplicate of existing issues. Please check:
If your issue is distinct from all of the above, please clarify the differences.
dlasher commentedon May 31, 2026
#22673 covers the same root cause (misclassification), but #30071 is a feature request for explicit modalities config support + documentation.
The "modalities": {"input": ["text", "image"]} field works as a current workaround but is undocumented — #30071 specifically requests adding it to the config schema docs.
If #22673 resolves the auto-classification, the modalities config won't be needed for most users, but documenting the field would still help with edge cases. #20802 and #26106 are related but distinct.
dlasher commentedon May 31, 2026
or you could close as duplicate of #22673 but cherry-pick the documentation request into a comment there
github-actions commentedon Jul 31, 2026
To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.