Repository navigation
Feat/image model switch - #26164
Feat/image model switch#26164SOUMITRO-SAHA wants to merge 7 commits into
Conversation
…t model lacks vision support When the active chat model doesn't support images, automatically uses a dedicated image-capable model to analyze and replace image parts with text descriptions before sending to the chat model. Adds image model picker to TUI command palette with persistent selection, CLI --image-model flag for the run command, and config set/show debug subcommands.
…spinner Display the selected image model name beside the current chat model in the prompt metadata row. Show spinner with '<model> is analyzing image...' in the status bar while images are being preprocessed.
|
Hey! Your PR title Please update it to start with one of:
Where See CONTRIBUTING.md for details. |
|
The following comment was made by an LLM, it may be inaccurate: I found a potential duplicate: Related PR:
The current PR #26164 appears to be an enhanced or refactored version of the work done in #24382, with added features like the command palette switch for image models and the |
Add logging and error recovery for image preprocessing failures Update SDK types to include optional imageModel configuration
|
Hi @SOUMITRO-SAHA! This is a really useful feature - many of us are eagerly waiting for it. Two suggestions to help this PR move forward:
Let me know if you need help with anything! |
- Fix image-preprocess.ts: replace old imports with current codebase patterns (SessionV1 types, Effect.logInfo/Error/Warning, ProviderV2/ModelV2 IDs) - Add image_model config option to ConfigV1.Info schema - Add getImageModel to Provider.Interface type and fix ProviderV2.ID usage - Fix prompt.ts: replace MessageV2.TextPart with SessionV1.TextPart, replace log.error with Effect.logError - Fix TUI dialog-image-model.tsx: use relative imports instead of @tui/ - Fix TUI app.tsx: add missing DialogImageModel import - Add image_processing variant to SDK SessionStatus type - Fix debug/config.ts: add missing imports for Config, CliError - Add image-preprocess.test.ts with 4 unit tests for skip conditions
|
This branch has been upgraded to V2. We're no longer taking PRs for V1. If you think this is still relevant, please port it over and open it for V2. — from 𝕺𝖕𝖊𝖓𝕮𝖔𝖉𝖊 |
|
Ported to V2 as requested: #62534 The V2 port keeps the core idea — images sent to a vision-less model are described by an image-capable model and the description replaces the image — but follows V2's architecture instead of V1's prompt-time part rewriting:
The TUI image-model picker from this PR isn't ported yet since V2 selects models server-side; happy to follow up with a picker if the core lands. |
Issue for this PR
Closes #26160
Type of change
What does this PR do?
Auto Image Read enables OpenCode to handle images even when the user's current model doesn't support image input. It automatically routes images to a dedicated image-capable model for analysis, converts the result to text, and continues the conversation with the original model.
How It Works
preprocessImages()Agent.PROMPT_IMAGE_DESCRIBEto analyze the imageImage Model Selection
Users can set their image model in two ways:
Ctrl+P→ typeSwitch Image Modelorsim/image-modelsResolution Chain
How did you verify your code works?
Screenshots / recordings
Checklist
If you do not follow this template your PR will be automatically rejected.