Skip to content

Commit ad09340

Browse files
fix(opencode): advertise per-model image capabilities in the exported config (#4300)
The OpenCode export emitted only `name` and `limit` per model, so opencode computed `capabilities.attachment` and `capabilities.input.image` as false for every opencodex model: the provider is absent from models.dev, and opencode's loader falls back to a hardcoded false for an entry that says nothing. Attachments were then refused client-side in the TUI before any request reached the proxy — including native OpenAI slugs that /api/models reports as ["text","image"], and text-only models the vision sidecar covers. Carry the catalog row's `inputModalities` through `OpencodeCatalogModel` and `opencodeCatalogFromProxyRows`, then serialize them as opencode's own per-model fields (`attachment`, `modalities`) as declared by opencode's published model schema. Both provider generations get them, so the two spellings of one model list cannot disagree; the V2 model schema expresses capabilities as `capabilities.{tools,input,output}` (which opencode fills by migrating this same `modalities` field) and its loader decodes with `onExcessProperty: "ignore"`. A row that declares nothing keeps the previous entry shape, which opencode already treats as text-only. Values outside opencode's enum (text|audio|image|video|pdf) are dropped rather than written through, the way `audio` had to be for Pi and Gajae; a row left with nothing acceptable keeps its entry without capability keys instead of being retyped as text. `exportModelsFromProxyRows` no longer re-joins modalities by `namespaced` — the catalog entry carries them now, from the same visibility-filtered row as the model itself, so a disabled duplicate cannot donate them. Closes #4286 Co-authored-by: Valerio Coltre <[email protected]>
1 parent cb7f96c commit ad09340

10 files changed

Lines changed: 275 additions & 27 deletions

File tree

‎docs-site/src/content/docs/guides/opencode.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,34 @@ No model-level default effort is written. The proxy keeps applying its own confi
5151
default whenever a request carries no effort, so a default you change in opencodex stays
5252
in force instead of being frozen into the config.
5353

54+
## Images and attachments
55+
56+
opencode decides whether a model takes an image from the model entry itself, and it cannot ask
57+
models.dev about `opencodex` — this provider is not there. The generated blocks therefore
58+
carry opencode's own per-model capability fields, `attachment` and `modalities`, taken from
59+
the metadata the proxy reports at `GET /api/models`:
60+
61+
```json
62+
"gpt-5.6-luna": {
63+
"name": "gpt-5.6-luna (native)",
64+
"limit": { "context": 272000, "output": 32000 },
65+
"attachment": true,
66+
"modalities": { "input": ["text", "image"], "output": ["text"] }
67+
}
68+
```
69+
70+
Without those fields opencode assumes the model is text-only and refuses the paste on the
71+
client side, so the image never reaches the proxy. That applies to text-only models too: when
72+
the catalog reports image input for a model the vision sidecar covers, opencode lets the
73+
attachment through so the sidecar can describe it before the upstream call.
74+
75+
What is written comes from the row's declared input modalities in `GET /api/models`. For a
76+
discovered model the catalog adds `image` itself for the sidecar case; a custom row is written
77+
from the modalities stored on it, so a custom entry that declares text only stays text-only
78+
even when the sidecar would cover it. A row that declares none — an undeclared custom model,
79+
for example — keeps the plain entry (`name`, plus `limit` when its context window is known),
80+
and opencode treats it as text-only.
81+
5482
## Your own config is never modified
5583

5684
The launcher does not copy or rewrite `~/.config/opencode/opencode.json`,

‎src/cli/export-command.ts‎

Lines changed: 7 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -60,16 +60,6 @@ export interface ExportCommandDeps extends RuntimeApiDeps {
6060
configImpl?: () => OcxConfig;
6161
}
6262

63-
/**
64-
* `/api/models` row plus the modality list Pi consumes. The launcher's row type predates
65-
* the Pi exporter and stops at the fields OpenCode needs.
66-
*/
67-
type ExportProxyModelRow = OpencodeProxyModelRow & {
68-
inputModalities?: string[];
69-
reasoningEfforts?: string[];
70-
defaultReasoningEffort?: string;
71-
};
72-
7363
/**
7464
* Export rows from proxy `/api/models` rows.
7565
*
@@ -80,20 +70,13 @@ type ExportProxyModelRow = OpencodeProxyModelRow & {
8070
* row as the model itself: a second lookup over the raw rows would let a hidden or disabled
8171
* duplicate donate its ladder to the visible entry.
8272
*
83-
* Only modalities are re-joined by `namespaced`, because the catalog type does not carry them.
73+
* Modalities need no such lookup: `opencodeCatalogFromProxyRows` carries them on the catalog
74+
* entry, so the clients that filter them are handed the same filtered, deduped row.
8475
*/
8576
export function exportModelsFromProxyRows(
86-
rows: readonly ExportProxyModelRow[],
77+
rows: readonly OpencodeProxyModelRow[],
8778
config: OcxConfig,
8879
): ExportModel[] {
89-
const modalities = new Map<string, string[]>();
90-
for (const row of rows) {
91-
const namespaced = row.namespaced?.trim();
92-
if (!namespaced || modalities.has(namespaced)) continue;
93-
if (Array.isArray(row.inputModalities) && row.inputModalities.length > 0) {
94-
modalities.set(namespaced, [...row.inputModalities]);
95-
}
96-
}
9780
return opencodeCatalogFromProxyRows(rows, config).map(entry => {
9881
const model: ExportModel = {
9982
namespaced: entry.namespaced,
@@ -108,8 +91,9 @@ export function exportModelsFromProxyRows(
10891
model.reasoningEfforts = [...entry.reasoningEfforts];
10992
}
11093
if (entry.defaultReasoningEffort) model.defaultReasoningEffort = entry.defaultReasoningEffort;
111-
const input = modalities.get(entry.namespaced);
112-
if (input) model.inputModalities = [...input];
94+
if (entry.inputModalities && entry.inputModalities.length > 0) {
95+
model.inputModalities = [...entry.inputModalities];
96+
}
11397
return model;
11498
});
11599
}
@@ -188,7 +172,7 @@ export async function handleExportCommand(argv: string[], deps: ExportCommandDep
188172
}
189173
built = { document: exported.config, text: exported.text };
190174
} else {
191-
const rows = await runtimeRequest<ExportProxyModelRow[]>("/api/models", {}, { ...deps, baseUrl: root });
175+
const rows = await runtimeRequest<OpencodeProxyModelRow[]>("/api/models", {}, { ...deps, baseUrl: root });
192176
if (!Array.isArray(rows)) {
193177
throw new RuntimeApiError("Management API returned an unexpected /api/models payload.", 502, rows);
194178
}

‎src/cli/opencode.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,8 @@ export interface OpencodeProxyModelRow {
9898
displayName?: string;
9999
displayNameSource?: "operator" | "provider" | "fallback";
100100
contextWindow?: number;
101+
/** Declared input modalities from `/api/models`; carried into opencode model capabilities. */
102+
inputModalities?: string[];
101103
/** Declared effort ladder from `/api/models`; carried into opencode model variants. */
102104
reasoningEfforts?: string[];
103105
/** Declared default effort from `/api/models`. */
@@ -396,6 +398,9 @@ export function opencodeCatalogFromProxyRows(
396398
id: row.id,
397399
contextWindow: row.contextWindow,
398400
displayName: row.displayNameSource === "fallback" ? undefined : row.displayName,
401+
...(Array.isArray(row.inputModalities) && row.inputModalities.length > 0
402+
? { inputModalities: [...row.inputModalities] }
403+
: {}),
399404
...(typeof row.fastRowAvailable === "boolean" ? { fastRowAvailable: row.fastRowAvailable } : {}),
400405
...(Array.isArray(row.reasoningEfforts) && row.reasoningEfforts.length > 0
401406
? { reasoningEfforts: [...row.reasoningEfforts] }

‎src/clients/config-export.ts‎

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ export { buildRaycastClientConfig, summarizeRaycast, buildRaycastContribution }
4242

4343
import type { OpencodeLaunchEnv, OpencodeCatalogModel, ExportContext, PiModelEntry, ManagedContribution, ManagedFragment, ExportClientId, ExportClientSpec } from "./config-export/contracts";
4444
import { OPENCODE_API_KEY_ENV_REF, OPENCODE_PROVIDER_BLOCK_DEFAULT_CONFIG, OPENCODE_CONFIG_SCHEMA, OPENCODE_PROVIDER_ID, PI_API_DIALECT, LOOPBACK_API_KEY_PLACEHOLDER, HERMES_API_KEY_ENV_REF, OPENCLAW_API_KEY_ENV_REF, GAJAE_API_KEY_ENV, OPENCODE_API_KEY_ENV, HERMES_API_KEY_ENV, OPENCLAW_API_KEY_ENV } from "./config-export/constants";
45-
import { exportModelLabel, authoritativeContextWindow, outputBudgetFor, normalizeExportModels, inputModalitiesForClient, proxyAdmissionHeaders, singleFragment } from "./config-export/model-metadata";
45+
import { exportModelLabel, authoritativeContextWindow, outputBudgetFor, normalizeExportModels, inputModalitiesForClient, opencodeModelCapabilities, proxyAdmissionHeaders, singleFragment } from "./config-export/model-metadata";
4646
import { buildOmpClientConfig, summarizeOmp, buildOmpContribution } from "./config-export/omp";
4747
import { buildDshClientConfig, summarizeDsh, buildDshContribution } from "./config-export/dsh";
4848
import { buildMcodeClientConfig, summarizeMcode, buildMcodeContribution } from "./config-export/mcode";
@@ -54,6 +54,14 @@ import { buildRaycastClientConfig, summarizeRaycast, buildRaycastContribution }
5454
export interface OpencodeModelEntry {
5555
name: string;
5656
limit?: { context: number; output: number };
57+
/**
58+
* opencode's own capability fields, derived from the catalog row's declared input
59+
* modalities. Written only when the row declares at least one — an entry without them is
60+
* what opencode already treats as text-only, and omitting them keeps an undeclared model
61+
* byte-identical to what shipped before.
62+
*/
63+
attachment?: boolean;
64+
modalities?: { input: string[]; output: string[] };
5765
}
5866

5967
/**
@@ -659,13 +667,30 @@ export function opencodeProviderBlocks(
659667
if (context !== undefined) {
660668
entry.limit = { context, output: outputBudgetFor(context) };
661669
}
670+
// `attachment` / `modalities` are fields of opencode's V1 model schema — the shape its
671+
// published config.json defines and the one its loader reads (verified against opencode
672+
// 1.18.30, src/provider/provider.ts: `model.attachment ?? …` / `model.modalities?.input`).
673+
// They ride on both generations anyway: the two blocks are two spellings of one model list,
674+
// and the V2 model schema (capabilities.{tools,input,output}, which opencode fills by
675+
// migrating this same `modalities` field) ignores keys it does not define — its loader
676+
// decodes with `onExcessProperty: "ignore"`. Same values on both, so a merge cannot make
677+
// the two entries disagree.
678+
const capabilities = opencodeModelCapabilities(model.inputModalities);
679+
if (capabilities) {
680+
entry.attachment = capabilities.attachment;
681+
entry.modalities = capabilities.modalities;
682+
}
662683
v1Models[key] = entry;
663684
const variants = opencodeEffortVariants(model);
664-
// Own `limit` object, not a shared reference: the two blocks are serialized and reasoned
665-
// about separately, and an in-place edit of one must never move the other.
685+
// Own `limit` and `modalities` objects, not shared references: the two blocks are
686+
// serialized and reasoned about separately, and an in-place edit of one must never move
687+
// the other.
666688
v2Models[key] = {
667689
...entry,
668690
...(entry.limit ? { limit: { ...entry.limit } } : {}),
691+
...(entry.modalities
692+
? { modalities: { input: [...entry.modalities.input], output: [...entry.modalities.output] } }
693+
: {}),
669694
...(variants ? { variants } : {}),
670695
};
671696
}

‎src/clients/config-export/contracts.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,13 @@ export interface OpencodeCatalogModel {
3838
id?: string;
3939
contextWindow?: number;
4040
displayName?: string;
41+
/**
42+
* Declared input modalities, carried verbatim from `/api/models`. Serialized as opencode's
43+
* per-model `attachment` + `modalities`, because opencode gates attachments CLIENT-side:
44+
* without them every `opencodex` model is text-only in its picker and an image never
45+
* reaches the proxy or the vision sidecar (#4286).
46+
*/
47+
inputModalities?: readonly string[];
4148
/** Declared effort ladder. Exported as opencode model variants where the client reads them. */
4249
reasoningEfforts?: readonly string[];
4350
/**

‎src/clients/config-export/model-metadata.ts‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,39 @@ export function inputModalitiesForClient(
7272
return kept.length > 0 ? kept : null;
7373
}
7474

75+
/**
76+
* Input modalities opencode's model schema accepts (opencode.ai/config.json, both
77+
* `modalities.input` and `modalities.output`). Wider than our internal `text | image | audio`
78+
* vocabulary, so unlike Pi and Gajae this filter can only drop a value no current ingress
79+
* produces: `/api/custom-models`, `ocx models add` and the catalog writer all normalize to
80+
* the internal three. It exists so a future ingress cannot do to opencode what `audio` did
81+
* to Gajae, whose loader rejected the whole config file over one out-of-enum value.
82+
*/
83+
const OPENCODE_INPUT_MODALITIES: ReadonlySet<string> = new Set(["text", "audio", "image", "video", "pdf"]);
84+
85+
/**
86+
* opencode's per-model capability fields for one catalog row, or `undefined` when the row
87+
* declares nothing.
88+
*
89+
* `undefined` rather than `{ input: ["text"] }`: opencode already computes an entry without
90+
* capabilities as text-only, and leaving the keys out keeps every model that declares
91+
* nothing byte-identical to what shipped before. A declared list is carried across as-is, so
92+
* an audio-only row keeps `attachment: true` instead of being rewritten to text it cannot
93+
* read — the same call Pi's exporter makes, in the opposite direction.
94+
*/
95+
export function opencodeModelCapabilities(
96+
modalities: readonly string[] | undefined,
97+
): { attachment: boolean; modalities: { input: string[]; output: string[] } } | undefined {
98+
const input: string[] = [];
99+
for (const value of modalities ?? []) {
100+
if (OPENCODE_INPUT_MODALITIES.has(value) && !input.includes(value)) input.push(value);
101+
}
102+
if (input.length === 0) return undefined;
103+
// `attachment` is what opencode's client gates pasting on; `modalities` refines it into
104+
// which kinds. Output is always text — nothing in the catalog declares otherwise.
105+
return { attachment: input.some(value => value !== "text"), modalities: { input, output: ["text"] } };
106+
}
107+
75108
/**
76109
* Label shared by every client: `"<displayName|id> (<native|provider|routed>)"`. The
77110
* provider suffix is what makes two same-named models from different upstreams

‎tests/clients/client-export-modality-enum.test.ts‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import {
66
type ExportModel,
77
type GajaeGeneratedConfig,
88
type HermesGeneratedConfig,
9+
type OpencodeGeneratedConfig,
910
type PiGeneratedConfig,
1011
} from "../../src/clients/config-export";
1112
import type { OcxConfig } from "../../src/types";
@@ -52,6 +53,11 @@ function hermesModels(models: ExportModel[]) {
5253
.providers[OPENCODE_PROVIDER_ID].models;
5354
}
5455

56+
function opencodeModels(models: ExportModel[]) {
57+
return (buildClientConfig("opencode", ctx(models)) as OpencodeGeneratedConfig)
58+
.providers[OPENCODE_PROVIDER_ID].models;
59+
}
60+
5561
/** The live failure, by its real id and real modality list. */
5662
const MIXED: ExportModel = {
5763
namespaced: "zenmux/meta-muse-spark-1.1",
@@ -147,3 +153,94 @@ describe("exported modalities stay inside the enum each client accepts", () => {
147153
}
148154
});
149155
});
156+
157+
/**
158+
* opencode is the third shape of this problem, and the only one where the fix is a
159+
* capability field rather than a filter.
160+
*
161+
* Its model schema accepts a WIDER enum than our internal vocabulary
162+
* (`text | audio | image | video | pdf`, opencode.ai/config.json), and its client gates
163+
* pasting on `attachment` / `modalities.input` INSTEAD of rejecting the file we hand it. So
164+
* an out-of-enum value is dropped, but a row left with nothing acceptable keeps its entry
165+
* and carries no capability keys — never a fabricated `text`, which would advertise input
166+
* the model cannot read.
167+
*/
168+
describe("opencode receives the capability fields its client gates attachments on", () => {
169+
test("a declared model advertises attachment plus every modality opencode accepts", () => {
170+
// The live catalog shape: meta-muse-spark-1.1 declares text|image|audio, and audio is
171+
// INSIDE opencode's enum, so unlike Pi and Gajae nothing is dropped here.
172+
expect(opencodeModels([MIXED])["zenmux/meta-muse-spark-1.1"]).toEqual({
173+
name: "meta-muse-spark-1.1 (zenmux)",
174+
limit: { context: 1_048_576, output: 32_000 },
175+
attachment: true,
176+
modalities: { input: ["text", "image", "audio"], output: ["text"] },
177+
});
178+
});
179+
180+
test("an audio-only row stays audio-only instead of being retyped as text", () => {
181+
// opencode accepts audio, so the Pi/Gajae answer — omit the row — would lose a model for
182+
// no reason. Faithfulness costs nothing here.
183+
expect(opencodeModels([AUDIO_ONLY])["p/audio-only"]).toEqual({
184+
name: "audio-only (p)",
185+
attachment: true,
186+
modalities: { input: ["audio"], output: ["text"] },
187+
});
188+
});
189+
190+
test("a text-only declaration is advertised as text-only rather than omitted", () => {
191+
const textOnly: ExportModel = { namespaced: "p/text", provider: "p", id: "text", inputModalities: ["text"] };
192+
expect(opencodeModels([textOnly])["p/text"]).toEqual({
193+
name: "text (p)",
194+
attachment: false,
195+
modalities: { input: ["text"], output: ["text"] },
196+
});
197+
});
198+
199+
test("a row that declares nothing carries no capability keys at all", () => {
200+
// Not the same as `{ input: ["text"] }`: opencode already falls back to text-only for an
201+
// entry without capabilities, and the omission keeps the pre-#4286 bytes for every model
202+
// whose row says nothing.
203+
const bare: ExportModel = { namespaced: "p/bare", provider: "p", id: "bare" };
204+
const empty: ExportModel = { ...bare, namespaced: "p/empty", id: "empty", inputModalities: [] };
205+
const models = opencodeModels([bare, empty]);
206+
expect(models["p/bare"]).toEqual({ name: "bare (p)" });
207+
expect(models["p/empty"]).toEqual({ name: "empty (p)" });
208+
});
209+
210+
test("an out-of-enum value is dropped and duplicates collapse", () => {
211+
const odd: ExportModel = {
212+
namespaced: "p/odd", provider: "p", id: "odd", inputModalities: ["file", "image", "image"],
213+
};
214+
expect(opencodeModels([odd])["p/odd"]).toEqual({
215+
name: "odd (p)",
216+
attachment: true,
217+
modalities: { input: ["image"], output: ["text"] },
218+
});
219+
});
220+
221+
test("a model whose only declaration is out of enum keeps its entry, without capabilities", () => {
222+
const foreign: ExportModel = { namespaced: "p/foreign", provider: "p", id: "foreign", inputModalities: ["file"] };
223+
expect(opencodeModels([foreign])["p/foreign"]).toEqual({ name: "foreign (p)" });
224+
});
225+
226+
test("no emitted entry in a whole catalog carries a value opencode rejects", () => {
227+
const catalog: ExportModel[] = [
228+
MIXED,
229+
AUDIO_ONLY,
230+
{ namespaced: "p/bare", provider: "p", id: "bare" },
231+
{ namespaced: "p/foreign", provider: "p", id: "foreign", inputModalities: ["file"] },
232+
{ namespaced: "p/vision", provider: "p", id: "vision", inputModalities: ["text", "image"] },
233+
];
234+
const models = opencodeModels(catalog);
235+
// The entry survives where Pi and Gajae would have dropped it; only its bad value goes.
236+
expect(Object.keys(models)).toContain("p/foreign");
237+
for (const entry of Object.values(models)) {
238+
for (const value of entry.modalities?.input ?? []) {
239+
expect(["text", "audio", "image", "video", "pdf"]).toContain(value);
240+
}
241+
for (const value of entry.modalities?.output ?? []) {
242+
expect(["text", "audio", "image", "video", "pdf"]).toContain(value);
243+
}
244+
}
245+
});
246+
});

0 commit comments

Comments
 (0)