Skip to content

Commit 907d486

Browse files
mogulOpenCode Agent
andauthored
feat(usai-catalog): publish verified image input modality for 12 USAi models (#575)
🤖: AI agent (First Mate), AI-assisted change (OpenCode). Human review required before merge. > PR #578 merged; verified values are preserved by refresh (checked on the merged tree on 2026-11-04, head 9a1c5f6). ## Summary Publishes the verified image input modality for 12 USAi models: `inputModalities: ["text","image"]` in `integrations/providers/usai/catalog.json`, the regenerated neutral snapshot, and `sources.capabilitiesVerified: "2026-11-04; opencode 2.0.30"`. Adds the per-model evidence file required by ADR 0006 (isolation). Nothing else is published: no `reasoning` for any model, and the other 12 models stay absent (unknown). Base: origin/main at fcd94e5 (includes the verified-capability carry PR #578, and also the verification script PR #572 and the refresh entrypoint PR #573, both merged; this branch has been merged with main). ## Issues - #541 (this change) - #366 (parent) - #540 (the verification procedure and script) ## What is published (every model, field, value) Field `inputModalities`, value `["text","image"]`, for exactly these 12 models: `claude-sonnet-5.5`, `claude-sonnet-5`, `claude_4_8_opus`, `claude_4_7_opus`, `claude_4_6_sonnet`, `gpt-6.1-sol-latest-guardrails-defaultv2`, `gpt-6-astra`, `gpt_5_4_default_v2`, `gpt_5_2_default_v2`, `gemini-3.8-flash`, `gemini-3-7-flash`, `gemini-3.5-flash`. (The list was checked against `summary.qualifying` in the raw `results.json`: identical.) Evidence (full table and verbatim token replies in `integrations/isolation/acq-kits/opencode/docs/capability-evidence/2026-11-04-opencode-2.0.30.md`): - **Procedure:** synthetic-image procedure, live run through OpenCode 2.0.30 against the USAi gateway, on 2026-11-04. - **Samples:** 3 independently generated random-token images per model, per configuration. Configurations: `modalities.input` unset; explicit `["text","image"]`; explicit `["text"]` (negative control). - **Result for each of the 12:** exact token on 3 of 3 images with explicit `["text","image"]`. Text control passed. - **Control:** under explicit `["text"]`, 63 of 63 samples (21 models x 3) refused or did not return the token; 0 token leaks. - **`claude_4_7_opus`:** read 2 of 3 with the setting unset, 3 of 3 explicit. The decision rule counts the explicit configuration, so it qualifies. Called out in the evidence file. - **Script:** `verify_image_capability.py` v1.0.0. The evidence was produced by the pre-merge head `fa813f63` (with its live-run fixes), in the maintainer's sandbox. The script has since merged as `f0a951f3`; the merged version differs from `fa813f63`, so the evidence was **not** produced by the merged script. The evidence file says so. - Models tested but not published (9, misreads in at least one explicit text+image sample) and the 3 unavailable models (HTTP 403 `model_blocked_globally` x2; HTTP 400 no tool use in streaming mode) are listed with reasons in the evidence file, so a transient outage is not read as "unsupported". ## Narrowing (reviewer must accept) ADR 0006: a present `inputModalities` array asserts the listed modalities only, and the OpenCode mapping treats an unlisted modality as disabled. Publishing `["text","image"]` for these 12 models **disables `audio`, `video`, and `pdf` input for them in OpenCode**, though those were never probed and may work at the gateway. **Please explicitly accept this narrowing for each of the 12 models**, or ask for the field to stay absent for any of them. ## Before / After | | Before | After | |---|---|---| | `catalog.json` models with `inputModalities` | 0 | 12 | | `catalog.json` `sources.capabilitiesVerified` | absent | `2026-11-04; opencode 2.0.30` | | Neutral snapshot models with `inputModalities` | 0 | 12 (regenerated by `normalize-catalog.mjs`, equal to the normalizer output) | | `reasoning` on any model | none | none | | Rendered OpenCode config | no `modalities` | `modalities.input: ["text","image"]` for the 12 (checked with `renderProviderConfig`) | Files: `catalog.json` (+12 models, +1 source key), the snapshot (same), the evidence file (new), `tests/catalog-check.test.mjs` (one added assertion, see below), and `integrations/providers/usai/README.md` (a link to the evidence file; the earlier sentence that a refresh does not carry values forward was removed because #578 documents the carry-forward). Evidence files added beside the evidence file (all in `capability-evidence/`): `2026-11-04-opencode-2.0.30-replies.md` (step-1 config facts for all 24 models, text-control result, and the verbatim reply for every one of the 189 samples, including all 63 text-only control replies, capped at the stored 500 characters), and the run's sanitized `results.json` and `results.tsv` copied unchanged, with SHA-256 recorded. Run id `70f8e8da5746`, script commit `fa813f63` (merged version `f0a951f3`), OpenCode 2.0.30. The run console log is omitted because it adds nothing beyond `results.json`. **Provenance:** the positive-token table is author-reported evidence from a run the author executed; it is not independently verified. A human must accept the audio/video/pdf narrowing per model. ### Test change after merging main The earlier edit to `default --check uses the committed 24-model production catalog` is dropped. The refresh entrypoint PR removed the bootstrap fixture and bootstrap mode, so that test no longer exists on main; `--check` without a source now refuses. The conflict was resolved by taking main's version of `catalog-check.test.mjs` and adding one assertion to `the committed catalog is a valid catalog with at least one model`: the committed catalog still has exactly 24 models. Nothing else in that file differs from main. ## Verification | Check | Result | |---|---| | After merging main (`9a1c5f6`) | `make test-kits`: opencode 69, usai-provider 18, paseo 87, providers/usai 210: all pass | | `make validate`, `make validate-kits`, `make validate-provider-input-contract`, `make generate-check` | pass | | `PYTHONPATH=scripts pytest scripts/tests` | 827 passed | | `ruff check scripts`, `ruff format --check scripts` | pass | | `.github/linters/run-markdownlint.mjs` | pass | | Guard test for `sources.capabilitiesVerified` shape | pass (in providers/usai suite) | | Byte identity | `serialize(JSON.parse(catalog.json))` equals the committed bytes; snapshot equals fresh `normalize-catalog.mjs` output (`cmp`) | | Older consumers | the OpenCode renderer accepts the snapshot with capabilities present and renders `modalities.input` for exactly 12 models; renderer tests pass | | Prime Agent | `models-json-regen.test.mjs` still passes; the Prime Agent emitter does not carry these fields; **no Prime Agent file changed** | | Independent read-only AI review (blockers only) | NO BLOCKERS. Checked the 12 ids, every evidence row against `results.json`, canonical form, no Prime Agent change. One non-blocking comment wording fix applied in the second commit. | The data was written with the repo's own `serialize` from `build-catalog.mjs` (not hand edits and not `--include-capabilities`, which copies unverified upstream values). ## Before merge Confirm per ADR 0006 rollout order that consumers validating with a vendored copy of the neutral schema (Quickstart fixtures, any Prime Agent consumer) have adopted the schema that allows these fields. **Refresh interaction (verified on the merged tree, 2026-11-04):** #578 has merged, so a refresh carries these values forward (#576). On this branch merged with main (`fcd94e5e`): - `refresh.mjs --from tests/fixtures/refresh --check`: catalog and neutral snapshot "up to date", exit 0. - Write-mode refresh in a scratch copy: exit 0; all 12 `inputModalities`, the whole `models[]` array and `sources.capabilitiesVerified` byte-identical (only `sources` provenance paths differ, as expected for a fixture source); snapshot unchanged. - `build-catalog.mjs ... --include-capabilities` against the verified catalog: refused with exit 1, catalog not written. This PR changes no refresh behavior. `catalog.json` and the snapshot bytes are unchanged by the merge. ## Rollback Revert the commit. Models return to "unknown"; OpenCode 2.0.30 then defaults to text and image for them, so behavior on the pinned version is unchanged either way. ## Security Impact No change to authentication, authorization, data handling, or network surface. Data only. The evidence file holds synthetic random tokens and verbatim model replies; no credentials, no raw transcripts. No secret values were used in any request recorded here. Related: #357. --------- Co-authored-by: OpenCode Agent <[email protected]>
1 parent 1b24d7e commit 907d486

8 files changed

Lines changed: 3076 additions & 27 deletions

File tree

‎integrations/isolation/acq-kits/opencode/docs/capability-evidence/2026-11-04-opencode-2.0.30-replies.md‎

Lines changed: 428 additions & 0 deletions
Large diffs are not rendered by default.

‎integrations/isolation/acq-kits/opencode/docs/capability-evidence/2026-11-04-opencode-2.0.30-results.json‎

Lines changed: 2121 additions & 0 deletions
Large diffs are not rendered by default.

‎integrations/isolation/acq-kits/opencode/docs/capability-evidence/2026-11-04-opencode-2.0.30-results.tsv‎

Lines changed: 193 additions & 0 deletions
Large diffs are not rendered by default.

‎integrations/isolation/acq-kits/opencode/docs/capability-evidence/2026-11-04-opencode-2.0.30.md‎

Lines changed: 207 additions & 0 deletions
Large diffs are not rendered by default.

‎integrations/isolation/acq-kits/usai-provider/files/home/usai-config/model-catalog-snapshot.json‎

Lines changed: 60 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,11 @@
1212
"output": 10,
1313
"cacheRead": 0.1,
1414
"cacheWrite": 2.5
15-
}
15+
},
16+
"inputModalities": [
17+
"text",
18+
"image"
19+
]
1620
},
1721
{
1822
"id": "claude-opus-5",
@@ -36,7 +40,11 @@
3640
"output": 10,
3741
"cacheRead": 0.2,
3842
"cacheWrite": 2.5
39-
}
43+
},
44+
"inputModalities": [
45+
"text",
46+
"image"
47+
]
4048
},
4149
{
4250
"id": "claude_4_8_opus",
@@ -48,7 +56,11 @@
4856
"output": 25,
4957
"cacheRead": 0.5,
5058
"cacheWrite": 6.25
51-
}
59+
},
60+
"inputModalities": [
61+
"text",
62+
"image"
63+
]
5264
},
5365
{
5466
"id": "claude_4_7_opus",
@@ -60,7 +72,11 @@
6072
"output": 25,
6173
"cacheRead": 0.5,
6274
"cacheWrite": 6.25
63-
}
75+
},
76+
"inputModalities": [
77+
"text",
78+
"image"
79+
]
6480
},
6581
{
6682
"id": "claude_4_6_sonnet",
@@ -72,7 +88,11 @@
7288
"output": 16.5,
7389
"cacheRead": 0.33,
7490
"cacheWrite": 4.125
75-
}
91+
},
92+
"inputModalities": [
93+
"text",
94+
"image"
95+
]
7696
},
7797
{
7898
"id": "claude_4_5_haiku",
@@ -114,7 +134,11 @@
114134
"output": 15,
115135
"cacheRead": 0.2,
116136
"cacheWrite": 5
117-
}
137+
},
138+
"inputModalities": [
139+
"text",
140+
"image"
141+
]
118142
},
119143
{
120144
"id": "gpt-6-astra",
@@ -132,7 +156,11 @@
132156
"output": 75,
133157
"cacheRead": 2,
134158
"cacheWrite": 25
135-
}
159+
},
160+
"inputModalities": [
161+
"text",
162+
"image"
163+
]
136164
},
137165
{
138166
"id": "gpt-6-luna",
@@ -254,7 +282,11 @@
254282
"input": 5,
255283
"output": 22.5,
256284
"cacheRead": 0.5
257-
}
285+
},
286+
"inputModalities": [
287+
"text",
288+
"image"
289+
]
258290
},
259291
{
260292
"id": "gpt_5_2_default_v2",
@@ -265,7 +297,11 @@
265297
"input": 1.75,
266298
"output": 14,
267299
"cacheRead": 0.175
268-
}
300+
},
301+
"inputModalities": [
302+
"text",
303+
"image"
304+
]
269305
},
270306
{
271307
"id": "gemini-3.8-flash",
@@ -276,7 +312,11 @@
276312
"input": 0.75,
277313
"output": 3.75,
278314
"cacheRead": 0.075
279-
}
315+
},
316+
"inputModalities": [
317+
"text",
318+
"image"
319+
]
280320
},
281321
{
282322
"id": "gemini-3-7-flash",
@@ -287,7 +327,11 @@
287327
"input": 0.75,
288328
"output": 3.75,
289329
"cacheRead": 0.075
290-
}
330+
},
331+
"inputModalities": [
332+
"text",
333+
"image"
334+
]
291335
},
292336
{
293337
"id": "gemini-3.5-flash",
@@ -298,7 +342,11 @@
298342
"input": 1.5,
299343
"output": 9,
300344
"cacheRead": 0.15
301-
}
345+
},
346+
"inputModalities": [
347+
"text",
348+
"image"
349+
]
302350
},
303351
{
304352
"id": "gemini-2.5-flash",

‎integrations/providers/usai/README.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -180,8 +180,10 @@ value is confirmed against the gateway with per-model, per-field evidence (a
180180
data PR description plus a committed evidence file; see the ADR). The
181181
`sources.capabilitiesVerified` string (`<date>; opencode <version>`) is only a
182182
catalog-wide release reminder, not evidence. Malformed upstream values are
183-
treated as unknown with a warning. Nothing is inferred from model ids or names. See
184-
[ADR 0006](../../isolation/docs/decisions/0006-neutral-model-capability-metadata.md).
183+
treated as unknown with a warning. Nothing is inferred from model ids or names.
184+
The committed `inputModalities` values are listed, with their evidence, in
185+
[the capability evidence file](../../isolation/acq-kits/opencode/docs/capability-evidence/2026-11-04-opencode-2.0.30.md).
186+
See [ADR 0006](../../isolation/docs/decisions/0006-neutral-model-capability-metadata.md).
185187

186188
## Safety guarantee
187189

‎integrations/providers/usai/catalog.json‎

Lines changed: 62 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,8 @@
22
"schemaVersion": "usai-model-catalog/v1",
33
"generatedBy": "build-catalog.mjs (generated — do not hand-edit)",
44
"sources": {
5-
"bootstrappedFrom": "integrations/providers/usai/tests/fixtures/opencode-renderer-contract.jsonc"
5+
"bootstrappedFrom": "integrations/providers/usai/tests/fixtures/opencode-renderer-contract.jsonc",
6+
"capabilitiesVerified": "2026-11-04; opencode 2.0.30"
67
},
78
"recommendations": {
89
"primary": "claude-sonnet-5",
@@ -55,7 +56,11 @@
5556
"output": 10,
5657
"cacheRead": 0.1,
5758
"cacheWrite": 2.5
58-
}
59+
},
60+
"inputModalities": [
61+
"text",
62+
"image"
63+
]
5964
},
6065
{
6166
"id": "claude-opus-5",
@@ -81,7 +86,11 @@
8186
"output": 10,
8287
"cacheRead": 0.2,
8388
"cacheWrite": 2.5
84-
}
89+
},
90+
"inputModalities": [
91+
"text",
92+
"image"
93+
]
8594
},
8695
{
8796
"id": "claude_4_8_opus",
@@ -94,7 +103,11 @@
94103
"output": 25,
95104
"cacheRead": 0.5,
96105
"cacheWrite": 6.25
97-
}
106+
},
107+
"inputModalities": [
108+
"text",
109+
"image"
110+
]
98111
},
99112
{
100113
"id": "claude_4_7_opus",
@@ -107,7 +120,11 @@
107120
"output": 25,
108121
"cacheRead": 0.5,
109122
"cacheWrite": 6.25
110-
}
123+
},
124+
"inputModalities": [
125+
"text",
126+
"image"
127+
]
111128
},
112129
{
113130
"id": "claude_4_6_sonnet",
@@ -120,7 +137,11 @@
120137
"output": 16.5,
121138
"cacheRead": 0.33,
122139
"cacheWrite": 4.125
123-
}
140+
},
141+
"inputModalities": [
142+
"text",
143+
"image"
144+
]
124145
},
125146
{
126147
"id": "claude_4_5_haiku",
@@ -165,7 +186,11 @@
165186
"output": 15,
166187
"cacheRead": 0.2,
167188
"cacheWrite": 5
168-
}
189+
},
190+
"inputModalities": [
191+
"text",
192+
"image"
193+
]
169194
},
170195
{
171196
"id": "gpt-6-astra",
@@ -184,7 +209,11 @@
184209
"output": 75,
185210
"cacheRead": 2,
186211
"cacheWrite": 25
187-
}
212+
},
213+
"inputModalities": [
214+
"text",
215+
"image"
216+
]
188217
},
189218
{
190219
"id": "gpt-6-luna",
@@ -313,7 +342,11 @@
313342
"input": 5,
314343
"output": 22.5,
315344
"cacheRead": 0.5
316-
}
345+
},
346+
"inputModalities": [
347+
"text",
348+
"image"
349+
]
317350
},
318351
{
319352
"id": "gpt_5_2_default_v2",
@@ -325,7 +358,11 @@
325358
"input": 1.75,
326359
"output": 14,
327360
"cacheRead": 0.175
328-
}
361+
},
362+
"inputModalities": [
363+
"text",
364+
"image"
365+
]
329366
},
330367
{
331368
"id": "gemini-3.8-flash",
@@ -337,7 +374,11 @@
337374
"input": 0.75,
338375
"output": 3.75,
339376
"cacheRead": 0.075
340-
}
377+
},
378+
"inputModalities": [
379+
"text",
380+
"image"
381+
]
341382
},
342383
{
343384
"id": "gemini-3-7-flash",
@@ -349,7 +390,11 @@
349390
"input": 0.75,
350391
"output": 3.75,
351392
"cacheRead": 0.075
352-
}
393+
},
394+
"inputModalities": [
395+
"text",
396+
"image"
397+
]
353398
},
354399
{
355400
"id": "gemini-3.5-flash",
@@ -361,7 +406,11 @@
361406
"input": 1.5,
362407
"output": 9,
363408
"cacheRead": 0.15
364-
}
409+
},
410+
"inputModalities": [
411+
"text",
412+
"image"
413+
]
365414
},
366415
{
367416
"id": "gemini-2.5-flash",

‎integrations/providers/usai/tests/catalog-check.test.mjs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ test("the committed catalog is a valid catalog with at least one model", () => {
5050
const catalog = loadCatalog()
5151
assert.equal(catalog.schemaVersion, "usai-model-catalog/v1")
5252
assert.ok(catalog.models.length > 0)
53+
assert.equal(catalog.models.length, 24)
5354
})
5455

5556
test("--check with no source refuses instead of comparing against anything", () => {

0 commit comments

Comments
 (0)