Repository navigation
Commit 5993471
fix: build release binaries against the live models.dev catalog (#1188)
* fix: build release binaries against the live models.dev catalog
`.github/workflows/release.yml` built every platform binary with
`MODELS_DEV_API_JSON: test/tool/fixtures/models-api.json`, so each shipped
binary embedded a checked-in test fixture as its bundled models.dev catalog
instead of a release-time one. The fixture's newest entry is `2026-03-30`.
Verified against the shipped `0.9.7` binary in an isolated `HOME`: its
snapshot-only `openai` set is exactly the fixture's (46 of 47 ids, missing only
`gpt-5-chat-latest`, which is filtered elsewhere), with zero ids the fixture
does not have. That catalog still offers 11 ids the live catalog has dropped
(`gpt-5-codex`, `gpt-5.1-codex`, `gpt-5.1-codex-max`, `gpt-5.1-codex-mini`,
`gpt-5.2-codex`, `codex-mini-latest`, `o1-mini`, `o1-preview`, and three
others) and is missing `gpt-5.5` plus every `gpt-5.6` variant. It carries 105
providers against the live catalog's 207, so the staleness is not
OpenAI-specific.
Upstream's `publish.yml` sets no such override and fetches live at release
time; `git log -S` shows ours has been there since the initial fork commit
`f2cd5c1245` with no explanatory comment, and reads as copy-paste from the CI
job where a hermetic build genuinely is correct.
Removing the override makes release builds depend on models.dev, so
`build.ts` now validates the payload before writing the snapshot:
- non-2xx responses throw instead of flowing an error page into the snapshot
(`fetch` resolves for 4xx/5xx, and a JSON error body is valid TypeScript
that would have shipped as a catalog with no providers)
- the payload must parse, be a provider object, carry at least 50 providers,
and include `anthropic`, `openai` and `google`
`ci.yml` and `pre-release-check.ts` keep the fixture pin — a hermetic build is
correct there.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VqnuBDGkh1ZT65Ti7e6DHZ
* fix: harden the catalog guard against hangs and hollow payloads
Addresses review findings on the guard added in the previous commit.
Bound the fetch with `AbortSignal.timeout(60s)`. A blackholed connection is the
one failure `fetch` does not surface on its own — no error, no bytes, just a
hang until the job's own timeout kills it with no useful message. The previous
version claimed to fail loudly but would have hung in that case.
Validate provider records, not just top-level keys. A payload can carry 50+
keys whose values are junk: that passed the old key-count check and would have
shipped a catalog with nothing selectable. Each required provider must now
carry a non-empty `models` object. Verified against a crafted payload of 80
junk keys and one where every provider is present but hollow.
Also drops the duplicated `${modelsUrl}/api.json` construction and the repeated
`process.env.MODELS_DEV_API_JSON` reads, and avoids an unsafe type assertion by
reading the parsed catalog through a `Map<string, unknown>`.
Not changed: the eight matrix builds still fetch independently, so a models.dev
update mid-release could in principle give different binaries different
catalogs. Fixing that needs a fetch-once job passing the catalog as an artifact,
which is a larger change than this PR; upstream has the same property.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VqnuBDGkh1ZT65Ti7e6DHZ
* fix: repair the empty-env regression and harden the catalog guard further
Second review wave on the guard. Three real defects, one of them mine.
MINE — an empty `MODELS_DEV_API_JSON` broke the build. The previous commit
replaced a consistent truthiness check with `??` when deduplicating the origin,
so a variable that is SET BUT EMPTY kept `""` as `modelsOrigin` while the data
branch fell through to the fetch. Reproduced: `error: models.dev fetch from
failed ... code: "ERR_INVALID_URL"`. Now normalized with `|| undefined` so all
three uses agree, and an empty value reads as unset again.
The 60s bound did not cover a blocked DNS lookup. `AbortSignal.timeout` cannot
cancel `getaddrinfo()` — this repo already documents that in
src/provider/models.ts (#1052 D14), where a sandboxed-network blackhole
outlived the signal. Added a `setTimeout` backstop that prints the origin and
exits non-zero, so an unresolvable host fails the build with a reason instead of
hanging every matrix job until the workflow timeout.
`res.text()` sat outside the try, so a host that sends headers promptly then
stalls mid-body surfaced a bare `AbortError` with none of the context. Body read
now has its own catch and its own message.
Custom catalogs were broken by an unconditional size floor. `OPENCODE_MODELS_URL`
and `MODELS_DEV_API_JSON` are legitimate ways to point a build at a small private
catalog, and those worked before this PR. The size floor and required-provider
checks now run in `strict` mode only — ON for every release build (keyed on
`OPENCODE_RELEASE`, so pointing a release at a custom catalog cannot skip them)
and for any plain default-endpoint build; OFF only for a non-release build with
an explicit override.
Structural validation is now stronger and runs in BOTH modes: the catalog must be
non-empty and every provider entry must carry a string `id` and an object
`models`. This uses the same predicate the runtime screens entries with
(`isCatalogEntry`), deliberately not the zod `Provider` schema, which requires an
`options` record real models.dev entries lack.
Verified across a 9-case matrix in both modes. Strict rejects all nine, including
a hollow catalog and a 1-provider custom one; non-strict accepts the small custom
catalog while still rejecting a single malformed entry hidden among 104 valid
ones. Real fixture and live catalog both pass strict; all 105/144/207 providers
in the fixture, committed blob and live catalog satisfy the structural check.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VqnuBDGkh1ZT65Ti7e6DHZ
* fix: validate nested model records before embedding the catalog
Review found a real unguarded dereference. `Provider.fromModelsDevModel` reads
`model.limit.context` directly while every neighbouring field uses `?.`/`??`:
limit: {
context: model.limit.context,
and the runtime's per-entry screen (`isCatalogEntry`) only checks that a provider
has a string `id` and an object `models` — it does not look inside the map. So a
catalog carrying a null or `limit`-less model record passes every existing check
and crashes provider initialisation on cold start.
Provider validation now walks each provider's models and requires every value to
be an object with a `limit` object whose `context` is a number — exactly the
fields dereferenced without a guard, and nothing more. Errors name the offending
model (`openai/gpt-5.2-codex (no limit)`) rather than just the provider.
Still deliberately not the zod `Provider` schema, which requires an `options`
record real models.dev entries lack and would reject valid live data — the
history is recorded on `isCatalogEntry` in src/provider/models-catalog.ts.
Checked for false positives before adding it: every model in the release fixture
(4108), the committed snapshot (5299) and the live catalog (7487) passes, so
zero real records are rejected.
Runs in both strict and relaxed modes, since a custom catalog with a malformed
model record would crash the runtime just the same. Verified against a fixture
with `limit` deleted from one model and another with a model value set to null;
both are rejected in both modes, and the real fixture still passes both.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VqnuBDGkh1ZT65Ti7e6DHZ
* fix: give the fetch backstop a margin and stop overclaiming what it covers
Two review findings on the backstop added in eeea7c9, both correct.
The backstop shared CATALOG_FETCH_TIMEOUT_MS with the abort signal and was armed
before the fetch, so at the timeout boundary `process.exit(1)` always won the
race and replaced the precise per-stage messages ("fetch failed", "body read
failed") with its own generic one. A slow-but-reachable catalog or a mid-body
stall would have been reported as an unresolvable host. The backstop now runs at
CATALOG_FETCH_TIMEOUT_MS + 15s, so the abort path fires first wherever the abort
signal can act at all, and the backstop is reserved for the case it exists for.
The comment also overclaimed. A `setTimeout` callback runs on the event loop, so
if `getaddrinfo` blocks that loop outright the timer cannot fire any more than
the abort signal can — the claim that this bounds the DNS-blackhole case was
wrong. The comment now states the real coverage: it catches a request that hangs
unresolved while the loop still ticks, turning a silent full-length job timeout
into a fast labelled failure, and the workflow `timeout-minutes` remains the
backstop for a genuinely blocked thread. Either way the build fails and never
falls through to a stale catalog.
Message reworded from "exceeded Nms (unresolvable host or blackholed network)"
to "did not settle within Nms (host unreachable or unresolvable)", which is what
is actually known at that point.
Verified the ordering: a refused connection still reports "models.dev fetch from
... failed or timed out after 60000ms", not the backstop message.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VqnuBDGkh1ZT65Ti7e6DHZ
* fix: validate release model catalog safely
* fix: validate required catalog model fields
* fix: reject unsafe catalog prototype keys
---------
Co-authored-by: Claude Opus 5 <[email protected]>1 parent c59a5a2 commit 5993471
4 files changed
Lines changed: 595 additions & 6 deletions
File tree
- .github/workflows
- packages/opencode
- script
- test/provider
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
98 | 98 | | |
99 | 99 | | |
100 | 100 | | |
101 | | - | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
102 | 107 | | |
103 | 108 | | |
104 | 109 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
21 | 22 | | |
22 | 23 | | |
23 | 24 | | |
| |||
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
35 | 120 | | |
36 | 121 | | |
37 | 122 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
0 commit comments