Skip to content

Commit dca7360

Browse files
sahrizviclaude
andcommitted
fix(models): [#1052] structural catalog validation, not the zod schema
Fixes a regression I introduced in round 3, caught by CI. Round 3 gated the models.dev cache on `Provider.safeParse`, as a review suggested. It looked right and was wrong: the schema requires a `Model.options` record that real models.dev entries do not carry, so it rejects 0 of the 144 providers in our own bundled snapshot. Nothing was ever cached, and every load fell through to a network fetch — which hangs in a sandboxed environment. The only signal was an unrelated CLI smoke test timing out at 60s ("fresh-DB production path adopts the core schema"). Bisected to round 3, then to this file. The validator is now structural: a non-empty map whose values are provider objects carrying a `models` map. That still rejects everything the reviewers were worried about — `null`, `[]`, scalars, and object-shaped error payloads like `{"error": "rate limited"}` — without coupling cache acceptance to a schema that does not match the payload. It moved to `provider/models-catalog.ts` (flat exports + self-reexport per AGENTS.md) so it is directly testable. Trying to cover it through `ModelsDev.get()` did not work: `Flag.OPENCODE_MODELS_PATH` is captured at module load, so a test cannot redirect the cache path afterwards. 12 new tests pin the behaviour, and the first one is the guard that was missing: the real bundled catalog must be accepted. `output-hygiene` is back to 3/3; provider, scanner and install suites total 811 pass. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
1 parent cafdcf4 commit dca7360

3 files changed

Lines changed: 108 additions & 26 deletions

File tree

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
// altimate_change start — bot-review fix: models.dev catalog validation.
2+
//
3+
// Split out of models.ts so it can be tested directly. Every path that can
4+
// populate the on-disk cache runs through it: the cold fetch, the hourly
5+
// refresh, and the cache READ (an entry poisoned by an older build must not be
6+
// trusted either).
7+
//
8+
// A 2xx body being valid JSON does not make it the catalog. `null`, `[]`, a
9+
// scalar and an object-shaped error payload (`{"error": "rate limited"}`) all
10+
// survive JSON.parse, and a misconfigured proxy returns exactly those with a
11+
// JSON content-type. Caching one poisons the cache for the whole TTL, and
12+
// `Provider.state` then maps over it reading `provider.models` / `provider.id`.
13+
//
14+
// This is a STRUCTURAL check, deliberately NOT the zod `Provider` schema. A
15+
// review suggested the schema and it looked right, but `Provider.safeParse`
16+
// rejects 0 of the 144 providers in our own bundled snapshot: `Model` requires
17+
// an `options` record that real models.dev entries do not carry. Gating the
18+
// cache on it meant nothing was ever cached and every load fell through to a
19+
// network fetch, which hangs in sandboxed CI. So the check asserts only what
20+
// consumers actually rely on: a non-empty map whose values are provider objects
21+
// carrying a `models` map.
22+
//
23+
// Module shape follows packages/opencode/AGENTS.md: flat exports plus a
24+
// self-reexport, not `export namespace`.
25+
26+
/** True when `value` is shaped like a models.dev catalog rather than junk. */
27+
export function isCatalog(value: unknown): value is Record<string, unknown> {
28+
if (!value || typeof value !== "object" || Array.isArray(value)) return false
29+
const entries = Object.values(value)
30+
if (entries.length === 0) return false
31+
return entries.some((entry) => {
32+
if (!entry || typeof entry !== "object" || Array.isArray(entry)) return false
33+
const models = Reflect.get(entry, "models")
34+
return !!models && typeof models === "object" && !Array.isArray(models)
35+
})
36+
}
37+
38+
/** Parse a models.dev response body, returning undefined unless it is a catalog. */
39+
export function parseCatalog(text: string): Record<string, unknown> | undefined {
40+
try {
41+
const parsed: unknown = JSON.parse(text)
42+
return isCatalog(parsed) ? parsed : undefined
43+
} catch {
44+
return undefined
45+
}
46+
}
47+
48+
export * as ModelsCatalog from "./models-catalog"
49+
// altimate_change end

‎packages/opencode/src/provider/models.ts‎

Lines changed: 7 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ import { lazy } from "@/util/lazy"
88
import { Filesystem } from "../util/filesystem"
99
import { Flock } from "@/util/flock"
1010
import { Hash } from "@/util/hash"
11+
import { ModelsCatalog } from "./models-catalog"
1112

1213
// Try to import bundled snapshot (generated at build time)
1314
// Falls back to undefined in dev mode when snapshot doesn't exist
@@ -108,32 +109,12 @@ export namespace ModelsDev {
108109
return { ok: result.ok, text: await result.text() }
109110
}
110111

111-
// altimate_change — bot-review fix (round 2): one validator for every path that
112-
// can populate the cache. A 2xx body being valid JSON does not make it the
113-
// catalog — `null`, `[]`, a scalar, and an object-shaped error payload
114-
// (`{"error": "rate limited"}`) all survive `JSON.parse`, and a misconfigured
115-
// proxy returns exactly those with a JSON content-type. Caching one poisons
116-
// the disk cache for the whole TTL, and `Provider.state` then maps over it
117-
// reading `provider.models` / `provider.id`.
118-
//
119-
// At least ONE value must validate as a Provider. Requiring ALL of them would
120-
// let a single new upstream field empty the catalog, which is a worse failure
121-
// than the one being prevented.
122-
function isCatalog(value: unknown): value is Record<string, unknown> {
123-
if (!value || typeof value !== "object" || Array.isArray(value)) return false
124-
const entries = Object.values(value)
125-
if (entries.length === 0) return false
126-
return entries.some((entry) => Provider.safeParse(entry).success)
127-
}
128-
129-
function parseCatalog(text: string): Record<string, unknown> | undefined {
130-
try {
131-
const parsed: unknown = JSON.parse(text)
132-
return isCatalog(parsed) ? parsed : undefined
133-
} catch {
134-
return undefined
135-
}
136-
}
112+
// altimate_change start — bot-review fix: catalog validation lives in
113+
// ./models-catalog so it can be tested directly; see that file for why it is
114+
// structural rather than schema-based.
115+
const isCatalog = ModelsCatalog.isCatalog
116+
const parseCatalog = ModelsCatalog.parseCatalog
117+
// altimate_change end
137118

138119
export const Data = lazy(async () => {
139120
// Bot-review fix: a cache poisoned by an older build must not be trusted on
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
import { describe, expect, test } from "bun:test"
2+
import { ModelsCatalog } from "../../src/provider/models-catalog"
3+
import { snapshot } from "../../src/provider/models-snapshot"
4+
5+
// altimate_change — bot-review fix: pin the models.dev catalog validator.
6+
//
7+
// A review asked for cached bodies to be validated with `Provider.safeParse`.
8+
// That looked right and was wrong: the schema requires a `Model.options` record
9+
// real models.dev entries do not carry, so it rejected 0 of the 144 providers
10+
// in our own bundled snapshot. Nothing was ever cached and every load fell
11+
// through to a network fetch, which hangs in sandboxed CI — a 60s timeout in an
12+
// unrelated CLI smoke test was the only signal. The first case below is the
13+
// guard that would have caught it.
14+
describe("ModelsCatalog.isCatalog", () => {
15+
test("accepts the real bundled catalog", () => {
16+
expect(ModelsCatalog.isCatalog(snapshot)).toBe(true)
17+
})
18+
19+
test("accepts a minimal catalog-shaped record", () => {
20+
expect(ModelsCatalog.isCatalog({ acme: { id: "acme", models: { "acme/one": {} } } })).toBe(true)
21+
})
22+
23+
for (const [label, value] of [
24+
["an object-shaped error payload", { error: "rate limited" }],
25+
["a record whose values are strings", { acme: "nope" }],
26+
["a provider without a models map", { acme: { id: "acme" } }],
27+
["a JSON array", []],
28+
["a JSON scalar", 5],
29+
["null", null],
30+
["an empty object", {}],
31+
] as const) {
32+
test(`rejects ${label}`, () => {
33+
expect(ModelsCatalog.isCatalog(value)).toBe(false)
34+
})
35+
}
36+
})
37+
38+
describe("ModelsCatalog.parseCatalog", () => {
39+
test("returns the catalog for a real body", () => {
40+
expect(Object.keys(ModelsCatalog.parseCatalog(JSON.stringify(snapshot)) ?? {}).length).toBe(
41+
Object.keys(snapshot).length,
42+
)
43+
})
44+
45+
test("returns undefined for non-JSON, so an HTML error page is never cached", () => {
46+
expect(ModelsCatalog.parseCatalog("<html>502 Bad Gateway</html>")).toBeUndefined()
47+
})
48+
49+
test("returns undefined for JSON that is not a catalog", () => {
50+
expect(ModelsCatalog.parseCatalog('{"error":"rate limited"}')).toBeUndefined()
51+
})
52+
})

0 commit comments

Comments
 (0)