Skip to content

Commit bbc2a2b

Browse files
anandgupta42claude
andcommitted
fix: match the OAuth model filter on api.id, not the config map key
Addresses review on #1179. `shouldAllowOAuthModel` was called with the model map KEY. For every models.dev catalog entry the key equals `api.id`, so this was invisible — but a user can alias a model in config: provider.openai.models.fast-spark.id = "gpt-5.3-codex-spark" Config models are folded into the provider database ("extend database from config" in `Provider.state`) BEFORE auth loaders run, so that entry reaches this loader keyed `fast-spark` with the real id on `api.id`. Matching the key deleted it, hiding a model the backend actually serves. The sibling `plugin/openai/codex.ts` filter already matches on `model.api.id`. Extracted `disallowedOAuthModelKeys`, which resolves `api.id ?? key` — the fallback matters because the database only backfills `model.api.id ?? model.id ?? modelID` after this hook runs, so the field is not guaranteed populated despite the type. Behaviour is unchanged for every catalog model (key === api.id verified against models.dev/api.json); only aliases move, and an alias of a rejected id is still deleted. Comment corrections, no behaviour change: - The allowlist claimed every other gpt-5.x catalog id had been probed. It had not: gpt-5-mini, gpt-5-nano, gpt-5-pro and gpt-5.2-chat-latest are in the catalog and in neither list. Recorded as unprobed and fail-closed rather than left as an overclaim. - Recorded that the truth table is one Pro account's entitlements and has not been checked against a Plus credential. - Recorded that the codex-tagged ids reviewers keep asking about (gpt-5.1-codex, gpt-5.1-codex-max, gpt-5.1-codex-mini, gpt-5.2-codex, gpt-5.3-codex-xhigh) are absent from the models.dev catalog entirely, so no filter here can hide them. - Disambiguated the two version numbers. What reaches the wire is `Installation.VERSION` (the npm-published 0.9.7, or `local` for a dev build), not the 1.17.9 in packages/opencode/package.json, which is inherited from upstream and never sent. Also stated that the blocker is not arithmetic: bumping our number would not lift it, because the field asserts which Codex CLI release we are. Six new tests cover the alias fix in both directions, the missing-api.id fallback, and that the catalog set resolves identically either way. The allowlist membership is byte-for-byte unchanged, as is the `originator: "altimate"` identity. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VqnuBDGkh1ZT65Ti7e6DHZ
1 parent a279b9d commit bbc2a2b

2 files changed

Lines changed: 156 additions & 28 deletions

File tree

‎packages/opencode/src/plugin/codex.ts‎

Lines changed: 77 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -20,14 +20,35 @@ const OAUTH_POLLING_SAFETY_MARGIN_MS = 3000
2020
*
2121
* Every entry was verified against the live backend
2222
* (POST https://chatgpt.com/backend-api/codex/responses) on a ChatGPT Pro
23-
* credential: entries here returned HTTP 200, and every other gpt-5.x id in
24-
* the models.dev catalog returned
23+
* credential: entries here returned HTTP 200.
24+
*
25+
* CONFIRMED REJECTED, so deliberately absent — each returned
2526
* ``400 {"detail":"The '<id>' model is not supported when using Codex with a
26-
* ChatGPT account."}``.
27+
* ChatGPT account."}``: gpt-5, gpt-5.1, gpt-5.2, gpt-5.2-pro,
28+
* gpt-5.3-chat-latest, gpt-5.3-codex, gpt-5.4-nano, gpt-5.4-pro, gpt-5.5-pro,
29+
* gpt-5.6.
30+
*
31+
* NOT PROBED, and excluded by default because this list is fail-closed:
32+
* gpt-5-mini, gpt-5-nano, gpt-5-pro, gpt-5.2-chat-latest. They are in the
33+
* models.dev catalog but are not plausible Codex-tier models, and the
34+
* discovery endpoint below does not list them either. An earlier revision of
35+
* this comment claimed every other catalog id had been probed; that overstated
36+
* the evidence, and these four are the exception.
37+
*
38+
* TIER SCOPE. All of the above is what ONE ChatGPT Pro account was served.
39+
* It has not been verified against a Plus credential, so it is possible Plus
40+
* is entitled to a narrower (or wider) set. If a Plus subscriber reports a
41+
* model missing from the picker that the official client offers them, that is
42+
* the likely cause and the fix is a per-account list, not another id here.
2743
*
28-
* Confirmed rejected, so deliberately absent: gpt-5, gpt-5.1, gpt-5.2,
29-
* gpt-5.2-pro, gpt-5.3-chat-latest, gpt-5.3-codex, gpt-5.4-nano, gpt-5.4-pro,
30-
* gpt-5.5-pro, gpt-5.6.
44+
* NOT A SILENT DROP. Reviewers have twice asked why codex-tagged ids such as
45+
* gpt-5.1-codex, gpt-5.1-codex-max, gpt-5.1-codex-mini, gpt-5.2-codex and
46+
* gpt-5.3-codex-xhigh appear in neither list. They are not in the models.dev
47+
* catalog at all (checked against https://models.dev/api.json — the only codex
48+
* ids it carries are gpt-5.3-codex and gpt-5.3-codex-spark), so no filter here
49+
* can hide them: they never reach this loader. The removed
50+
* ``includes("codex")`` rule would have admitted them had they existed, which
51+
* is why the old tests named them, but nothing in the picker changed for them.
3152
*
3253
* There is no derivable rule here — the tier accepts ``gpt-5.3-codex-spark``
3354
* but rejects ``gpt-5.3-codex``, and accepts the gpt-5.6 sol/luna/terra
@@ -45,17 +66,28 @@ const OAUTH_POLLING_SAFETY_MARGIN_MS = 3000
4566
* that list is reproduced here rather than derived at runtime:
4667
*
4768
* ``client_version`` is mandatory (omitted or unparseable ⇒ HTTP 400) and is
48-
* gated against each model's ``minimal_client_version``, which today ranges
49-
* from 0.98.0 to 0.144.0. Those are Codex CLI release numbers, a numbering line
50-
* we are not on. Our own versions sit below all of them, and the gate fails
51-
* SILENTLY — ``client_version=0.9.7`` (our published version) returns
52-
* ``HTTP 200 {"models":[]}``, and a dev build's ``local`` returns
53-
* ``400 {"detail":"Invalid client_version format"}``. So discovery yields
54-
* nothing usable unless we assert a Codex CLI version we are not, which would
55-
* be impersonating the first-party client. We do not do that, so the list stays
56-
* static. If we ever have a legitimate client_version to send, the mechanism is
57-
* a small drop-in: fetch, keep ``visibility === "list"``, and fall back to this
58-
* set whenever the response is empty or the call fails.
69+
* gated against each model's ``minimal_client_version``, which at the time of
70+
* probing ranged from 0.98.0 to 0.144.0. Those are Codex CLI release numbers —
71+
* a numbering line we are not on, so comparing our number against theirs is
72+
* meaningless regardless of which way it happens to sort.
73+
*
74+
* Two different version numbers exist in this repo and it matters which one
75+
* reaches the wire. What we would send is ``Installation.VERSION``, i.e. the
76+
* build-injected ``OPENCODE_VERSION`` — the NPM-published version, 0.9.7 at
77+
* the time of probing, or the literal ``local`` for a dev build. It is NOT the
78+
* 1.17.9 in packages/opencode/package.json, which is inherited from the
79+
* upstream numbering line and never sent. Both observed values fail, and the
80+
* failure is SILENT for one of them: ``client_version=0.9.7`` returns
81+
* ``HTTP 200 {"models":[]}`` — no error to debug, just an empty picker — while
82+
* a dev build's ``local`` returns
83+
* ``400 {"detail":"Invalid client_version format"}``.
84+
*
85+
* The blocker is not arithmetic, so bumping our version would not lift it: the
86+
* field means "which Codex CLI release am I", and answering it is impersonating
87+
* the first-party client whatever number we put there. We do not do that, so
88+
* the list stays static. If we ever have a legitimate client_version to send,
89+
* the mechanism is a small drop-in: fetch, keep ``visibility === "list"``, and
90+
* fall back to this set whenever the response is empty or the call fails.
5991
*
6092
* Exported for unit-test coverage — see test/plugin/codex-allowlist.test.ts. */
6193
export const OAUTH_ALLOWED_MODELS = new Set([
@@ -86,6 +118,28 @@ export function shouldAllowOAuthModel(modelId: string): boolean {
86118
return OAUTH_ALLOWED_MODELS.has(modelId)
87119
}
88120

121+
/** Map keys to delete from an OAuth (subscription) provider's model record.
122+
*
123+
* Matches on the UPSTREAM api id, not the map key. The two are equal for every
124+
* models.dev catalog entry, but a user can alias a model in config —
125+
* ``provider.openai.models.fast-spark.id = "gpt-5.3-codex-spark"`` — which
126+
* produces an entry keyed ``fast-spark`` whose ``api.id`` carries the real id.
127+
* Config models are folded into the provider database (Provider.state, "extend
128+
* database from config") BEFORE auth loaders run, so they do reach this filter;
129+
* matching the key would delete a model the backend actually serves.
130+
*
131+
* Falls back to the key when ``api.id`` is absent: the database only backfills
132+
* ``model.api.id ?? model.id ?? modelID`` after this hook has run, so the field
133+
* is not guaranteed populated at this point even though the type says so.
134+
*
135+
* Returns keys rather than mutating, so the caller keeps the in-place delete on
136+
* the shared database object and this stays unit-testable. */
137+
export function disallowedOAuthModelKeys(models: Record<string, { api?: { id?: string } }>): string[] {
138+
return Object.entries(models)
139+
.filter(([key, model]) => !shouldAllowOAuthModel(model?.api?.id ?? key))
140+
.map(([key]) => key)
141+
}
142+
89143
interface PkceCodes {
90144
verifier: string
91145
challenge: string
@@ -469,18 +523,19 @@ export async function CodexAuthPlugin(input: PluginInput): Promise<Hooks> {
469523
if (auth.type !== "oauth") return {}
470524

471525
// Filter models to only those the ChatGPT-subscription (Codex) tier
472-
// accepts. Delegates to ``shouldAllowOAuthModel`` (module-level,
473-
// above). See OAUTH_ALLOWED_MODELS + shouldAllowOAuthModel for the
474-
// criteria + how to add new gpt-5.N releases.
526+
// accepts. Delegates to ``disallowedOAuthModelKeys`` (module-level,
527+
// above), which matches on each model's upstream ``api.id`` so a
528+
// config alias of a supported model survives. See OAUTH_ALLOWED_MODELS
529+
// for the criteria + how to add new gpt-5.N releases.
475530
//
476531
// NOTE: this file is the ACTIVE plugin (wired via plugin/index.ts).
477532
// The sibling plugin/openai/codex.ts is an unwired in-progress
478533
// refactor that keeps its OWN ALLOWED_MODELS + parseFloat > 5.4
479534
// fallback — this filter does NOT share a source of truth with it.
480535
// Adopting shouldAllowOAuthModel there is followup on that refactor.
481536
// (Closes #1132 — GPT 5.6 missing from picker.)
482-
for (const modelId of Object.keys(provider.models)) {
483-
if (!shouldAllowOAuthModel(modelId)) delete provider.models[modelId]
537+
for (const modelId of disallowedOAuthModelKeys(provider.models)) {
538+
delete provider.models[modelId]
484539
}
485540

486541
// Zero out costs for Codex (included with ChatGPT subscription)

‎packages/opencode/test/plugin/codex-allowlist.test.ts‎

Lines changed: 79 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,24 @@
22
// in packages/opencode/src/plugin/codex.ts — the ACTIVE plugin wired
33
// via plugin/index.ts.
44
//
5-
// The allowlist is a verified truth table, not a guess. Every id asserted
6-
// below was probed against the live backend
7-
// (POST https://chatgpt.com/backend-api/codex/responses) on a ChatGPT Pro
8-
// credential, using our own `originator: altimate` client identity over plain
9-
// HTTP. Accepted ids returned HTTP 200; rejected ids returned
5+
// The allowlist is a verified truth table, not a guess. Every id in
6+
// VERIFIED_ACCEPTED and VERIFIED_REJECTED below was probed against the live
7+
// backend (POST https://chatgpt.com/backend-api/codex/responses) on a ChatGPT
8+
// Pro credential, using our own `originator: altimate` client identity over
9+
// plain HTTP. Accepted ids returned HTTP 200; rejected ids returned
1010
// 400 {"detail":"The '<id>' model is not supported when using Codex with a
1111
// ChatGPT account."}
1212
//
13+
// Ids asserted OUTSIDE those two lists were NOT probed and are not claimed to
14+
// have been. They fall in two groups, both deliberate:
15+
// * ids absent from the models.dev catalog (gpt-5.1-codex, gpt-5.1-codex-max,
16+
// codex-hypothetical-future-name, …). These cannot reach the loader at all,
17+
// so probing them would be meaningless; they are here purely as shape
18+
// assertions against a substring rule creeping back in.
19+
// * obviously-unrelated ids (claude-*, gemini-*, gpt-4o), same reason.
20+
// The probe scope, and the four catalog ids left unprobed, are recorded on
21+
// OAUTH_ALLOWED_MODELS in src/plugin/codex.ts.
22+
//
1323
// Two directions of breakage this file guards:
1424
// * false positives — an id offered in the picker that 400s at request time
1525
// (previously gpt-5.2, gpt-5.6, gpt-5.3-codex)
@@ -30,7 +40,7 @@
3040
// adopting ``shouldAllowOAuthModel`` here (and expanding this file's
3141
// coverage to the newly-active filter) is followup work.
3242
import { describe, expect, test } from "bun:test"
33-
import { OAUTH_ALLOWED_MODELS, shouldAllowOAuthModel } from "../../src/plugin/codex"
43+
import { OAUTH_ALLOWED_MODELS, disallowedOAuthModelKeys, shouldAllowOAuthModel } from "../../src/plugin/codex"
3444

3545
/** Verified HTTP 200 on a ChatGPT Pro subscription credential. */
3646
const VERIFIED_ACCEPTED = [
@@ -120,3 +130,66 @@ describe("shouldAllowOAuthModel — behavior of the filter itself", () => {
120130
}
121131
})
122132
})
133+
134+
describe("disallowedOAuthModelKeys — what the loader actually deletes", () => {
135+
const model = (apiId?: string) => (apiId === undefined ? {} : { api: { id: apiId } })
136+
137+
test("a config alias of a supported model is kept, keyed by its alias", () => {
138+
// The bug this guards: the loader used to match the MAP KEY. Config models
139+
// are folded into the provider database before auth loaders run, so
140+
// `provider.openai.models.fast-spark.id = "gpt-5.3-codex-spark"` arrives
141+
// keyed `fast-spark` with the real id on `api.id`. Matching the key deleted
142+
// a model the backend serves.
143+
const models = {
144+
"fast-spark": model("gpt-5.3-codex-spark"),
145+
"my-flagship": model("gpt-5.6-sol"),
146+
}
147+
expect(disallowedOAuthModelKeys(models)).toEqual([])
148+
})
149+
150+
test("an alias of an unsupported model is still deleted", () => {
151+
// The alias must not become a way to smuggle a rejected id past the filter
152+
// — it is the api.id that reaches the backend, so that is what is judged.
153+
const models = {
154+
"totally-fine-name": model("gpt-5.6"),
155+
"gpt-5.6-sol": model("gpt-5.6-sol"),
156+
}
157+
expect(disallowedOAuthModelKeys(models)).toEqual(["totally-fine-name"])
158+
})
159+
160+
test("api.id wins over the key in both directions", () => {
161+
const models = {
162+
// key allowed, api.id rejected -> delete
163+
"gpt-5.6-sol": model("gpt-5.6"),
164+
// key rejected, api.id allowed -> keep
165+
"gpt-5.6": model("gpt-5.6-sol"),
166+
}
167+
expect(disallowedOAuthModelKeys(models)).toEqual(["gpt-5.6-sol"])
168+
})
169+
170+
test("falls back to the key when api.id is absent", () => {
171+
// The database backfills api.id only AFTER this hook runs, so the field can
172+
// be missing despite the type. Behaviour must degrade to the old key match,
173+
// not throw and not allow everything through.
174+
const models = {
175+
"gpt-5.6-sol": model(undefined),
176+
"gpt-5.6": model(undefined),
177+
"gpt-5.4": { api: {} },
178+
"gpt-5.2": { api: {} },
179+
}
180+
expect(disallowedOAuthModelKeys(models).sort()).toEqual(["gpt-5.2", "gpt-5.6"])
181+
})
182+
183+
test("the catalog set resolves identically whether matched by key or api.id", () => {
184+
// Every models.dev catalog entry has key === api.id (checked against
185+
// https://models.dev/api.json), so this change is behaviour-preserving for
186+
// catalog models — only aliases move. Guards against a future refactor
187+
// quietly changing which flagship models the picker offers.
188+
const catalog = Object.fromEntries([...VERIFIED_ACCEPTED, ...VERIFIED_REJECTED].map((id) => [id, model(id)]))
189+
expect(disallowedOAuthModelKeys(catalog).sort()).toEqual([...VERIFIED_REJECTED].sort())
190+
})
191+
192+
test("an empty model map yields nothing to delete", () => {
193+
expect(disallowedOAuthModelKeys({})).toEqual([])
194+
})
195+
})

0 commit comments

Comments
 (0)