Repository navigation
feat(discovery): list a custom endpoint's own models in the picker - #287
Conversation
Scoping to a custom endpoint in the picker opened an empty list. The provider row read `0 models` and no notice said why, because nothing had asked the endpoint for its dynamic models catalog. Cause: buildProviderDefinition (providers/custom-endpoints-loader.ts) set no modelDiscovery, so PickerDataSource.providerList() reported hasDiscovery: false for every customEndpoints entry and for every bundled predefined row. providerCountText's `lists its own models` phrasing, written in rows.tsx for a self-hosted vLLM, was unreachable for the same reason. An endpoint that declares the openai wire format now derives a descriptor: the `/models` path beside the `/chat/completions` it already declares. `/chat/completions` gives `/models`, `/v1/chat/completions` gives `/v1/models`, `/api/v2/chat/completions` gives `/api/v2/models`. That is a rule applied to the endpoint's own declaration, not a pinned URL. Discovery stays on demand. The anthropic, litellm and gemini transports derive nothing: their model lists are other shapes, each with its own headers and field names. Verified live against an OpenAI-compatible endpoint whose /v1/models had never been requested: served 427 models in 1164 ms, 140 chat-capable after isReportedChatCapable, including an id that could previously only be reached by typing it. custom-endpoints-loader.test.ts pins the four derivations and the three formats that derive nothing.
A keyless endpoint scoped in the picker was told it had no credentials, and the notice sent its user looking for an environment variable they correctly do not have. Cause: discoverProviderModelsCatalog (providers/model-discovery.ts) gated on hasAuthHeader alone. For authScheme "none" the authority answers static headers with no auth header by design (api-key-credential.ts:362), which that gate cannot distinguish from an unset key. The gate now exempts the scheme, which DECLARES that no credential was ever expected rather than that one is missing. Reachable only now that a custom endpoint derives a discovery descriptor at all: before, a keyless endpoint carried no modelDiscovery, so the gate was never reached. model-discovery-models-catalog.test.ts covers the scheme beside the blank-credential test it was being confused with. Reverting the one-line change fails it with kind "failed".
Both the architecture note and the settings reference said the models field "restricts the endpoint to listed models". No code has ever enforced that: --model my-endpoint@unlisted-name reaches the endpoint whichever ids are listed. The array's only consumer is discoverableModelIds in proxy-server.ts, which builds the proxy's own /v1/models response — the list the nested Claude Code's /model command reads. Both entries now say that, and say what it is not: neither an allowlist nor the picker's list, which the endpoint now answers for itself. Verified by grep: no read of the field outside that route.
d93bc59 to
6318ff3
Compare
# Conflicts: # packages/cli/src/providers/custom-endpoints-loader.ts
A bundled predefined row compiles to the same complex definition as a custom endpoint, so it now carries a derived openai-models-list descriptor and the picker asks the vendor's own /models. The new test in custom-endpoints-loader.test.ts loads a synthetic predefined row and asserts the descriptor is the sibling of its apiPath (/openai/v1/chat/completions -> /openai/v1/models). The catalog-row fallback on a failed request is already pinned by STATE D in model-selector-discovery.test.ts. Also drops an issue number and change-history narration from the comments in custom-endpoints-loader.ts and model-discovery.ts. Co-Authored-By: Magus <[email protected]> Crafted with agentic harness Magus (https://github.com/MadAppGang/magus) Claude-Session: https://claude.ai/code/session_01XTrcSehh8dM3tNFWKVB138
|
Merged. A custom endpoint showing an empty list in the picker was a bad first experience, and asking the endpoint for its own models is the right answer. My commit on top fixes the import that broke after main moved, adds a test that a predefined endpoint's row gets discovered too, and drops an issue number from a comment. The release review caught two more. Discovery returns complete wire ids, but the handler still put |
The picker never asked a custom endpoint what it serves
Scoping the model picker to a
customEndpointsprovider opens an empty list. Theprovider row reads
0 models, and no notice says why.The endpoint works —
--model my-endpoint@some-id "task"reaches it and streams. The idjust has to be typed from memory, because nothing ever asked the host for its dynamic
models catalog.
Cause
buildProviderDefinition(providers/custom-endpoints-loader.ts) set nomodelDiscoverydescriptor.
PickerDataSource.providerList()therefore reportedhasDiscovery: falseforevery
customEndpointsentry and for every one of the 27 bundledpredefined-catalog.tsrows, which compile to
CustomEndpointComplexand travel the same function.ModelPicker'slistmemo then falls back to catalog-scoped rows — empty for a private host — andproviderCountTextprints0 models.rows.tsx:315's other phrasing,lists its own models, was written for exactly this case("a self-hosted vLLM") and was unreachable, since that branch needs
hasDiscovery.What this changes
An endpoint that declares the
openaiwire format now derives a descriptor: the/modelspath beside the
/chat/completionsit already declares.simple,format: "openai"/modelscomplex,transport: "openai", defaultapiPath/v1/modelscomplex,apiPath: "/api/v2/chat/completions"/api/v2/modelsformat/transportDerived, never configured — no new config surface.
format: "openai"is already theuser's statement that the host speaks the OpenAI API, and
GET /modelsis part of that API,published as a sibling of
/chat/completions. So this is one rule applied to the path theentry already carries, not a pinned URL — which is the distinction the "never hardcode a
provider's models" invariant draws. All 27 bundled rows declare only
/v1/chat/completionsor
/chat/completions(the version segment lives inbaseUrl), so each lands on the pathits vendor actually serves.
Only
openai. Theanthropictransport's/v1/modelswants ananthropic-versionheaderand answers
created_atwhereopenai-models-listreadscreated;litellmandgeminiare different shapes again. Each deserves its own change with its own live verification.
Discovery stays on demand — it runs when the user scopes to that provider, never at
startup. An endpoint that does not serve the path becomes a
failedoutcome naming the URLand the status, which is strictly more than the silent empty list it replaces.
Three commits, separable
feat(discovery)— the derivation, its tests, and the docs for it.fix(discovery): stop reporting no-credentials for authScheme "none"— a keylessendpoint (Claudish configuration for Clawrouter #139) reaches
discoverProviderModelsCatalogwith static headers and no authheader by design (
api-key-credential.ts:362). The blank-credential gate could not tellthat from an unset key, so it would have sent the user hunting for a variable they
correctly do not have. One line; reachable only because of commit 1.
docs(endpoints)— both docs said themodelsfield "restricts the endpoint tolisted models". No code has ever enforced that: its only consumer is
discoverableModelIdsin
proxy-server.ts, which builds the proxy's own/v1/models— the list the nestedClaude Code's
/modelcommand reads. Drop this commit independently if you'd rather keepthe wording.
Verification
/v1/modelsthis code had neverrequested:
hasDiscovery: true, outcomeservedin 1164 ms, 427 models, 140chat-capable after
isReportedChatCapable— including ids previously reachable only bytyping them.
bun test packages/cli/src/providers packages/cli/src/picker— 1442 pass, 1 skip, 0 fail.Received: "failed".bun run typecheckclean;bun run formatno fixes;bun run lintexits 0.One note on lint: biome reports cognitive complexity 33 on
discoverProviderModelsCatalog.The baseline there is already 32 — the
authSchemecheck adds one — and it is one of 22pre-existing warnings repo-wide, so I left it rather than refactor a function this PR only
touches by a line.