Skip to content

feat(discovery): list a custom endpoint's own models in the picker - #287

Merged
erudenko merged 5 commits into
MadAppGang:mainfrom
anandghegde:feat/custom-endpoint-model-discovery
Nov 9, 2026
Merged

erudenko merged 5 commits into
MadAppGang:mainfrom
anandghegde:feat/custom-endpoint-model-discovery

Conversation

@anandghegde

Copy link
Copy Markdown
Contributor

The picker never asked a custom endpoint what it serves

Scoping the model picker to a customEndpoints provider opens an empty list. The
provider row reads 0 models, and no notice says why.

The endpoint works — --model my-endpoint@some-id "task" reaches it and streams. The id
just has to be typed from memory, because nothing ever asked the host for its dynamic
models catalog.

Cause

buildProviderDefinition (providers/custom-endpoints-loader.ts) set no modelDiscovery
descriptor. PickerDataSource.providerList() therefore reported hasDiscovery: false for
every customEndpoints entry and for every one of the 27 bundled predefined-catalog.ts
rows
, which compile to CustomEndpointComplex and travel the same function. ModelPicker's
list memo then falls back to catalog-scoped rows — empty for a private host — and
providerCountText prints 0 models.

rows.tsx:315's other phrasing, lists its own models, was written for exactly this case
("a self-hosted vLLM") and was unreachable, since that branch needs hasDiscovery.

What this changes

An endpoint that declares the openai wire format now derives a descriptor: the /models
path beside the /chat/completions it already declares.

declared discovery path
simple, format: "openai" /models
complex, transport: "openai", default apiPath /v1/models
complex, apiPath: "/api/v2/chat/completions" /api/v2/models
any other format/transport none — unchanged

Derived, never configured — no new config surface. format: "openai" is already the
user's statement that the host speaks the OpenAI API, and GET /models is part of that API,
published as a sibling of /chat/completions. So this is one rule applied to the path the
entry already carries, not a pinned URL — which is the distinction the "never hardcode a
provider's models" invariant draws. All 27 bundled rows declare only /v1/chat/completions
or /chat/completions (the version segment lives in baseUrl), so each lands on the path
its vendor actually serves.

Only openai. The anthropic transport's /v1/models wants an anthropic-version header
and answers created_at where openai-models-list reads created; litellm and gemini
are different shapes again. Each deserves its own change with its own live verification.

Discovery stays on demand — it runs when the user scopes to that provider, never at
startup. An endpoint that does not serve the path becomes a failed outcome naming the URL
and the status, which is strictly more than the silent empty list it replaces.

Three commits, separable

  1. feat(discovery) — the derivation, its tests, and the docs for it.
  2. fix(discovery): stop reporting no-credentials for authScheme "none" — a keyless
    endpoint (Claudish configuration for Clawrouter #139) reaches discoverProviderModelsCatalog with static headers and no auth
    header by design (api-key-credential.ts:362). The blank-credential gate could not tell
    that from an unset key, so it would have sent the user hunting for a variable they
    correctly do not have. One line; reachable only because of commit 1.
  3. docs(endpoints) — both docs said the models field "restricts the endpoint to
    listed models". No code has ever enforced that: its only consumer is discoverableModelIds
    in proxy-server.ts, which builds the proxy's own /v1/models — the list the nested
    Claude Code's /model command reads. Drop this commit independently if you'd rather keep
    the wording.

Verification

  • Live, against an OpenAI-compatible endpoint whose /v1/models this code had never
    requested: hasDiscovery: true, outcome served in 1164 ms, 427 models, 140
    chat-capable after isReportedChatCapable — including ids previously reachable only by
    typing them.
  • bun test packages/cli/src/providers packages/cli/src/picker — 1442 pass, 1 skip, 0 fail.
  • Mutation-checked: reverting the commit-2 line fails its new test with Received: "failed".
  • bun run typecheck clean; bun run format no fixes; bun run lint exits 0.

One note on lint: biome reports cognitive complexity 33 on discoverProviderModelsCatalog.
The baseline there is already 32 — the authScheme check adds one — and it is one of 22
pre-existing warnings repo-wide, so I left it rather than refactor a function this PR only
touches by a line.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Scoping to a custom endpoint in the picker opened an empty list. The
provider row read `0 models` and no notice said why, because nothing had
asked the endpoint for its dynamic models catalog.

Cause: buildProviderDefinition (providers/custom-endpoints-loader.ts)
set no modelDiscovery, so PickerDataSource.providerList() reported
hasDiscovery: false for every customEndpoints entry and for every
bundled predefined row. providerCountText's `lists its own models`
phrasing, written in rows.tsx for a self-hosted vLLM, was unreachable
for the same reason.

An endpoint that declares the openai wire format now derives a
descriptor: the `/models` path beside the `/chat/completions` it already
declares. `/chat/completions` gives `/models`, `/v1/chat/completions`
gives `/v1/models`, `/api/v2/chat/completions` gives `/api/v2/models`.
That is a rule applied to the endpoint's own declaration, not a pinned
URL. Discovery stays on demand. The anthropic, litellm and gemini
transports derive nothing: their model lists are other shapes, each with
its own headers and field names.

Verified live against an OpenAI-compatible endpoint whose /v1/models had
never been requested: served 427 models in 1164 ms, 140 chat-capable
after isReportedChatCapable, including an id that could previously only
be reached by typing it. custom-endpoints-loader.test.ts pins the four
derivations and the three formats that derive nothing.
A keyless endpoint scoped in the picker was told it had no credentials,
and the notice sent its user looking for an environment variable they
correctly do not have.

Cause: discoverProviderModelsCatalog (providers/model-discovery.ts)
gated on hasAuthHeader alone. For authScheme "none" the authority
answers static headers with no auth header by design
(api-key-credential.ts:362), which that gate cannot distinguish from an
unset key. The gate now exempts the scheme, which DECLARES that no
credential was ever expected rather than that one is missing.

Reachable only now that a custom endpoint derives a discovery descriptor
at all: before, a keyless endpoint carried no modelDiscovery, so the
gate was never reached.

model-discovery-models-catalog.test.ts covers the scheme beside the
blank-credential test it was being confused with. Reverting the one-line
change fails it with kind "failed".
Both the architecture note and the settings reference said the models
field "restricts the endpoint to listed models". No code has ever
enforced that: --model my-endpoint@unlisted-name reaches the endpoint
whichever ids are listed.

The array's only consumer is discoverableModelIds in proxy-server.ts,
which builds the proxy's own /v1/models response — the list the nested
Claude Code's /model command reads. Both entries now say that, and say
what it is not: neither an allowlist nor the picker's list, which the
endpoint now answers for itself.

Verified by grep: no read of the field outside that route.
@anandghegde
anandghegde force-pushed the feat/custom-endpoint-model-discovery branch from d93bc59 to 6318ff3 Compare October 14, 2026 07:33
@erudenko erudenko self-assigned this Oct 20, 2026
@erudenko

Copy link
Copy Markdown
Member

Thanks for this. It now has one conflict with main (7e83473). Could you rebase onto current main (0306fec) by 2026-10-28? This PR allows maintainer edits, so if you are busy we can rebase it ourselves after that date.

# Conflicts:
#	packages/cli/src/providers/custom-endpoints-loader.ts
A bundled predefined row compiles to the same complex definition as a
custom endpoint, so it now carries a derived openai-models-list
descriptor and the picker asks the vendor's own /models. The new test in
custom-endpoints-loader.test.ts loads a synthetic predefined row and
asserts the descriptor is the sibling of its apiPath
(/openai/v1/chat/completions -> /openai/v1/models). The catalog-row
fallback on a failed request is already pinned by STATE D in
model-selector-discovery.test.ts.

Also drops an issue number and change-history narration from the
comments in custom-endpoints-loader.ts and model-discovery.ts.

Co-Authored-By: Magus <[email protected]>

Crafted with agentic harness Magus (https://github.com/MadAppGang/magus)
Claude-Session: https://claude.ai/code/session_01XTrcSehh8dM3tNFWKVB138
@erudenko
erudenko merged commit e73954a into MadAppGang:main Nov 9, 2026
13 checks passed
@erudenko

erudenko commented Nov 9, 2026

Copy link
Copy Markdown
Member

Merged. A custom endpoint showing an empty list in the picker was a bad first experience, and asking the endpoint for its own models is the right answer.

My commit on top fixes the import that broke after main moved, adds a test that a predefined endpoint's row gets discovered too, and drops an issue number from a comment.

The release review caught two more. Discovery returns complete wire ids, but the handler still put modelPrefix in front, so an endpoint with modelPrefix: "gpt-" sent gpt-gpt-6.1-sol. I tried stripping the prefix in the picker first. Then the availability check, the launch context window and Test All each disagreed about which name they had. So in v14.17.0 an endpoint with a modelPrefix gets no discovered list, same as before your PR, and every endpoint without one gets the full feature. If you want discovery for prefixed endpoints, it needs one place that turns a wire id into a callable name, and I'd review that gladly. The other one: an endpoint on the Responses API (apiPath: "/v1/responses") got no list at all, so it reads /v1/models now too. All of it ships in v14.17.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants