Skip to content

fix(models): use catalog for models list --all#39678

Closed
sd1114820 wants to merge 4 commits intoopenclaw:mainfrom
sd1114820:fix/models-list-all-use-catalog
Closed

fix(models): use catalog for models list --all#39678
sd1114820 wants to merge 4 commits intoopenclaw:mainfrom
sd1114820:fix/models-list-all-use-catalog

Conversation

@sd1114820
Copy link
Copy Markdown

Summary

openclaw models list --all was reading the raw runtime registry (discoverModels(...).getAll()) instead of the OpenClaw model catalog.

That meant synthetic catalog fallback entries from src/agents/model-catalog.ts (for example openai-codex/gpt-5.4, openai/gpt-5.4, and openai/gpt-5.4-pro) could exist in the official catalog but still not appear in CLI output.

This change makes models list --all use loadModelCatalog({ config: cfg }) for the list source, while still reusing runtime registry discovery for availability/auth hints and local filtering.

What changed

  • switched the --all code path in src/commands/models/list.list-command.ts from raw registry getAll() to loadModelCatalog()
  • preserved availability/auth behavior via availableKeys + provider fallback heuristics
  • preserved --local behavior by filtering against discovered runtime models when needed
  • added a focused test covering synthetic catalog-only openai-codex/gpt-5.4 visibility in --all output

Why

Official upstream already defines GPT-5.4 synthetic fallbacks in src/agents/model-catalog.ts, but models list --all bypassed that layer entirely. This caused a mismatch between the official catalog and CLI output.

Validation

Validated in a container (no host dependency pollution) with:

  • src/commands/models/list.list-command.forward-compat.test.ts
  • src/commands/models.list.auth-sync.test.ts
  • src/commands/models/list.status.test.ts

Result: 15/15 tests passed.

@openclaw-barnacle openclaw-barnacle bot added commands Command implementations size: S labels Mar 8, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 8, 2026

Greptile Summary

This PR fixes models list --all so it reads from the OpenClaw model catalog (loadModelCatalog) rather than the raw runtime registry (discoverModels(...).getAll()), ensuring synthetic catalog fallback entries (e.g. openai-codex/gpt-5.4) are visible in CLI output. The non---all path and availability/auth hinting logic are preserved.

Issues found:

  • Runtime crash on catalog entries with no input field (list.list-command.ts:96–99): ModelCatalogEntry.input is optional (ModelInputType[] | undefined). For catalog-only entries without a discovered runtime counterpart, the fallback object { ...entry, baseUrl: "" } is cast to Model<Api> and passed to toModelRow, which calls model.input.join("+") unconditionally. If input is undefined the call throws a TypeError at runtime. The new test does not catch this because its mock always supplies input: ["text"]. A simple input: entry.input ?? [] guard in the fallback object fixes this.
  • Test assertion targets wrong invocation (list.list-command.forward-compat.test.ts:151): The existing "does not mark configured codex model as missing" test uses mock.calls[0] to read printModelTable output. With the new test inserted before it, calls[0] now refers to the first test's --all call, not this test's own call. The assertions pass coincidentally, but the test is no longer verifying what it claims to verify. Changing to .at(-1) (consistent with the rest of the suite) restores the intent.

Confidence Score: 3/5

  • The core catalog-switch logic is sound, but a runtime crash exists for catalog entries whose input field is absent.
  • The main behavioral fix is correct and well-scoped. However, the fallback model object spread from ModelCatalogEntry can carry input: undefined, which will throw in toModelRow for any real-world catalog entry lacking an input value. This path is reachable for synthetic fallbacks or opt-in provider models that omit the input field, so the risk is not theoretical. The test isolation issue is minor but further reduces confidence in test coverage.
  • src/commands/models/list.list-command.ts — the fallback model object construction at lines 96-99 needs an input null-guard before merge.

Comments Outside Diff (1)

  1. src/commands/models/list.list-command.forward-compat.test.ts, line 151 (link)

    Test silently asserts the wrong invocation

    The newly-added test runs first and records a printModelTable call at index 0. This existing test then reads calls[0], which now points to the first test's --all invocation rather than this test's own invocation. The assertions pass coincidentally, but the test is no longer verifying its own call.

    Use .at(-1) (consistent with the rest of the suite) so this test always references its own invocation:

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: src/commands/models/list.list-command.forward-compat.test.ts
    Line: 151
    
    Comment:
    **Test silently asserts the wrong invocation**
    
    The newly-added test runs first and records a `printModelTable` call at index `0`. This existing test then reads `calls[0]`, which now points to the first test's `--all` invocation rather than this test's own invocation. The assertions pass coincidentally, but the test is no longer verifying its own call.
    
    Use `.at(-1)` (consistent with the rest of the suite) so this test always references its own invocation:
    
    
    
    How can I resolve this? If you propose a fix, please make it concise.

Last reviewed commit: 9ac774f

Comment on lines +96 to +99
model: (discoveredModel ?? {
...entry,
baseUrl: "",
}) as Model<Api>,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Runtime crash when catalog entry has no input field

ModelCatalogEntry.input is typed as ModelInputType[] | undefined (optional). When discoveredModel is undefined (i.e. the entry only exists in the catalog, not in the runtime registry), the fallback object { ...entry, baseUrl: "" } may carry input: undefined. This value is then cast to Model<Api> and passed to toModelRow, which unconditionally calls model.input.join("+") on line 162 of list.registry.ts — with no null-guard. This will throw TypeError: Cannot read properties of undefined (reading 'join') for any catalog-only entry that has no input in its catalog record (e.g. synthetic fallbacks derived from a template that lacked an input field, or opt-in provider models without an input value).

The existing test doesn't catch this because the mock always provides input: ["text"].

Suggested change
model: (discoveredModel ?? {
...entry,
baseUrl: "",
}) as Model<Api>,
model: (discoveredModel ?? {
...entry,
baseUrl: "",
input: entry.input ?? [],
}) as Model<Api>,
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/commands/models/list.list-command.ts
Line: 96-99

Comment:
**Runtime crash when catalog entry has no `input` field**

`ModelCatalogEntry.input` is typed as `ModelInputType[] | undefined` (optional). When `discoveredModel` is `undefined` (i.e. the entry only exists in the catalog, not in the runtime registry), the fallback object `{ ...entry, baseUrl: "" }` may carry `input: undefined`. This value is then cast to `Model<Api>` and passed to `toModelRow`, which unconditionally calls `model.input.join("+")` on line 162 of `list.registry.ts` — with no null-guard. This will throw `TypeError: Cannot read properties of undefined (reading 'join')` for any catalog-only entry that has no `input` in its catalog record (e.g. synthetic fallbacks derived from a template that lacked an `input` field, or opt-in provider models without an `input` value).

The existing test doesn't catch this because the mock always provides `input: ["text"]`.

```suggestion
          model: (discoveredModel ?? {
            ...entry,
            baseUrl: "",
            input: entry.input ?? [],
          }) as Model<Api>,
```

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6eabe656f4

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".


if (opts.all) {
const sorted = [...models].toSorted((a, b) => {
const catalog = await loadModelCatalog({ config: cfg });
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fallback to discovered models when catalog is empty

Using loadModelCatalog as the sole source in --all can incorrectly hide all models when the catalog loader transiently fails and returns [] (it intentionally swallows errors in src/agents/model-catalog.ts and may return an empty list). In that case loadModelRegistry has already succeeded and provided discovered models, but this path still prints No models found, which is a user-visible regression from the previous behavior where --all listed runtime-discovered models.

Useful? React with 👍 / 👎.

@dorukardahan
Copy link
Copy Markdown
Contributor

i opened #40160 as a smaller follow-up on latest main. it keeps discovered rows when catalog lookup returns [], but still fills synthetic catalog rows like openai-codex/gpt-5.4. i think it supersedes this one: #40160

@steipete
Copy link
Copy Markdown
Contributor

steipete commented Mar 8, 2026

Closing as superseded by #40160.

That PR lands the same models list --all synthetic-catalog visibility fix in a smaller/current shape on top of latest main.
EOF && gh pr close 39678 --repo openclaw/openclaw

@steipete steipete closed this Mar 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

commands Command implementations size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants