Skip to content

ModelRegistry: a single invalid plugin catalog aborts the entire custom-models load, leaving zero models and an unlogged error #92553

Description

@fxstein

Summary

ModelRegistry.loadCustomModels aborts the entire custom-models load when any single generated plugin catalog file (agents/<id>/agent/plugins/<plugin>/catalog.json) fails schema/config validation. One bad provider entry in one plugin catalog leaves the registry with zero models — including all valid models from models.json and every other plugin catalog.

The resulting user-facing failure is misleading: tools that resolve models (e.g. image/media understanding) report every candidate as Unknown model, while the real cause is only available via ModelRegistry.getError(), which nothing logs.

Observed on 2026.5.28; the code path is unchanged on current main (8d9ce35) and in 2026.6.1.

Symptom

All image models failed (4): google/gemini-3.1-pro-preview: Unknown model; anthropic/claude-opus-4-8: Unknown model; ...

Every configured model fails with Unknown model because registry.getAll() is empty. The actual error (from getError()):

Provider google-vertex, model gemini-3.1-pro-preview: no "api" specified. Set at provider or model level.

…is never surfaced in any log line or error message.

Root cause

In src/agents/sessions/model-registry.ts, the plugin-catalog loop propagates a per-catalog failure as a failure of the whole load:

const models = this.parseModels(configForUse);
if (options.includePluginCatalogs !== false) {
for (const pluginCatalog of listPluginModelCatalogFiles(dirname(modelsJsonPath))) {
const pluginResult = this.loadCustomModels(pluginCatalog.path, {
catalogPluginId: pluginCatalog.pluginId,
includePluginCatalogs: false,
requireGeneratedCatalog: true,
});
if (pluginResult.error) {
return pluginResult;
}
models.push(...pluginResult.models);
}

for (const pluginCatalog of listPluginModelCatalogFiles(dirname(modelsJsonPath))) {
  const pluginResult = this.loadCustomModels(pluginCatalog.path, { ... });
  if (pluginResult.error) {
    return pluginResult;   // <-- discards `models` accumulated so far; registry ends up empty
  }
  models.push(...pluginResult.models);
}

loadModels() then stores the error in this.loadError (exposed via getError()) but nothing in the runtime logs it:

private loadModels(): void {
// Load configured models and request settings from models.json plus
// generated plugin-owned catalog shards under the agent plugin state.
const { models: customModels, error } = this.modelsJsonPath
? this.loadCustomModels(this.modelsJsonPath)
: emptyCustomModelsResult();
if (error) {
this.loadError = error;
// Keep the prior empty/default registry shape when models.json failed to load.
}

The aggregated failure message in src/media-generation/runtime-shared.ts (All <capability> models failed (N): ..., L589) reports only per-model resolution errors and never consults registry.getError().

How a bad catalog entry gets there (and stays there)

The write-time merge in src/agents/models-config.ts (ensureOpenClawModelsJsonmergeGeneratedPluginCatalogProvidersIntoExistingParsed) preserves existing provider entries from previous generations indefinitely. A legacy entry written by an older plugin version (in our case a Vertex-era google-vertex provider with "apiKey": "<authenticated>" and no "api" field) is merged back into the generated catalog forever, and is never re-validated against the current schema. Once the registry-side validation tightened, that fossil entry started failing validation — and took the whole registry down with it.

Concrete trigger

agents/main/agent/plugins/google/catalog.json containing (alongside valid providers):

{
  "providers": {
    "google-vertex": {
      "baseUrl": "https://us-central1-aiplatform.googleapis.com/...",
      "apiKey": "<authenticated>",
      // no "api" field
      "models": [{ "id": "gemini-3.1-pro-preview", ... }]
    }
  }
}

validateConfig throws Provider google-vertex, model gemini-3.1-pro-preview: no "api" specified., the recursive loadCustomModels call returns that as CustomModelsResult.error, and the outer call returns it — dropping all previously parsed models.

Reproduction

  1. On any working install, add a provider entry without "api" (but with models) to any generated plugin catalog, e.g. agents/<id>/agent/plugins/google/catalog.json.
  2. Restart the gateway and invoke any tool that resolves models (e.g. send an image).
  3. Observe All image models failed (N): <provider>/<model>: Unknown model for every candidate; no log line mentions the catalog validation failure.

Or directly via a node probe against the dist:

// import the agent-model-discovery module from dist (bundled as sessions-*.js)
const auth = await discoverAuthStorage(agentDir);
const registry = await discoverModels(auth, agentDir);
console.log(registry.getError());        // -> the real schema error
console.log(registry.getAll().length);   // -> 0

Suggested fixes

  1. Failure isolation (main fix): in the plugin-catalog loop, skip just the invalid catalog/provider — collect its error (e.g. into a warnings/aggregated error field) and continue, instead of return pluginResult wiping the whole registry. A single plugin's stale shard shouldn't take down models from models.json and every other plugin.
  2. Surface the error: log loadError at WARN when loadModels() stores it, and/or append registry.getError() to the All <capability> models failed message when the registry is empty — Unknown model × N with the real cause hidden in an unlogged getter made this very expensive to debug.
  3. Generator hygiene: have ensureOpenClawModelsJson's merge validate preserved legacy provider entries against the current schema and drop (or at least flag) invalid ones, instead of merging them forward forever.

Happy to send a PR for (1)+(2) if the approach sounds right.

Environment

  • OpenClaw 2026.5.28 (gateway where this was hit); code path identical in 2026.6.1 and main @ 8d9ce35
  • Node 22, linux/arm64 (docker)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

Priority

None yet

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions