Skip to content

fix: filter implicit providers from final result when models.providers is configured#33402

Open
bhalliburton wants to merge 4 commits intoopenclaw:mainfrom
bhalliburton:fix/skip-implicit-providers-when-explicit-configured
Open

fix: filter implicit providers from final result when models.providers is configured#33402
bhalliburton wants to merge 4 commits intoopenclaw:mainfrom
bhalliburton:fix/skip-implicit-providers-when-explicit-configured

Conversation

@bhalliburton
Copy link
Copy Markdown

@bhalliburton bhalliburton commented Mar 3, 2026

Summary

When models.providers is explicitly configured, implicit providers (auto-detected from environment variables) are filtered out of the final provider list. This prevents:

  1. Confusing startup errors — e.g., Bedrock discovery failing on AWS_PROFILE set for an unrelated tool
  2. Silent cost leaks — e.g., ANTHROPIC_API_KEY in env acting as fallback, bypassing a configured proxy

Zero-config onboarding (no models.providers set) preserves current auto-detection behavior.

Important nuance

The implicit catalog is still consulted for model capability enrichment (context windows, reasoning flags, input modalities, additional model variants) on explicitly configured providers — only implicit provider registration is suppressed. Bedrock and Copilot discovery are fully skipped since they only add new providers, not enrich existing ones.

A future optimization could split resolveImplicitProviders() into catalog-only lookup (no network) vs. full discovery (with probes like Ollama's 5s HTTP timeout) to avoid unnecessary network calls when explicit providers are configured.

Changes

src/agents/models-config.ts — In resolveProvidersForModelsJson(), after the implicit/explicit merge, strip any provider not in the explicit config. Early-return skips Bedrock and Copilot discovery. Comment explains why resolveImplicitProviders() is still called (catalog enrichment).

src/agents/models-config.skips-implicit-when-explicit-configured.test.ts — 3 new tests:

  1. Explicit providers + MOONSHOT_API_KEY in env → only explicit providers appear
  2. Explicit providers + AWS_PROFILE in env → no amazon-bedrock discovery
  3. No explicit providers → implicit detection still works (preserves onboarding)

All 73 models-config tests pass with no regressions.

AI Assisted

Built with Claude Code on Bedrock (AI-assisted, fully tested). Lightly reviewed by human.

Fixes #33327

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4cfede78a7

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

// routing and confusing startup errors (e.g. Bedrock discovery from
// ambient AWS_PROFILE). Catalog enrichment for explicitly-configured
// providers is preserved via the merge above. See #33327.
const explicitKeys = new Set(Object.keys(explicitProviders));
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Normalize explicit provider IDs before implicit-provider filter

The new implicit-provider stripping uses raw Object.keys(explicitProviders) for membership checks, but mergeProviders() normalizes keys with trim(). If a config key has leading/trailing whitespace (allowed by the current z.record(z.string(), ...) schema), the merged provider is stored under the trimmed ID and then deleted here as “implicit-only,” which can leave providers empty and skip writing models.json even though the user configured a provider. Align this filter with the same key normalization used during merge.

Useful? React with 👍 / 👎.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 3, 2026

Greptile Summary

This PR fixes a real-world pain point: when models.providers is explicitly configured, ambient environment variables (e.g. AWS_PROFILE, ANTHROPIC_API_KEY) no longer cause implicit providers to bleed into the resolved provider set. The implementation adds a post-merge strip pass in resolveProvidersForModelsJson and an early return that bypasses Bedrock and Copilot discovery, while preserving catalog enrichment for explicitly-configured providers.

Key changes:

  • hasExplicitProviders guard strips any provider not in the explicit list after the implicit/explicit merge.
  • Early return prevents resolveImplicitBedrockProvider and resolveImplicitCopilotProvider from running when explicit providers are set.
  • Three new tests cover: implicit env vars with explicit config (e.g. MOONSHOT_API_KEYmoonshot absent), Bedrock suppression via AWS_PROFILE, and zero-config onboarding still working.

One performance concern: resolveImplicitProviders is still invoked unconditionally before the guard. In production environments (where VITEST/NODE_ENV=test short-circuit is absent), the Ollama discovery path issues a real HTTP probe to http://127.0.0.1:11434/api/tags with a 5-second timeout. When Ollama isn't running, this adds ~5 s of hidden startup latency even though the discovered result is immediately thrown away. The PR's goal is to "skip implicit provider auto-detection," but this is only partially achieved — Bedrock and Copilot discovery are skipped, but the Ollama (and other providers') probes inside resolveImplicitProviders still fire.

Confidence Score: 4/5

  • Safe to merge; the core fix is logically correct and well-tested, with one non-blocking startup performance concern.
  • The fix correctly solves its primary goal: explicit providers reliably prevent ambient implicit providers from appearing in the final result. The three new tests cover the main scenarios. The one concern is that resolveImplicitProviders is still invoked unconditionally before the guard, causing unnecessary network probes (especially the Ollama probe with a 5-second timeout) even when their results will be discarded. This adds ~5 seconds of startup latency in production environments where Ollama isn't running, but it doesn't affect correctness — only efficiency in specific scenarios.
  • src/agents/models-config.ts lines 122–141: Consider moving resolveImplicitProviders into the if (!hasExplicitProviders) block to avoid unnecessary network probes when explicit providers are configured.

Last reviewed commit: 4cfede7

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Comment on lines 122 to +141
const implicitProviders = await resolveImplicitProviders({ agentDir, explicitProviders });
const providers: Record<string, ProviderConfig> = mergeProviders({
implicit: implicitProviders,
explicit: explicitProviders,
});

if (hasExplicitProviders) {
// User has explicitly configured providers — strip any implicit-only
// providers that were not in the explicit list to avoid unexpected cost
// routing and confusing startup errors (e.g. Bedrock discovery from
// ambient AWS_PROFILE). Catalog enrichment for explicitly-configured
// providers is preserved via the merge above. See #33327.
const explicitKeys = new Set(Object.keys(explicitProviders));
for (const key of Object.keys(providers)) {
if (!explicitKeys.has(key)) {
delete providers[key];
}
}
return providers;
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ollama (and other) network probes still fire when explicit providers are set.

resolveImplicitProviders is called unconditionally at line 122, before the hasExplicitProviders guard at line 128. Inside that function, buildOllamaProvider always issues a real HTTP request to http://127.0.0.1:11434/api/tags (with a 5 s timeout) unless process.env.VITEST or NODE_ENV=test is set.

In a production environment where Ollama is not running, every startup with explicit providers configured will silently wait up to 5 seconds for the probe to time out, and then the discovered result is immediately discarded by the filter loop (lines 134–139). The same applies to other providers discovered via resolveImplicitProviders (Venice, vLLM, HuggingFace, etc.) — the discovery work is done but thrown away.

The stated goal of the PR is to skip implicit auto-detection when explicit providers are configured. Achieving that fully would require either:

  • Skipping the resolveImplicitProviders call entirely when hasExplicitProviders, or
  • Passing a hint to resolveImplicitProviders to skip network-bound discovery steps when explicit providers exist

Consider moving the resolveImplicitProviders call inside the if (!hasExplicitProviders) block to avoid unnecessary network probes in production.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/models-config.ts
Line: 122-141

Comment:
Ollama (and other) network probes still fire when explicit providers are set.

`resolveImplicitProviders` is called unconditionally at line 122, before the `hasExplicitProviders` guard at line 128. Inside that function, `buildOllamaProvider` always issues a real HTTP request to `http://127.0.0.1:11434/api/tags` (with a 5 s timeout) unless `process.env.VITEST` or `NODE_ENV=test` is set.

In a production environment where Ollama is not running, every startup with explicit providers configured will silently wait up to 5 seconds for the probe to time out, and then the discovered result is immediately discarded by the filter loop (lines 134–139). The same applies to other providers discovered via `resolveImplicitProviders` (Venice, vLLM, HuggingFace, etc.) — the discovery work is done but thrown away.

The stated goal of the PR is to *skip* implicit auto-detection when explicit providers are configured. Achieving that fully would require either:
- Skipping the `resolveImplicitProviders` call entirely when `hasExplicitProviders`, or
- Passing a hint to `resolveImplicitProviders` to skip network-bound discovery steps when explicit providers exist

Consider moving the `resolveImplicitProviders` call inside the `if (!hasExplicitProviders)` block to avoid unnecessary network probes in production.

How can I resolve this? If you propose a fix, please make it concise.

@bhalliburton
Copy link
Copy Markdown
Author

Thanks for the review! Good catch on the resolveImplicitProviders() still running unconditionally.

I investigated moving it into the !hasExplicitProviders block, but it breaks 3 existing tests — the implicit catalog provides model capability enrichment (context windows, reasoning flags, input modalities, additional model variants) that gets merged into explicitly configured providers via mergeProviderModels(). Skipping it entirely means explicit providers lose catalog refresh.

The ideal fix would be splitting resolveImplicitProviders() into catalog-only lookup (no network) vs. full discovery (with probes), but that's a larger refactor of models-config.providers.ts.

For now I've added a comment explaining why the call is still needed and noting the optimization opportunity. The core goal — preventing implicit providers from appearing in the final result and skipping Bedrock/Copilot discovery — is preserved.

@bhalliburton bhalliburton changed the title fix: skip implicit provider discovery when models.providers is configured fix: filter implicit providers from final result when models.providers is configured Mar 3, 2026
@openclaw-barnacle openclaw-barnacle bot added agents Agent runtime and tooling size: S labels Mar 3, 2026
…ured

When users explicitly configure models.providers, implicit provider
auto-detection from environment variables (AWS_PROFILE, GH_TOKEN,
MOONSHOT_API_KEY, etc.) is now skipped. This prevents:

- Confusing startup errors (e.g. Bedrock discovery failing on
  unrelated AWS_PROFILE)
- Silent cost leaks (e.g. ANTHROPIC_API_KEY acting as fallback,
  bypassing configured proxy)

Zero-config onboarding (no models.providers set) preserves current
auto-detection behavior.

Fixes openclaw#33327
Address review feedback: resolveImplicitProviders() is called even
with explicit providers because it provides built-in catalog
enrichment (model capabilities, context windows, reasoning flags).
Skipping it entirely breaks catalog refresh for explicitly configured
providers. Added comment noting that a future optimization could
split catalog lookup from network discovery to avoid unnecessary
probes (e.g. Ollama's 5s timeout).
@bhalliburton bhalliburton force-pushed the fix/skip-implicit-providers-when-explicit-configured branch from 89bf09f to 12c7cdc Compare March 5, 2026 16:42
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 12c7cdc477

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

delete providers[key];
}
}
return providers;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep Bedrock discovery for explicit Bedrock configs

When models.providers is non-empty, this early return skips the resolveImplicitBedrockProvider path below, so explicit amazon-bedrock configs no longer get discovery-based model refresh/merge. In setups that enable models.bedrockDiscovery (or rely on AWS credential auto-enable), newly discovered Bedrock models and capability updates stop propagating into models.json, even though the same explicit Bedrock config previously went through mergeProviderModels(implicitBedrock, existing) and stayed current.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed in 0f82cd0.

When amazon-bedrock or github-copilot is present in the explicit providers list, we now still run their respective discovery so mergeProviderModels can apply capability updates. The early return is moved to after those targeted discovery calls.

For providers like custom-proxy that aren't Bedrock/Copilot, the behavior is unchanged — no unnecessary discovery runs.

When models.providers is non-empty and includes amazon-bedrock or
github-copilot, still run their respective discovery so those providers
receive model refresh and capability updates via mergeProviderModels.

Previously the early return skipped all discovery for explicit configs,
which broke bedrockDiscovery model propagation for users who explicitly
list amazon-bedrock in their providers config.

Fixes the concern raised in code review: openclaw#33402

Adds a test: explicit amazon-bedrock config is preserved and discovery
still runs when that provider is in the explicit list.
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0f82cd0679

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +140 to +144
const explicitKeys = new Set(Object.keys(explicitProviders));
for (const key of Object.keys(providers)) {
if (!explicitKeys.has(key)) {
delete providers[key];
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Drop stale implicit providers when explicit config is present

In the explicit-provider path, this deletion only affects the in-memory providers map before resolveProvidersForMode() runs, but default mode is merge, and mergeWithExistingProviderSecrets() copies all existing models.json providers back in. That means users who already have an implicitly discovered provider (for example from earlier zero-config runs) will keep it even after switching to explicit models.providers, so the new guard against unintended provider routing does not actually take effect for existing installs.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed in f479b90.

Added a post-merge filter in the main ensureOpenClawModelsJson path: after resolveProvidersForMode() merges with existing models.json, we now re-apply the explicit-provider filter to strip any stale implicit providers that were re-introduced from previous zero-config runs.

Also added trim() normalization on explicit provider keys to match the merge step's normalization, addressing the P2 whitespace concern.

Updated the "merges providers by default" test to expect the corrected behavior: non-explicit providers in models.json are now properly stripped. 74/74 tests passing.

When explicit providers are configured and mode is 'merge',
mergeWithExistingProviderSecrets() re-introduces implicit providers
that were previously written to models.json from earlier zero-config
runs. This adds a post-merge filter that strips non-explicit providers,
ensuring the explicit-provider intent from resolveProvidersForModelsJson
is honored end-to-end.

Also normalizes explicit provider keys with trim() to match the
normalization used during the merge step.

Addresses review feedback from Codex bot on openclaw#33402. See openclaw#33327.
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f479b90759

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +140 to +143
const explicitKeys = new Set(Object.keys(explicitProviders));
for (const key of Object.keys(providers)) {
if (!explicitKeys.has(key)) {
delete providers[key];
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Trim explicit provider keys before filtering merged providers

Use normalized provider IDs for this membership check. mergeProviders() trims config keys, so an explicit entry like " amazon-bedrock " is stored under "amazon-bedrock", but explicitKeys here keeps the untrimmed key and deletes the provider as “implicit-only.” In that case resolveProvidersForModelsJson() can return an empty map and ensureOpenClawModelsJson() exits early without writing models.json, effectively dropping a user-configured provider.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implicit provider auto-discovery silently merges with explicit config — can cause unexpected costs and routing

1 participant