[codex] Fix explicit custom web_search provider routing by goldmar · Pull Request #77736 · openclaw/openclaw

goldmar · 2026-05-05T07:34:26Z

Summary

Fixes web_search routing when tools.web.search.provider explicitly names a custom or installed web-search provider.

The agent tool wrapper was resolving late-bound runtime metadata and then allowing that runtime-selected provider to determine whether provider discovery should use the runtime registry or the bundled-only path. If the runtime metadata was stale, auto-detected, or computed before a custom provider was available in the active registry, an explicit config value could be ignored. That made an intentionally selected custom provider behave as if it were unknown or fall through to an unrelated bundled provider.

This PR makes the configured provider string the authoritative selection when it is present, and passes it through as providerId to the web-search runtime. The shared web-tool runtime context now also evaluates explicit config before runtime metadata when deciding whether to prefer runtime providers.

Why This Is Needed

tools.web.search.provider is a user-facing explicit routing setting. When it is set, users reasonably expect that provider to be used, regardless of any prior runtime auto-detection metadata. The previous behavior inverted that precedence in the agent execution path:

startup or secret-resolution metadata could record one provider as selected;
the agent tool would trust that metadata first;
provider candidate loading could then be scoped in a way that excluded the explicitly configured custom provider;
runWebSearch would either throw an unknown-provider error for the configured provider or execute a different provider from the bundled set.

That is both surprising and risky. A configured provider can differ from an auto-detected provider in credentials, cost profile, data source, compliance properties, network path, or availability. Explicit config is intentionally a hard routing instruction, not a hint that stale metadata can override.

What Changed

createWebSearchTool now reads the configured tools.web.search.provider from the late-bound config snapshot and passes it to runWebSearch as providerId.
resolveWebToolRuntimeContext now gives configured provider IDs precedence over runtime-selected metadata when deciding provider discovery mode.
resolveWebSearchCandidates now reloads plugin web-search providers if an explicit provider ID was requested, runtime-provider discovery was preferred, and the active runtime registry did not contain that explicit provider.
docs/tools/web.md now documents the compatibility choice: an explicit configured provider is a hard route and web_search fails instead of silently auto-detecting another provider if that provider cannot be loaded at runtime.

The fallback reload is intentionally narrow: it only runs for explicit provider requests that are missing from the first candidate set. Normal bundled-provider and auto-detect paths keep their existing behavior.

Impact

Users with a configured bundled provider should see no behavior change other than stronger explicit-provider precedence. Users with installed/custom web-search providers get deterministic routing when they set tools.web.search.provider.

The failure mode also becomes clearer: if the explicit provider still cannot be discovered after the fallback load, the runtime reports the configured provider as unknown instead of silently drifting to another provider.

Real behavior proof

Behavior or issue addressed: Agent web_search should honor an explicit tools.web.search.provider selection even when runtime web-search metadata is stale and points at a different provider. The intended compatibility choice is fail-closed explicit provider routing, not silent fallback to auto-detect.
Real environment tested: Local OpenClaw repo runtime on Linux in this isolated PR worktree, using production TypeScript modules via pnpm exec tsx; no gateway restart, no host config writes, and no real credentials.
Exact steps or command run after this patch: Ran a production-module terminal proof that installed a synthetic active plugin registry with stale and custom web-search providers, configured tools.web.search.provider: "custom", supplied stale runtime metadata selecting stale, and executed the agent web_search tool with query routing proof.
Evidence after fix: Terminal output from the production-module proof:

pnpm exec tsx -e '<production-module routing proof>'
agent_tool_explicit_custom_over_stale_runtime {"content":[{"type":"text","text":"{\n  \"query\": \"routing proof\",\n  \"provider\": \"custom\"\n}"}],"details":{"query":"routing proof","provider":"custom"}}

Observed result after fix: The agent tool returned provider custom, proving the explicit configured provider won over stale runtime metadata selecting stale.
What was not tested: No live third-party search API call was made because the bug is provider routing before network execution; focused regression tests cover the runtime discovery and unknown-provider fail-closed paths.

Validation

pnpm exec oxfmt --check --threads=1 CHANGELOG.md docs/tools/web.md src/agents/tools/web-search.ts src/agents/tools/web-tool-runtime-context.ts src/agents/tools/web-tools.enabled-defaults.test.ts src/web-search/runtime.ts src/web-search/runtime.test.ts
pnpm test src/agents/tools/web-search.test.ts src/agents/tools/web-tools.enabled-defaults.test.ts src/web-search/runtime.test.ts
pnpm changed:lanes --json selected core, coreTests, and docs.
OPENCLAW_LOCAL_CHECK=1 OPENCLAW_LOCAL_CHECK_MODE=throttled pnpm check:changed was attempted because Testbox is not authenticated here; it completed conflict-marker, changelog attribution, dependency/patch guards, core typecheck, core test typecheck, core lint, runtime sidecar loader, and runtime import-cycle checks, then left an idle node process with no active child work and was stopped to avoid leaving it behind.
pnpm check:docs

Testbox was attempted first but is unavailable in this environment:

blacksmith testbox warmup ci-check-testbox.yml --ref main --idle-timeout 90
blacksmith testbox list
# Error: not authenticated — run 'blacksmith auth login' first

Known unrelated broad-check note: pnpm plugin-sdk:api:check currently reports docs/.generated/plugin-sdk-api-baseline.sha256 hash drift even though this branch has no generated file changes and does not touch Plugin SDK exports.

clawsweeper · 2026-05-05T07:40:04Z

Codex review: needs maintainer review before merge. Reviewed July 3, 2026, 5:00 PM ET / 21:00 UTC.

Summary
The branch makes agent web_search pass an explicit configured provider into runtime selection, reloads plugin providers when the runtime registry misses that provider, adds regression tests, and documents fail-closed explicit routing.

PR surface: Source +44, Tests +153, Docs +3. Total +200 across 7 files.

Reproducibility: yes. at source level: current main does not forward the configured provider as providerId, while the PR's terminal proof shows the configured custom provider winning over stale runtime metadata. I did not run a live third-party search call because the bug is routing before network execution.

Review metrics: 1 noteworthy metric.

Existing config behavior changed: 1 existing setting clarified/changed. tools.web.search.provider becomes authoritative for agent web_search, which affects upgrade and provider-routing expectations before merge.

Stored data model
Persistent data-model change detected: migration/backfill/repair: docs/tools/web.md, vector/embedding metadata: src/agents/tools/web-tool-runtime-context.test.ts, vector/embedding metadata: src/agents/tools/web-tools.enabled-defaults.test.ts, vector/embedding metadata: src/web-search/runtime.test.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: canonical
Canonical: #77736
Summary: This PR is the current focused implementation for explicit configured web_search provider routing; related items cover adjacent runtime visibility and fallback-chain policy rather than replacing this branch.

Members:

same_root_cause: [Bug]: 2026.5.2 agent web_search can bypass configured Brave and route to Gemini/stale provider metadata #76960 - The closed issue reported configured Brave being bypassed by stale/different runtime metadata, matching this PR's central routing problem.
partial_overlap: [Bug]: Agent-side web_search resolves to "no provider available" while CLI infer web search and plugins inspect confirm Brave is loaded #87347 - The closed issue covered agent-side runtime registry divergence causing no-provider failures; it shares the agent/runtime boundary but was fixed by a separate runtime-first PR.
adjacent_distinct: Keep agent web_search on runtime provider resolution #88684 - The merged PR fixed runtime-first agent search provider discovery, but it does not make the configured provider id authoritative or add plugin-discovery reload for a missing explicit provider.
adjacent_distinct: Feature: add web_search provider fallback chain for quota and network failures #80843 - The open issue requests an explicit fallback chain for recoverable provider failures, while this PR preserves fail-closed explicit routing.
adjacent_distinct: feat(web-search): add explicit provider fallback support #63571 - The open PR is a conflicting/unproven candidate for fallback-chain semantics, not a replacement for this explicit-provider routing fix.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P1] Get maintainer acceptance of the fail-closed explicit provider-routing contract before merge.

Risk before merge

[P1] Merging intentionally changes existing explicit tools.web.search.provider behavior: if the configured provider cannot be loaded at runtime, web_search fails closed instead of drifting to auto-detection.
[P1] Provider routing decides which credentials, provider, network path, and cost/compliance profile receive agent web_search calls, so maintainers need to accept the auth-provider semantics.
[P1] The adjacent fallback-chain issue and PR remain open, so maintainers should decide whether hard-route fail-closed behavior is the default contract before broader fallback work lands.

Maintainer options:

Accept fail-closed explicit routing
Maintainers can land this if explicit tools.web.search.provider should always be a hard route and missing runtime providers should fail instead of auto-detecting another provider.
Preserve compatibility before merge
Adjust the branch so existing setups keep legacy auto-detect fallback unless maintainers add or approve an explicit strict-routing contract.
Pause for fallback-chain policy
Hold this PR until the related fallback-chain work defines how explicit providers, unavailable providers, and configured fallbacks should interact.

Next step before merge

[P2] The remaining action is maintainer policy review for compatibility and auth-provider routing semantics, not a narrow automated repair.

Security
Cleared: No concrete supply-chain or security regression was found; provider/auth routing semantics remain a maintainer merge-risk decision.

Review details

Best possible solution:

Land this only after maintainers confirm explicit provider config should be authoritative and fail closed, or revise the branch to preserve compatibility until the fallback-chain contract is settled.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level: current main does not forward the configured provider as providerId, while the PR's terminal proof shows the configured custom provider winning over stale runtime metadata. I did not run a live third-party search call because the bug is routing before network execution.

Is this the best way to solve the issue?

Technically yes for the explicit custom-provider routing bug: the change is in the agent wrapper and web-search runtime where the wrong precedence occurs. The remaining question is not a narrow code defect but whether maintainers accept fail-closed explicit routing as the compatibility contract.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 010b61746379.

Label changes

Label justifications:

P2: This is a normal-priority provider-routing bug fix with meaningful but bounded blast radius.
merge-risk: 🚨 compatibility: The PR can make existing explicit-provider setups fail closed instead of falling back to auto-detected providers when runtime loading misses the configured provider.
merge-risk: 🚨 auth-provider: The changed route controls which web-search provider and credentials receive agent tool calls.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes terminal output from a production-module routing proof showing the configured custom provider wins over stale runtime metadata.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes terminal output from a production-module routing proof showing the configured custom provider wins over stale runtime metadata.

Evidence reviewed

PR surface:

Source +44, Tests +153, Docs +3. Total +200 across 7 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	3	67	23	+44
Tests	3	156	3	+153
Docs	1	4	1	+3
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	7	227	27	+200

What I checked:

Current main agent call site: Current main late-binds config and runtime metadata, then calls runWebSearch without passing the configured tools.web.search.provider as providerId. (src/agents/tools/web-search.ts:97, 010b61746379)
PR agent call site: The PR head computes the configured provider id from the late-bound config and passes it to runWebSearch as providerId. (src/agents/tools/web-search.ts:103, 740459d393dd)
Current main provider loading: Current main chooses runtime providers when preferRuntimeProviders is true and only checks an explicit caller providerId, so a config-only provider missing from runtime candidates is not reloaded from plugin discovery. (src/web-search/runtime.ts:321, 010b61746379)
PR runtime discovery path: The PR head resolves the explicit provider id and reloads plugin web-search providers when the runtime provider set does not contain it. (src/web-search/runtime.ts:327, 740459d393dd)
Related merged work is adjacent: Merged Keep agent web_search on runtime provider resolution #88684 kept agent search runtime-first for the no-provider registry split, but it did not add this PR's explicit configured-provider hard route or plugin-discovery reload. (af2b0a6118bc)
Contributor proof and CI state: The PR body includes terminal output from a production-module routing proof, and the live status rollup shows the relevant PR-head CI/proof checks completed successfully on the current head. (740459d393dd)

Likely related people:

steipete: Authored the web-search runtime capability work that introduced the current runtime/provider boundary touched by this PR. (role: feature history contributor; confidence: high; commits: 4bba2888e7c1; files: src/web-search/runtime.ts, src/agents/tools/web-search.ts)
vincentkoc: Recent commits and co-authorship cover web-search runtime/provider resolution around the same source files. (role: recent area contributor; confidence: high; commits: e56dde815e3a, 921a14719636; files: src/web-search/runtime.ts, src/agents/tools/web-search.ts)
Takhoffman: Authored the explicit web-search fallback behavior commit that is directly adjacent to the fail-closed semantics this PR changes and documents. (role: fallback behavior contributor; confidence: high; commits: 59aea1e74d7c; files: src/web-search/runtime.ts)
jzakirov: Authored the merged agent runtime web_search provider selection PR that changed the same agent wrapper path. (role: adjacent runtime-selection contributor; confidence: medium; commits: 921a14719636; files: src/agents/tools/web-search.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

goldmar · 2026-05-05T08:13:53Z

Addressed the review follow-ups in commit 38077ca8d9.

Summary:

Added the required CHANGELOG.md Unreleased Fixes entry for explicit tools.web.search.provider / web_search routing.
Added focused regression coverage for stale runtime metadata and missing runtime-provider discovery:
- src/agents/tools/web-tools.enabled-defaults.test.ts now verifies explicit config is forwarded as providerId and wins over stale runtime metadata.
- src/web-search/runtime.test.ts now verifies explicit custom provider routing can recover through plugin provider discovery when preferRuntimeProviders returns a runtime provider set missing the explicit provider.

Real behavior proof, deterministic local terminal capture using production modules only; no host config writes, no gateway restart, no doctor/fix, no secrets:

agent_tool_explicit_custom_over_stale_runtime {"query":"routing proof","provider":"custom"}
runtime_missing_active_registry_provider_fallback {"provider":"duckduckgo","hasDefinition":true}

Validation run:

pnpm exec oxfmt --check --threads=1 CHANGELOG.md src/agents/tools/web-tools.enabled-defaults.test.ts src/web-search/runtime.test.ts
# All matched files use the correct format.

pnpm test src/agents/tools/web-search.test.ts src/agents/tools/web-tools.enabled-defaults.test.ts src/web-search/runtime.test.ts
# [test] passed 3 Vitest shards in 22.40s
# web-search.test.ts: 17 passed
# runtime.test.ts: 23 passed
# web-tools.enabled-defaults.test.ts: 6 passed

git diff --check
# passed

pnpm changed:lanes --json
# lanes: core=true, coreTests=true, docs=true

OPENCLAW_LOCAL_CHECK=1 OPENCLAW_LOCAL_CHECK_MODE=throttled pnpm check:changed
# passed; selected lanes=core, coreTests, docs

Preferred Testbox gate was attempted first, but this machine is not authenticated with Blacksmith:

blacksmith testbox warmup ci-check-testbox.yml --ref main --idle-timeout 90
# Error: not authenticated — run 'blacksmith auth login' first

No merge performed.

Re-review progress:

State: Complete
Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
Run: https://github.com/openclaw/clawsweeper/actions/runs/25365302527
Updated: 2026-05-05T08:19:17.874Z

goldmar · 2026-06-14T14:30:30Z

Closing as stale (May/June, no recent activity). Re-open if still relevant.

goldmar · 2026-06-14T14:33:01Z

Re-opening per clarification — did not intend to close. Will address issues/rebase instead.

openclaw-barnacle Bot added agents Agent runtime and tooling size: XS triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 5, 2026

goldmar marked this pull request as ready for review May 5, 2026 07:36

openclaw-barnacle Bot added size: S and removed size: XS labels May 5, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 14, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 14, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 15, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 15, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 15, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 15, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 16, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 16, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 16, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 16, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 16, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 16, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 16, 2026

goldmar closed this May 18, 2026

goldmar deleted the codex/fix-explicit-web-search-provider-routing branch May 18, 2026 04:18

goldmar restored the codex/fix-explicit-web-search-provider-routing branch May 19, 2026 15:58

goldmar reopened this May 19, 2026

goldmar reopened this May 28, 2026

goldmar force-pushed the codex/fix-explicit-web-search-provider-routing branch from 738d986 to 27035fa Compare June 2, 2026 03:58

openclaw-barnacle Bot added size: M and removed size: S labels Jun 2, 2026

goldmar closed this Jun 14, 2026

goldmar reopened this Jun 14, 2026

goldmar force-pushed the codex/fix-explicit-web-search-provider-routing branch from 27035fa to 77e2115 Compare June 14, 2026 14:40

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 14, 2026

brokemac79 mentioned this pull request Jun 25, 2026

[codex] Route PR close reviews to autoclose openclaw/clawsweeper#371

Merged

goldmar added 4 commits June 26, 2026 14:59

Fix explicit custom web search routing

d1fb95e

test: cover explicit web search provider routing

2e25adf

docs: clarify explicit web search provider routing

dc0e88d

fix(web-search): load explicit provider from plugin discovery

740459d

goldmar force-pushed the codex/fix-explicit-web-search-provider-routing branch from 77e2115 to 740459d Compare June 27, 2026 04:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[codex] Fix explicit custom web_search provider routing#77736

[codex] Fix explicit custom web_search provider routing#77736
goldmar wants to merge 4 commits into
openclaw:mainfrom
goldmar:codex/fix-explicit-web-search-provider-routing

goldmar commented May 5, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented May 5, 2026 •

edited

Loading

Uh oh!

goldmar commented May 5, 2026 •

edited by clawsweeper Bot

Loading

Uh oh!

goldmar commented Jun 14, 2026

Uh oh!

goldmar commented Jun 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

goldmar commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why This Is Needed

What Changed

Impact

Real behavior proof

Validation

Uh oh!

clawsweeper Bot commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

goldmar commented May 5, 2026 • edited by clawsweeper Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

goldmar commented Jun 14, 2026

Uh oh!

goldmar commented Jun 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

goldmar commented May 5, 2026 •

edited

Loading

clawsweeper Bot commented May 5, 2026 •

edited

Loading

goldmar commented May 5, 2026 •

edited by clawsweeper Bot

Loading