feat: add Tavily as a `web_search` provider by lakshyaag-tavily · Pull Request #44343 · openclaw/openclaw

lakshyaag-tavily · 2026-03-12T19:29:46Z

Summary

Problem: OpenClaw's web_search tool supports Brave, Gemini, Grok, Kimi, and Perplexity but not Tavily Search API.
Why it matters: Tavily provides structured search results with optional AI-generated answer summaries and configurable search depth (basic/advanced).
What changed: Added Tavily as a new web_search provider with full integration: API key resolution (TAVILY_API_KEY / config), auto-detection, tool schema with Tavily-specific parameters (search_depth, include_answer, domain_filter), freshness/date filtering, config validation (Zod schema + types), onboarding wizard entry, docs (EN + zh-CN), and comprehensive tests.
What did NOT change (scope boundary): No changes to existing providers (Brave, Gemini, Grok, Kimi, Perplexity). No changes to web_fetch. No changes to the UI dashboard or gateway server logic.

Change Type (select all)

Scope (select all touched areas)

User-visible / Behavior Changes

New web_search provider option: "tavily".
New config section: tools.web.search.tavily with apiKey and baseUrl fields.
New env var support: TAVILY_API_KEY.
Auto-detection: if TAVILY_API_KEY is set and no higher-priority provider key is found, Tavily is auto-selected.
Tavily-specific tool parameters: search_depth (basic/advanced), include_answer (boolean), domain_filter (supports mixed allowlist + denylist).
Tavily allows up to 20 results per query (vs 10 for other providers).
Onboarding wizard now includes Tavily in the search provider picker.

Security Impact (required)

New permissions/capabilities? No
Secrets/tokens handling changed? Yes - new TAVILY_API_KEY env var and tools.web.search.tavily.apiKey config field, following the same SecretInput / SecretRef pattern as all other providers.
New/changed network calls? Yes - new outbound POST to https://api.tavily.com/search (or user-configured baseUrl), using the existing withTrustedWebSearchEndpoint dispatcher (proxy-aware, sandboxed).
Command/tool execution surface changed? Yes - web_search tool gains Tavily as a provider option with three new schema parameters.
Data access scope changed? No
Risk + mitigation: The Tavily API key is handled identically to existing provider keys (SecretRef, env fallback, sensitive registration). Network calls go through the existing trusted-endpoint dispatcher with proxy support and timeout enforcement. All external content is wrapped with untrusted: true markers.

Repro + Verification

Environment

OS: macOS
Runtime/container: Node 22+ / Bun
Model/provider: N/A (tool-level change)
Integration/channel (if any): N/A
Relevant config (redacted): { tools: { web: { search: { provider: "tavily", tavily: { apiKey: "tvly-..." } } } } }

Steps

Set TAVILY_API_KEY=tvly-... in environment (or configure via openclaw configure --section web).
Run web_search with query: "test query".
Verify structured results with title, url, snippet fields are returned.

Expected

Tavily returns structured search results; optional answer field when include_answer: true.

Actual

Confirmed via unit tests (see Evidence).

Evidence

Failing test/log before + passing after
Trace/log snippets
Screenshot/recording
Perf numbers (if relevant)

Tests added/updated:

src/agents/tools/web-search.test.ts — Tavily config resolution, provider auto-detection, freshness normalization
src/agents/tools/web-tools.enabled-defaults.test.ts — Full provider integration tests (basic search, count up to 20, AI answer, search depth, freshness, domain filters including mixed allowlist+denylist, schema validation, proxy dispatch)
src/config/config-misc.test.ts — Zod schema validation for Tavily config (accepts valid, rejects unknown fields)

Screenshots

Human Verification (required)

Verified scenarios: All new tests pass (pnpm test). Config validation accepts valid Tavily config and rejects unknown fields. Provider auto-detection selects Tavily when only TAVILY_API_KEY is present and falls back to Brave when no keys exist.
Edge cases checked: Missing API key returns setup hint. Blank/whitespace baseUrl falls back to default. Invalid search_depth values are rejected. Mixed allowlist+denylist domain filters work for Tavily (blocked for other providers). Freshness normalization handles Brave shortcuts (pd/pw/pm/py) and standard values.
What you did not verify: Live Tavily API calls (mocked in tests). UI dashboard display of Tavily config (no UI changes in this PR).

Review Conversations

I replied to or resolved every bot review conversation I addressed in this PR.
I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

Backward compatible? Yes - purely additive; no existing behavior changed.
Config/env changes? Yes - new optional tools.web.search.tavily config section and TAVILY_API_KEY env var.
Migration needed? No

Failure Recovery (if this breaks)

How to disable/revert this change quickly: Set tools.web.search.provider to any other provider, or remove TAVILY_API_KEY from environment.
Files/config to restore: N/A (additive change).
Known bad symptoms reviewers should watch for: missing_tavily_api_key errors if provider is set to "tavily" without a key configured.

Risks and Mitigations

Risk: Tavily API rate limits or availability issues could affect users who auto-detect this provider.
- Mitigation: Tavily is lowest priority in auto-detection (position 6); users only get it if no other provider key is configured. Explicit provider: "tavily" is opt-in.

greptile-apps · 2026-03-12T19:34:51Z

Greptile Summary

This PR adds Tavily as a sixth web_search provider, following the established integration patterns for Brave, Gemini, Grok, Kimi, and Perplexity. The implementation is additive-only: no existing provider behaviour is changed.

What's added:

runTavilySearch — POSTs to https://api.tavily.com/search via the existing withTrustedWebSearchEndpoint dispatcher with Authorization: Bearer header
Config/type plumbing: TavilyConfig, Zod schema (.strict()), types.tools.ts, schema.help.ts, schema.labels.ts
Auto-detection as the lowest-priority (position 6) provider when TAVILY_API_KEY is in the environment
Tavily-specific tool parameters: search_depth, include_answer, and domain_filter with mixed allowlist+denylist support
Tavily-specific max count of 20 (vs 10 for other providers), surfaced in both the tool schema and resolveSearchCount
Freshness normalization reuses the existing normalizeFreshness Tavily path (day/week/month/year), including Brave shortcut mapping (pd/pw/pm/py)
Onboarding wizard entry, EN + zh-CN docs, comprehensive unit and integration tests

Issues found:

TAVILY_FRESHNESS_MAP is a pure identity map (each key maps to itself) — if the map is not kept in sync with future normalizeFreshness output values, freshness would silently be dropped rather than raising an error
search_depth in the TypeBox schema uses Type.String() instead of a Type.Union([Type.Literal("basic"), Type.Literal("advanced")]), so the LLM receives no enum hint from the schema itself
"tools.web.search.maxResults" help text still reads "(1-10)" — slightly misleading when Tavily is the provider

Confidence Score: 4/5

This PR is safe to merge — it is purely additive, follows all existing provider patterns, and has comprehensive tests.
The integration is correct and well-tested. No existing provider behaviour is touched. The three flagged items are all style/documentation issues (identity map, missing schema enum, stale help text) that do not affect correctness or security. API key handling follows the same SecretRef/env-fallback pattern as all other providers, and network calls go through the existing trusted-endpoint dispatcher.
No files require special attention; the style issues in src/agents/tools/web-search.ts and src/config/schema.help.ts are minor and non-breaking.

Comments Outside Diff (2)

src/agents/tools/web-search.ts, line 649-654 (link)

search_depth schema should use an enum union, not a free-form string

The search_depth field is declared as Type.String(), which exposes it as an unconstrained string in the JSON Schema surfaced to the LLM. Since only "basic" and "advanced" are valid, the schema should communicate these choices explicitly with a union of literals — the same pattern TypeBox uses elsewhere in this codebase for constrained string values. This prevents the LLM from guessing arbitrary values that would be caught only at runtime with a "invalid_search_depth" error.

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/agents/tools/web-search.ts
Line: 649-654

Comment:
**`search_depth` schema should use an enum union, not a free-form string**

The `search_depth` field is declared as `Type.String()`, which exposes it as an unconstrained string in the JSON Schema surfaced to the LLM. Since only `"basic"` and `"advanced"` are valid, the schema should communicate these choices explicitly with a union of literals — the same pattern TypeBox uses elsewhere in this codebase for constrained string values. This prevents the LLM from guessing arbitrary values that would be caught only at runtime with a `"invalid_search_depth"` error.



How can I resolve this? If you propose a fix, please make it concise.

src/config/schema.help.ts, line 1521 (link)

maxResults help text doesn't reflect Tavily's 20-result limit

The help string still says "(1-10)", but Tavily allows up to 20 results. A user configuring tools.web.search.maxResults with Tavily as the provider would see misleading guidance. Consider updating to mention the provider-specific ceiling:

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/config/schema.help.ts
Line: 1521

Comment:
**`maxResults` help text doesn't reflect Tavily's 20-result limit**

The help string still says `"(1-10)"`, but Tavily allows up to 20 results. A user configuring `tools.web.search.maxResults` with Tavily as the provider would see misleading guidance. Consider updating to mention the provider-specific ceiling:



How can I resolve this? If you propose a fix, please make it concise.

Prompt To Fix All With AI

This is a comment left during a code review.
Path: src/agents/tools/web-search.ts
Line: 50-57

Comment:
**Redundant identity map for `TAVILY_FRESHNESS_MAP`**

`TAVILY_FRESHNESS_MAP` is a pure identity map (every key maps to itself). Its own comment even states "Tavily time_range values match the normalized freshness values". This means the lookup `TAVILY_FRESHNESS_MAP[params.freshness]` always returns the same value as `params.freshness` — but silently returns `undefined` for any value that isn't in the map, which could cause freshness to be dropped without any error if a new normalized value is introduced in `normalizeFreshness` without also updating this map.

The simpler and more robust approach is to pass `params.freshness` directly:

```suggestion
// Tavily time_range field accepts the same values as normalized freshness (day/week/month/year).
```

And update the usage on line 1954:
```ts
timeRange: params.freshness || undefined,
```
This removes the risk of silent freshness loss due to an out-of-sync map.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/agents/tools/web-search.ts
Line: 649-654

Comment:
**`search_depth` schema should use an enum union, not a free-form string**

The `search_depth` field is declared as `Type.String()`, which exposes it as an unconstrained string in the JSON Schema surfaced to the LLM. Since only `"basic"` and `"advanced"` are valid, the schema should communicate these choices explicitly with a union of literals — the same pattern TypeBox uses elsewhere in this codebase for constrained string values. This prevents the LLM from guessing arbitrary values that would be caught only at runtime with a `"invalid_search_depth"` error.

```suggestion
      search_depth: Type.Optional(
        Type.Union(
          [Type.Literal("basic"), Type.Literal("advanced")],
          {
            description:
              "Search depth: 'basic' (default, faster) or 'advanced' (more thorough, higher cost).",
          },
        ),
      ),
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/config/schema.help.ts
Line: 1521

Comment:
**`maxResults` help text doesn't reflect Tavily's 20-result limit**

The help string still says `"(1-10)"`, but Tavily allows up to 20 results. A user configuring `tools.web.search.maxResults` with Tavily as the provider would see misleading guidance. Consider updating to mention the provider-specific ceiling:

```suggestion
  "tools.web.search.maxResults": "Number of results to return (1-10; Tavily supports up to 20).",
```

How can I resolve this? If you propose a fix, please make it concise.

_{Last reviewed commit: 249a9bc}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 249a9bc059

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c4e47e9122

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

…-specific test for mixed allowlist and denylist

…ort mixed allowlist and denylist in domain filters

…ovider

…d target registry

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 68e305d81c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

openclaw-barnacle Bot added docs Improvements or additions to documentation commands Command implementations agents Agent runtime and tooling size: L labels Mar 12, 2026

lakshyaag-tavily marked this pull request as ready for review March 12, 2026 19:30

This comment was marked as spam.

Sign in to view

greptile-apps Bot reviewed Mar 12, 2026

View reviewed changes

Comment thread src/agents/tools/web-search.ts Outdated

chatgpt-codex-connector Bot reviewed Mar 12, 2026

View reviewed changes

Comment thread src/config/types.tools.ts

This was referenced Mar 13, 2026

feat(web-search): add Tavily as native search provider #39299

Closed

feat(web-search): follow-up Tavily wiring (secrets + labels + tests) deggertsen/openclaw#1

Closed

chatgpt-codex-connector Bot reviewed Mar 13, 2026

View reviewed changes

Comment thread docs/zh-CN/tavily.md

lakshyaag-tavily added 9 commits March 16, 2026 10:50

feat: add Tavily as a new web search provider

3585464

feat: add tests

d83bb5f

feat: add Tavily documentation and update web search provider references

edf2760

fix: enhance domain filter handling in web search tool and add Tavily…

eae6bc6

…-specific test for mixed allowlist and denylist

feat: update Tavily documentation and enhance web search tool to supp…

1c2b12a

…ort mixed allowlist and denylist in domain filters

fix: update SEARCH_PROVIDER_OPTIONS to include Tavily as the sixth pr…

bc0444a

…ovider

feat: add Tavily API key support to secret reference documentation an…

091038a

…d target registry

fixes

4e98a6b

refactor: plugin system

68e305d

lakshyaag-tavily force-pushed the feat/add-web-search-tavily branch from 77edba4 to 68e305d Compare March 16, 2026 15:32

lakshyaag-tavily requested a review from a team as a code owner March 16, 2026 15:32

chatgpt-codex-connector Bot reviewed Mar 16, 2026

View reviewed changes

Comment thread src/plugins/web-search-providers.ts

test: add Tavily to web search providers test cases

169b72e

lakshyaag-tavily closed this Mar 17, 2026

lakshyaag-tavily mentioned this pull request Mar 17, 2026

feat: add Tavily as a bundled web search plugin with search and extract tools #49200

Merged

20 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat: add Tavily as a `web_search` provider#44343

feat: add Tavily as a `web_search` provider#44343
lakshyaag-tavily wants to merge 10 commits into
openclaw:mainfrom
lakshyaag-tavily:feat/add-web-search-tavily

lakshyaag-tavily commented Mar 12, 2026 •

edited

Loading

Uh oh!

This comment was marked as spam.

greptile-apps Bot commented Mar 12, 2026 •

edited

Loading

Comments Outside Diff (2)

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

lakshyaag-tavily commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Change Type (select all)

Scope (select all touched areas)

User-visible / Behavior Changes

Security Impact (required)

Repro + Verification

Environment

Steps

Expected

Actual

Evidence

Screenshots

Human Verification (required)

Review Conversations

Compatibility / Migration

Failure Recovery (if this breaks)

Risks and Mitigations

Uh oh!

This comment was marked as spam.

greptile-apps Bot commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 4/5

Comments Outside Diff (2)

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lakshyaag-tavily commented Mar 12, 2026 •

edited

Loading

greptile-apps Bot commented Mar 12, 2026 •

edited

Loading