feat: add Tavily as a bundled web search plugin with search and extract tools#49200
Conversation
openclaw-barnacle
Bot
added
docs
2b82273 to
376a6b5
Compare
Greptile SummaryThis PR adds Tavily as a bundled web search plugin, following the existing plugin architecture closely. It registers a Key issues found:
Confidence Score: 3/5
Prompt To Fix All With AIThis is a comment left during a code review.
Path: extensions/tavily/src/tavily-client.ts
Line: 54-69
Comment:
**Custom `baseUrl` with path silently ignores `pathname` for both endpoints**
When `baseUrl` is set to something like `https://proxy.example.com/tavily/`, the function detects a non-root pathname and returns the URL verbatim — discarding the `pathname` argument (`/search` or `/extract`). Both `runTavilySearch` and `runTavilyExtract` then POST to exactly the same URL. A user who configures `baseUrl: "https://proxy.example.com/api/v2/"` expecting the SDK to append `/search` or `/extract` will have both tools silently route to an unintended endpoint.
If the intent is "treat a URL-with-path as a complete endpoint override", that should be documented. If the intent is to support a base URL with a path prefix (e.g. a reverse proxy), consider always appending the pathname:
```typescript
url.pathname = url.pathname.replace(/\/$/, "") + pathname;
return url.toString();
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: extensions/tavily/src/tavily-client.ts
Line: 250
Comment:
**External image URLs not wrapped as untrusted content**
`r.images` is passed through from the Tavily API response without any `wrapExternalContent` treatment. The sibling fields `rawContent` and `content` on the same result object are both wrapped (lines 243–249). Image URLs from an external API are still external content and should be treated consistently — at minimum for auditing purposes.
```suggestion
...(Array.isArray(r.images)
? { images: (r.images as string[]).map((img) => wrapExternalContent(String(img), { source: "web_fetch", includeWarning: false })) }
: {}),
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: extensions/tavily/src/tavily-client.ts
Line: 29
Comment:
**`DEFAULT_TAVILY_BASE_URL` duplicated across modules**
This constant is already exported from `./config.js` (line 5 of `config.ts`). Defining it again here creates two sources of truth. If the default URL ever changes, one of them will be missed.
```suggestion
import {
DEFAULT_CACHE_TTL_MINUTES,
normalizeCacheKey,
readCache,
readResponseText,
resolveCacheTtlMs,
writeCache,
} from "openclaw/plugin-sdk/provider-web-search";
import { wrapExternalContent, wrapWebContent } from "openclaw/plugin-sdk/security-runtime";
import {
DEFAULT_TAVILY_BASE_URL,
resolveTavilyApiKey,
resolveTavilyBaseUrl,
resolveTavilyExtractTimeoutSeconds,
resolveTavilySearchTimeoutSeconds,
} from "./config.js";
```
Then remove the local `const DEFAULT_TAVILY_BASE_URL = "https://api.tavily.com";` declaration on this line.
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: extensions/tavily/src/tavily-client.ts
Line: 209-218
Comment:
**Extract cache key missing `baseUrl`**
The extract cache key omits `baseUrl`, but the search cache key (lines 122–135) correctly includes it. If the user changes their `baseUrl` configuration, stale results from the previous endpoint will be served from cache until the in-memory `EXTRACT_CACHE` is naturally evicted. Since `EXTRACT_CACHE` is a module-level `Map`, this persists for the life of the process.
`baseUrl` should be computed before the cache key block (mirroring how `runTavilySearch` resolves it on line 120 before building the cache key) and included in the JSON object passed to `normalizeCacheKey`.
How can I resolve this? If you propose a fix, please make it concise.Last reviewed commit: 376a6b5 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 376a6b5e41
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5cd38e3548
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7d6cb45f03
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
7d6cb45 to
e8c1817
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 31f5d691bf
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 90f3d2bffa
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b989db918d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
c07d927 to
1875c7a
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1875c7aa02
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
1875c7a to
bf7c4ac
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bf7c4ac87f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 25501d3a0c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ct tools (openclaw#49200) Merged via squash. Prepared head SHA: ece9226 Co-authored-by: lakshyaag-tavily <[email protected]> Co-authored-by: gumadeiras <[email protected]> Reviewed-by: @gumadeiras
…ct tools (openclaw#49200) Merged via squash. Prepared head SHA: ece9226 Co-authored-by: lakshyaag-tavily <[email protected]> Co-authored-by: gumadeiras <[email protected]> Reviewed-by: @gumadeiras
…ct tools (openclaw#49200) Merged via squash. Prepared head SHA: ece9226 Co-authored-by: lakshyaag-tavily <[email protected]> Co-authored-by: gumadeiras <[email protected]> Reviewed-by: @gumadeiras
…ct tools (openclaw#49200) Merged via squash. Prepared head SHA: ece9226 Co-authored-by: lakshyaag-tavily <[email protected]> Co-authored-by: gumadeiras <[email protected]> Reviewed-by: @gumadeiras
…ct tools (openclaw#49200) Merged via squash. Prepared head SHA: ece9226 Co-authored-by: lakshyaag-tavily <[email protected]> Co-authored-by: gumadeiras <[email protected]> Reviewed-by: @gumadeiras
…ct tools (openclaw#49200) Merged via squash. Prepared head SHA: ece9226 Co-authored-by: lakshyaag-tavily <[email protected]> Co-authored-by: gumadeiras <[email protected]> Reviewed-by: @gumadeiras
…ct tools (openclaw#49200) Merged via squash. Prepared head SHA: ece9226 Co-authored-by: lakshyaag-tavily <[email protected]> Co-authored-by: gumadeiras <[email protected]> Reviewed-by: @gumadeiras
…ct tools (openclaw#49200) Merged via squash. Prepared head SHA: ece9226 Co-authored-by: lakshyaag-tavily <[email protected]> Co-authored-by: gumadeiras <[email protected]> Reviewed-by: @gumadeiras
…ct tools (openclaw#49200) Merged via squash. Prepared head SHA: ece9226 Co-authored-by: lakshyaag-tavily <[email protected]> Co-authored-by: gumadeiras <[email protected]> Reviewed-by: @gumadeiras
2 participants
Summary
web_searchtool supports Brave, Firecrawl, Gemini, Grok, Kimi, and Perplexity but not Tavily Search API.extensions/tavily/): web search provider registration (autoDetectOrder: 70), two dedicated tools (tavily_search,tavily_extract), a bundled skill, API key resolution (TAVILY_API_KEY/ config), Zod schema + types, onboarding wizard entry, SecretRef surface registration, contract tests, and docs (dedicated page + web.md updates).web_fetch. No changes to the UI dashboard or gateway server logic.Change Type (select all)
Scope (select all touched areas)
Linked Issue/PR
web_searchprovider #44343 (supersedes the earlier native integration PR, now using the plugin architecture)User-visible / Behavior Changes
web_searchprovider option:"tavily".tools.web.search.tavilywithapiKeyandbaseUrlfields.TAVILY_API_KEY.TAVILY_API_KEYis set and no higher-priority provider key is found, Tavily is auto-selected (lowest priority, order 70).tavily_searchwith Tavily-specific parameters:search_depth(basic/advanced),topic(general/news/finance),include_answer,time_range,include_domains,exclude_domains, and up to 20 results.tavily_extractfor URL content extraction withquery-focused chunking,extract_depth(basic/advanced), and image inclusion.web_searchvstavily_searchvstavily_extract.Security Impact (required)
NoYes— newTAVILY_API_KEYenv var andtools.web.search.tavily.apiKeyconfig field, following the sameSecretInput/SecretRefpattern as all other providers.Yes— new outbound POST tohttps://api.tavily.com/searchandhttps://api.tavily.com/extract(or user-configuredbaseUrl), using the existingwithTrustedWebToolsEndpointdispatcher (proxy-aware, SSRF-guarded, timeout-enforced).Yes—web_searchtool gains Tavily as a provider option. Two new tools registered:tavily_search,tavily_extract.Nountrusted: truemarkers viawrapWebContent/wrapExternalContent.Repro + Verification
Environment
{ tools: { web: { search: { provider: "tavily", tavily: { apiKey: "tvly-..." } } } } }Steps
TAVILY_API_KEY=tvly-...in environment (or configure viaopenclaw configure --section web).tavily_searchwithquery: "test query".title,url,snippetfields are returned.Expected
answerfield wheninclude_answer: true.Actual
Evidence
Tests added/updated:
extensions/tavily/index.test.ts— plugin registration, web search provider + tool ownershipsrc/plugins/web-search-providers.test.ts— ordering: Tavily at position 70src/plugins/contracts/registry.contract.test.ts— contract: Tavily owns["tavily"]web search provider and["tavily_search", "tavily_extract"]toolssrc/commands/onboard-search.test.ts— 7 providers in onboarding picker, Tavily lastsrc/config/config.web-search-provider.test.ts— auto-detection selects Tavily when onlyTAVILY_API_KEYis presentAll gates pass:
pnpm build✅pnpm check✅pnpm test:extension tavily✅ (2 tests)pnpm test:contracts✅ (114 tests)Human Verification (required)
TAVILY_API_KEYis present and falls back to Brave when no keys exist. Onboarding picker lists all 7 providers with Tavily last. Contract test validates ownership. Build produces no warnings.baseUrlfalls back to default. Provider ordering respectsautoDetectOrder: 70(last position). Plugin auto-enables when selected in onboardingReview Conversations
Compatibility / Migration
Yes— purely additive; no existing behavior changed.Yes— new optionaltools.web.search.tavilyconfig section andTAVILY_API_KEYenv var.NoFailure Recovery (if this breaks)
plugins.entries.tavily.enabled: false, or settools.web.search.providerto any other provider, or removeTAVILY_API_KEYfrom environment.missing_tavily_api_keyerrors if provider is set to"tavily"without a key configured.Risks and Mitigations
provider: "tavily"is opt-in.