feat(tools): add Exa as native web_search provider#29322
feat(tools): add Exa as native web_search provider#29322V-Gutierrez wants to merge 1481 commits intoopenclaw:mainfrom
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ba89bfd60e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
src/agents/tools/web-search.ts
Outdated
| const startDate = freshnessToExaStartDate(params.freshness); | ||
| if (startDate) { | ||
| body.startPublishedDate = `${startDate}T00:00:00.000Z`; |
There was a problem hiding this comment.
Preserve end date when mapping Exa freshness ranges
When freshness is a range like YYYY-MM-DDtoYYYY-MM-DD, this code only sends startPublishedDate to Exa and drops the range end, so Exa can return documents newer than the requested end date. This makes range-based filtering incorrect specifically for the Exa provider and breaks users who rely on bounded historical windows.
Useful? React with 👍 / 👎.
.gitignore
Outdated
| vendor/ | ||
| !vendor/configs/ | ||
| !vendor/scripts/ |
There was a problem hiding this comment.
Unignore vendor root before re-including configs/scripts
The vendor/ rule ignores the whole directory tree, and the later negations for vendor/configs/ and vendor/scripts/ do not take effect unless the parent vendor/ directory is also unignored. As written, new files under those folders (for example vendor/configs/workspace/*) are still ignored, which breaks the documented vendor config workflow.
Useful? React with 👍 / 👎.
Greptile SummaryThis PR adds Exa as a fourth native web_search provider with neural/semantic search capabilities and reliable date filtering. The Exa implementation follows existing patterns for Brave and Perplexity, with proper config schema, Zod validation, and comprehensive test coverage (13 new tests). Key changes:
Critical issue found:
Scope issue:
Confidence Score: 2/5
Last reviewed commit: ba89bfd |
| if (params.freshness) { | ||
| const startDate = freshnessToExaStartDate(params.freshness); | ||
| if (startDate) { | ||
| body.startPublishedDate = `${startDate}T00:00:00.000Z`; | ||
| } | ||
| } |
There was a problem hiding this comment.
Date range end date is missing. PR description states ranges map to both startPublishedDate + endPublishedDate, but only start date is set here. For "2024-03-01to2024-03-31", this sets start but ignores end.
| if (params.freshness) { | |
| const startDate = freshnessToExaStartDate(params.freshness); | |
| if (startDate) { | |
| body.startPublishedDate = `${startDate}T00:00:00.000Z`; | |
| } | |
| } | |
| if (params.freshness) { | |
| const startDate = freshnessToExaStartDate(params.freshness); | |
| if (startDate) { | |
| body.startPublishedDate = `${startDate}T00:00:00.000Z`; | |
| // Extract end date from date range if present | |
| const endMatch = params.freshness.match(/to(\d{4}-\d{2}-\d{2})$/); | |
| if (endMatch?.[1]) { | |
| body.endPublishedDate = `${endMatch[1]}T23:59:59.999Z`; | |
| } | |
| } | |
| } |
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/tools/web-search.ts
Line: 461-466
Comment:
Date range end date is missing. PR description states ranges map to both `startPublishedDate` + `endPublishedDate`, but only start date is set here. For "2024-03-01to2024-03-31", this sets start but ignores end.
```suggestion
if (params.freshness) {
const startDate = freshnessToExaStartDate(params.freshness);
if (startDate) {
body.startPublishedDate = `${startDate}T00:00:00.000Z`;
// Extract end date from date range if present
const endMatch = params.freshness.match(/to(\d{4}-\d{2}-\d{2})$/);
if (endMatch?.[1]) {
body.endPublishedDate = `${endMatch[1]}T23:59:59.999Z`;
}
}
}
```
How can I resolve this? If you propose a fix, please make it concise.
Additional Comments (1)
Prompt To Fix With AIThis is a comment left during a code review.
Path: src/agents/tools/web-search.ts
Line: 61-66
Comment:
Update description - Exa also supports freshness now.
```suggestion
freshness: Type.Optional(
Type.String({
description:
"Filter results by discovery time (Brave and Exa). Values: 'pd' (past 24h), 'pw' (past week), 'pm' (past month), 'py' (past year), or date range 'YYYY-MM-DDtoYYYY-MM-DD'.",
}),
),
```
How can I resolve this? If you propose a fix, please make it concise. |
ba89bfd to
537f737
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 537f7374f9
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
src/agents/tools/web-search.ts
Outdated
| const res = await fetch(EXA_SEARCH_ENDPOINT, { | ||
| method: "POST", | ||
| headers: { | ||
| "Content-Type": "application/json", | ||
| "x-api-key": params.apiKey, | ||
| }, | ||
| body: JSON.stringify(body), | ||
| signal: withTimeout(undefined, params.timeoutSeconds * 1000), |
There was a problem hiding this comment.
Route Exa calls through the guarded web fetch path
This new Exa branch uses raw fetch instead of withTrustedWebSearchEndpoint, so it skips the shared web-tool network guard path used by the other providers. In deployments that depend on proxy-aware dispatch (HTTP_PROXY/HTTPS_PROXY) or the guarded redirect/DNS handling from fetchWithSsrFGuard, Exa requests can fail or behave inconsistently compared with the rest of web_search.
Useful? React with 👍 / 👎.
src/agents/tools/web-search.ts
Outdated
| if (raw === "exa") { | ||
| return "exa"; |
There was a problem hiding this comment.
Include Exa key in provider auto-detection
Adding explicit provider: "exa" handling here did not add Exa to the raw === "" auto-detection block, so setups with only EXA_API_KEY (or tools.web.search.exa.apiKey) still fall back to brave and then error with missing_brave_api_key. This makes Exa effectively unusable unless users manually set the provider, despite the existing auto-detect behavior for other providers.
Useful? React with 👍 / 👎.
|
|
|
This pull request has been automatically marked as stale due to inactivity. |
90bbe63 to
2b603d8
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2b603d8e19
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const rawFreshness = readStringParam(params, "freshness"); | ||
| if (rawFreshness && provider !== "brave" && provider !== "perplexity") { | ||
| // freshness is supported by brave, perplexity, and exa; grok/gemini/kimi do not support it | ||
| if (rawFreshness && provider !== "brave" && provider !== "perplexity" && provider !== "exa") { |
There was a problem hiding this comment.
Permit Exa date-range freshness values
This branch now treats exa as freshness-capable, but it still relies on normalizeFreshness(rawFreshness, provider), and that helper only accepts YYYY-MM-DDtoYYYY-MM-DD when provider === "brave" (src/agents/tools/web-search.ts:1220-1228). In the Exa path, a range like 2024-03-01to2024-03-31 normalizes to undefined and immediately triggers invalid_freshness, so bounded Exa windows never reach runExaSearch even though freshnessToExaDates supports them.
Useful? React with 👍 / 👎.
| /** Exa-specific configuration (used when provider="exa"). */ | ||
| exa?: { | ||
| /** Exa API key (defaults to EXA_API_KEY env var). */ | ||
| apiKey?: SecretInput; |
There was a problem hiding this comment.
Wire Exa API key into secrets runtime resolution
Adding tools.web.search.exa.apiKey here introduces a new SecretInput path, but Exa was not added to the secret-resolution surfaces (src/secrets/runtime-web-tools.ts:13-91 and src/secrets/target-registry-data.ts:704-757 still omit it). As a result, configs that store Exa credentials as SecretRefs are never resolved into runtime config, and web_search returns missing_exa_api_key unless a plain EXA_API_KEY env var is also present.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f68ac9d85f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const rawFreshness = readStringParam(params, "freshness"); | ||
| if (rawFreshness && provider !== "brave" && provider !== "perplexity") { | ||
| // freshness is supported by brave, perplexity, and exa; grok/gemini/kimi do not support it | ||
| if (rawFreshness && provider !== "brave" && provider !== "perplexity" && provider !== "exa") { |
There was a problem hiding this comment.
Allow Exa freshness ranges through normalization
This change treats Exa as a freshness-capable provider, but normalizeFreshness still only accepts YYYY-MM-DDtoYYYY-MM-DD when provider === "brave" (src/agents/tools/web-search.ts:1220-1228). For Exa, range inputs therefore normalize to undefined and immediately return invalid_freshness, so the new freshnessToExaDates range mapping never executes for bounded windows.
Useful? React with 👍 / 👎.
src/agents/tools/web-search.ts
Outdated
| const res = await fetch(EXA_SEARCH_ENDPOINT, { | ||
| method: "POST", |
There was a problem hiding this comment.
Route Exa requests through trusted web fetch guard
The new Exa path uses raw fetch here instead of withTrustedWebSearchEndpoint, unlike the other web_search providers. That skips the shared proxy-aware guarded network path, so deployments depending on trusted env proxy routing or the web-tools fetch guard behavior can see Exa fail or behave inconsistently with the rest of web search providers.
Useful? React with 👍 / 👎.
| /** Exa-specific configuration (used when provider="exa"). */ | ||
| exa?: { | ||
| /** Exa API key (defaults to EXA_API_KEY env var). */ | ||
| apiKey?: SecretInput; |
There was a problem hiding this comment.
Add Exa key path to secret target registry surfaces
This adds tools.web.search.exa.apiKey as a SecretInput, but the secret target registry still omits that path (src/secrets/target-registry-data.ts:704-757 lists only Brave/Gemini/Grok/Kimi/Perplexity web-search keys). Since configure/audit/plan discovery is registry-driven, Exa credentials are not discoverable through standard secret-target workflows even though the config type now advertises a secret input for them.
Useful? React with 👍 / 👎.
|
Addressed the Exa review items:
Verification:�[1m�[46m RUN �[49m�[22m �[36mv4.0.18 �[39m�[90m/Users/emptyhardware/Projetos/Codebases/Personal/openclaw�[39m �[32m✓�[39m src/agents/tools/web-search.test.ts �[2m(�[22m�[2m62 tests�[22m�[2m)�[22m�[32m 8�[2mms�[22m�[39m �[2m Test Files �[22m �[1m�[32m1 passed�[39m�[22m�[90m (1)�[39m
Posting this update to keep the PR active. |
|
Clean verification summary (previous comment had shell-expanded formatting):
Verification:
Keeping the PR active after the stale bot notice. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1678e2f3c9
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| // Brave date range support | ||
| if (provider === "brave") { | ||
| // Brave and Exa date range support | ||
| if (provider === "brave" || provider === "exa") { |
There was a problem hiding this comment.
Keep Exa freshness shortcuts in provider normalization
By adding Exa to freshness-capable providers here, normalizeFreshness now accepts Exa inputs but still maps pd/pw/pm/py through the non-Brave branch to day/week/month/year. runExaSearch only converts pd/pw/pm/py (or date ranges) in freshnessToExaDates, so Exa requests using documented shortcut values silently drop date filtering and return unbounded results. This affects Exa searches whenever callers send shortcut freshness values like pd.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 32294c8f6c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| }): Promise<{ results: Record<string, unknown>[]; tookMs: number }> { | ||
| const body: Record<string, unknown> = { | ||
| query: params.query, | ||
| num_results: params.count, |
There was a problem hiding this comment.
Use Exa count field name expected by the search API
The Exa request body sends the result limit as num_results, but this integration otherwise uses Exa's camelCase field names (startPublishedDate, endPublishedDate) and our own provider docs describe numResults; if Exa ignores unknown fields, count will be silently dropped and searches can return the wrong number of results. This should use the API’s expected count key so the tool honors caller limits.
Useful? React with 👍 / 👎.
| description: | ||
| "Filter results by discovery time (Brave and Exa). Values: 'pd' (past 24h), 'pw' (past week), 'pm' (past month), 'py' (past year), or date range 'YYYY-MM-DDtoYYYY-MM-DD'.", |
There was a problem hiding this comment.
Keep freshness schema text consistent for Perplexity mode
This shared freshness parameter description now advertises Brave/Exa-style values including date ranges, but Perplexity schemas reuse this same field while normalizeFreshness still rejects date ranges for Perplexity, so tool calls generated from the schema can request documented values and still fail with invalid_freshness. Please make the schema description provider-specific (or remove unsupported values) for Perplexity paths.
Useful? React with 👍 / 👎.
|
why still not merged? who merges? |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
21 similar comments
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
|
Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7128d94c76
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| // grok, gemini, kimi, etc. | ||
| return Type.Object({ | ||
| ...querySchema, | ||
| ...filterSchema, | ||
| }); |
There was a problem hiding this comment.
Split Exa schema from generic providers
The exa provider currently falls through this generic schema branch, which exposes country, language, date_after, and date_before, but execution later rejects those fields for Exa (unsupported_country / unsupported_language / unsupported_date_filter). In practice, tool-calling models rely on the declared schema, so Exa calls can be generated with these advertised params and fail at runtime instead of searching. Defining an Exa-specific schema (or removing unsupported fields from this fallback) keeps the contract consistent and prevents avoidable tool errors.
Useful? React with 👍 / 👎.
Implements Exa (exa.ai) as a plugin extension following the same architecture used by firecrawl and moonshot — registering via api.registerWebSearchProvider(). Features: - Neural, keyword, and auto search modes - Publication date filters (date_after, date_before, freshness) - Optional content highlights and full text extraction - Scoped credential support (tools.web.search.exa.apiKey) + EXA_API_KEY env - Auto-detect with order 25 - Full onboarding flow integration Supersedes openclaw#29322 which was built against the legacy inline provider pattern.
Implements Exa (exa.ai) as a bundled plugin extension following the same architecture used by brave, firecrawl, google, moonshot, perplexity, and xai. - New extension at extensions/exa/ with full provider implementation - Neural, keyword, and auto search types - Content extraction via highlights and text fields - Freshness filters with month-boundary overflow protection - Uses plugin-sdk helpers (resolveSearchTimeoutSeconds, buildSearchCacheKey, readCachedSearchPayload, normalizeFreshness, etc.) - Auto-detect order 25 (between gemini:20 and grok:30) - Config: EXA_API_KEY env or plugins.entries.exa.config.webSearch.apiKey - 16 pure tests (no vi.mock — compatible with extension CI runner) - Documentation updated in docs/tools/web.md Supersedes openclaw#29322 which used the legacy inline provider pattern.
Summary
Adds Exa (exa.ai) as a fourth native
web_searchprovider alongsidebrave,perplexity,grok,gemini, andkimi. Closes #20134.Exa uses neural/semantic search and provides reliable date filtering via ISO-8601 date ranges, making it useful for freshness-sensitive workflows (daily news crons, market monitoring, research pipelines).
Changes
src/config/types.tools.ts"exa"to the provider enum; definedExaSearchConfigtypesrc/config/zod-schema.agent-runtime.tsexaprovider config fieldssrc/agents/tools/web-search.tsfetchExaSearch,freshnessToExaDateshelper (start + end dates),resolveExaApiKey, response normalizationsrc/config/schema.tsFIELD_LABELSandFIELD_HELPentries for all Exa config keyssrc/agents/tools/web-search.test.tssrc/config/config.web-search-provider.test.tsFreshness mapping
pdstartPublishedDate: yesterday (ISO-8601)pwstartPublishedDate: 7 days agopmstartPublishedDate: 30 days agopystartPublishedDate: 365 days agoYYYY-MM-DDtoYYYY-MM-DDstartPublishedDate+endPublishedDate(full range, inclusive)Date ranges set both
startPublishedDate(midnight UTC) andendPublishedDate(23:59:59 UTC) for precise window filtering. Perplexity continues to reject the freshness param with an error (unchanged). Brave and Exa both support it.Config example
{ "tools": { "web": { "search": { "provider": "exa", "exa": { "type": "auto", "contents": "highlights" } } } } }API key can be set via
tools.web.search.exa.apiKeyconfig orEXA_API_KEYenvironment variable.No migration required
Purely additive change. Existing
brave/perplexity/grok/gemini/kimiconfigurations are unaffected. All Exa fields are optional with sensible defaults.Testing
pnpm vitest run src/agents/tools/web-search.test.ts src/config/config.web-search-provider.test.ts)AI-assisted PR checklist