Skip to content

feat: add Exa as web_search provider#31310

Closed
louiswalsh wants to merge 9 commits intoopenclaw:mainfrom
exa-labs:feat/exa-web-search-clean
Closed

feat: add Exa as web_search provider#31310
louiswalsh wants to merge 9 commits intoopenclaw:mainfrom
exa-labs:feat/exa-web-search-clean

Conversation

@louiswalsh
Copy link
Copy Markdown

Summary

  • Problem: OpenClaw has no native Exa search integration — users who want Exa's semantic search must use external workarounds.
  • Why it matters: Exa provides high-quality semantic search with token-efficient highlights, useful for AI assistant workflows.
  • What changed: Added Exa as a 6th web_search provider (explicit opt-in only via provider: "exa" config), with highlights-based content retrieval, config types, Zod schema, env var support (EXA_API_KEY), and tests.
  • What did NOT change (scope boundary): No changes to existing providers, no changes to web_fetch, no auto-detection — Exa is only activated when explicitly configured.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

User-visible / Behavior Changes

  • New provider: "exa" option for tools.web.search.provider
  • New config keys: tools.web.search.exa.apiKey, tools.web.search.exa.numResults (default: 5), tools.web.search.exa.highlightsMaxChars (default: 8000)
  • New env var: EXA_API_KEY
  • Tool description changes when Exa is selected: "Search the web using Exa search. Returns structured results with titles, URLs, and highlights."
  • No changes to default behavior — Brave remains the default provider

Security Impact (required)

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? Yes — new EXA_API_KEY env var / exa.apiKey config, follows same pattern as existing providers
  • New/changed network calls? Yes — new outbound call to https://api.exa.ai/search with x-exa-integration: "openclaw" header
  • Command/tool execution surface changed? No
  • Data access scope changed? No
  • If any Yes, explain risk + mitigation: API key is marked sensitive in Zod schema (same as all other provider keys). Network call only fires when user explicitly sets provider: "exa". Uses existing withTrustedWebSearchEndpoint wrapper with timeout and abort support.

Repro + Verification

Environment

  • OS: Linux
  • Runtime/container: Node.js (vitest)
  • Model/provider: N/A (unit tests only)
  • Integration/channel (if any): N/A
  • Relevant config (redacted): { tools: { web: { search: { provider: "exa", exa: { apiKey: "exa-..." } } } } }

Steps

  1. Set EXA_API_KEY or configure tools.web.search.exa.apiKey
  2. Set tools.web.search.provider: "exa"
  3. Invoke web_search tool with a query

Expected

  • Returns structured results with title, URL, highlights, published date, author
  • Results capped at numResults (default 5) with highlights limited to highlightsMaxChars (default 8000)

Actual

  • Matches expected (verified via unit tests)

Evidence

  • Failing test/log before + passing after

47 tests passing in web-search.test.ts, 18 in config.web-search-provider.test.ts.

Human Verification (required)

  • Verified scenarios: All resolver functions (apiKey, numResults, highlightsMaxChars), config validation, auto-detection exclusion, Exa not in provider priority chain
  • Edge cases checked: Invalid/missing/zero/negative/Infinity/NaN values for numResults and highlightsMaxChars, missing API key fallback to env var
  • What you did not verify: Live API call to Exa (unit tests mock the network layer)

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? Yes — new optional EXA_API_KEY env var and tools.web.search.exa config block
  • Migration needed? No
  • If yes, exact upgrade steps: N/A

Failure Recovery (if this breaks)

  • How to disable/revert this change quickly: Remove provider: "exa" from config — falls back to default Brave provider
  • Files/config to restore: N/A (additive change, no existing behavior modified)
  • Known bad symptoms reviewers should watch for: If EXA_API_KEY is invalid, the tool returns an API error message (same pattern as other providers)

Risks and Mitigations

  • Risk: Exa API availability/rate limits
    • Mitigation: Exa is only used when explicitly configured — no impact on default behavior

@openclaw-barnacle openclaw-barnacle bot added docs Improvements or additions to documentation agents Agent runtime and tooling size: M labels Mar 2, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f39bef2311

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines 34 to 35
suppressToolErrorWarnings: z.boolean().optional(),
lightContext: z.boolean().optional(),
})
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore heartbeat lightContext in config schema

Removing lightContext from HeartbeatSchema breaks existing configs that set defaults.heartbeat.lightContext, because this schema is .strict() and validateConfigObject* rejects unknown keys. The runtime still consumes heartbeat.lightContext (for lightweight heartbeat context), so this change turns a previously valid and used setting into a validation error and can block config loading for users already relying on it.

Useful? React with 👍 / 👎.

2. **Gemini** — `GEMINI_API_KEY` env var or `search.gemini.apiKey` config
3. **Perplexity** — `PERPLEXITY_API_KEY` / `OPENROUTER_API_KEY` env var or `search.perplexity.apiKey` config
4. **Grok** — `XAI_API_KEY` env var or `search.grok.apiKey` config
5. **Exa** — `EXA_API_KEY` env var or `search.exa.apiKey` config
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Correct Exa auto-detection behavior in docs

This section now states Exa is auto-detected from EXA_API_KEY, but the implementation and tests explicitly require provider: "exa" (auto-detection falls back through other providers and then Brave). Users following this doc can set only EXA_API_KEY and still not get Exa, which leads to confusing setup failures; the doc should match the actual resolver behavior.

Useful? React with 👍 / 👎.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 2, 2026

Greptile Summary

This PR adds Exa as a 6th web search provider with explicit opt-in configuration. The implementation is well-structured and follows established patterns from other providers (Brave, Perplexity, Gemini, Grok, Kimi).

Key changes:

  • Added Exa provider support in web-search.ts with highlights-based content retrieval
  • Configuration types and Zod schema validation in types.tools.ts and zod-schema.agent-runtime.ts
  • Comprehensive test coverage for config resolution and validation
  • Documentation updates for the new provider
  • Exa is NOT auto-detected (requires explicit provider: "exa" config)

Issue found:

  • docs/tools/web.md:242 contains incorrect documentation about Gemini's redirect resolver SSRF policy - states it uses "trusted-network model" (allows private networks) when the code actually uses withStrictWebToolsEndpoint (blocks private networks). This is unrelated to the Exa feature.

Confidence Score: 4/5

  • Safe to merge after fixing the documentation error about Gemini SSRF policy
  • Score reflects one factual documentation error (Gemini SSRF policy incorrectly documented) that needs correction. The Exa implementation itself is solid with proper tests, types, validation, and follows existing patterns. The unrelated documentation change should ideally be reverted or fixed.
  • docs/tools/web.md line 242 needs correction - revert to previous accurate description of Gemini redirect SSRF behavior

Last reviewed commit: f39bef2

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

6 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b1f886eb67

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +1400 to +1401
numResults: params.exaNumResults ?? DEFAULT_EXA_NUM_RESULTS,
highlightsMaxChars: params.exaHighlightsMaxChars ?? DEFAULT_EXA_HIGHLIGHTS_MAX_CHARS,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor count argument when using Exa provider

The Exa branch ignores the tool-level count input (and the derived search.maxResults fallback) by always sending numResults from exaNumResults/default 5. In practice, web_search calls like count: 1 or count: 10 with provider: "exa" will still return the configured/default Exa count, which breaks the existing web_search parameter contract for callers that expect per-request result sizing.

Useful? React with 👍 / 👎.

@louiswalsh louiswalsh force-pushed the feat/exa-web-search-clean branch from 04bfb21 to 3286b5f Compare March 3, 2026 05:11
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3286b5f8a6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 801725f441

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

louiswalsh and others added 5 commits March 4, 2026 15:39
Adds Exa as a 6th web_search provider (explicit opt-in only via
provider: "exa" config). Includes config types, Zod schema, highlights-
based content retrieval, env var support (EXA_API_KEY), and tests.

No changes to existing providers or default behavior.
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
@louiswalsh louiswalsh force-pushed the feat/exa-web-search-clean branch from 801725f to 78bfed1 Compare March 4, 2026 23:40
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 78bfed1dec

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

}

function resolveExaApiKey(exa?: ExaSearchConfig): string | undefined {
const fromConfig = normalizeApiKey(exa?.apiKey);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Resolve Exa SecretRefs before reading apiKey

resolveExaApiKey normalizes exa.apiKey as a plain string, so a valid SecretRef value gets coerced to empty and the Exa provider fails with missing_exa_api_key. Fresh evidence for re-raising this area: Exa is still absent from the web-search secret resolver paths in src/secrets/runtime-config-collectors-core.ts:284-290 and from the target registry in src/secrets/target-registry-data.ts:666-719, so tools.web.search.exa.apiKey is never resolved before this code executes.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7e5bce9e41

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines 284 to +286
const paths = [
"apiKey",
"exa.apiKey",
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include exa in provider gating for secret resolution

Adding "exa.apiKey" to this path list without updating the selectedProvider whitelist above means provider: "exa" falls into the selectedProvider === undefined branch, so all web-search provider secret refs are treated as active. In that configuration, stale refs like tools.web.search.grok.apiKey or tools.web.search.kimi.apiKey are now resolved (instead of ignored as inactive), and missing/invalid refs can fail config secret resolution even though only Exa is selected.

Useful? React with 👍 / 👎.

@vincentkoc
Copy link
Copy Markdown
Member

Nice work surfacing this early.

I'm closing this as a duplicate of #52617. The Exa provider is now merged through the current bundled-plugin architecture, so this earlier branch is fully superseded by the landed path.

If that overlap looks wrong from your side, tell me what differs and I can re-check it quickly.

@vincentkoc vincentkoc closed this Mar 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling docs Improvements or additions to documentation size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants