fix(tavily): reject blank extract URLs by VectorPeak · Pull Request #111333 · openclaw/openclaw

VectorPeak · 2026-07-19T09:10:11Z

Additional instructions

MUST: Keep Allow edits from maintainers enabled for this PR so maintainers
can help update the branch when needed.

What Problem This Solves

Fixes an issue where users or model-generated tool calls could invoke tavily_extract with only whitespace in the urls array and bypass OpenClaw's local "requires at least one URL" validation.

The extract tool previously used truthiness filtering, so urls: [""] was rejected locally, but urls: [" "] was treated as one URL and forwarded to Tavily. That pushed an invalid request across the provider boundary instead of returning the same local validation error users already get for an empty URL list.

Why This Change Was Made

The extract tool now trims each string URL entry before filtering blank values. This keeps the fix inside Tavily's local tavily_extract parameter normalization and leaves Tavily search, provider config, HTTP client behavior, cache keys, and response wrapping unchanged.

The implementation also preserves useful surrounding whitespace handling: a valid value such as " https://example.com/article " is normalized to "https://example.com/article" before the Tavily request is built.

User Impact

Users get a fast, deterministic OpenClaw validation error when tavily_extract receives only blank URL entries, instead of waiting for the Tavily API to reject an invalid URL payload.

Valid pasted URLs with accidental leading or trailing whitespace continue to work and are sent to Tavily in normalized form.

Evidence

User-realistic call chain:

model/user tool call: tavily_extract({ urls: ["   "] })
  -> extensions/tavily/src/tavily-extract-tool.ts execute()
  -> local urls normalization
  -> local "tavily_extract requires at least one URL." guard

Before this change, the same payload survived urls.filter(Boolean) because a whitespace-only string is truthy, then reached runTavilyExtract({ urls: [" "] }).

After this change, the focused regression test proves both sides of the behavior:

urls: ["   "]                         -> rejects locally before runTavilyExtract()
urls: [" https://example.com/article "] -> forwards ["https://example.com/article"]

Real before/after runtime proof, using the actual createTavilyExtractTool() path, the real Tavily client HTTP path, a redacted fake Tavily key, and a public request-capture endpoint instead of the real Tavily API:

# Before: upstream/main at ccea4ea4400
node --import tsx <redacted-local-proof-script>

{"label":"blank whitespace URL","outcome":"rejected","message":"Tavily Extract: malformed JSON response"}
{"label":"padded valid URL","outcome":"rejected","message":"Tavily Extract: malformed JSON response"}

Captured outbound requests:
POST https://webhook.site/<redacted-token>/tavily-before-1784469360058/extract
x-client-source: openclaw
authorization: Bearer redacted-test-key
content: {"urls":["   "]}

POST https://webhook.site/<redacted-token>/tavily-before-1784469360058/extract
x-client-source: openclaw
authorization: Bearer redacted-test-key
content: {"urls":[" https://example.com/article "]}

That before-run shows both inputs crossed OpenClaw's local tool boundary and reached the provider request path with the original untrimmed values.

# After: this PR head at cfe24e7fb6876ae51df864ec52729224070281fe
node --import tsx <redacted-local-proof-script>

{"label":"blank whitespace URL","outcome":"rejected","message":"tavily_extract requires at least one URL."}
{"label":"padded valid URL","outcome":"rejected","message":"Tavily Extract: malformed JSON response"}

Captured outbound requests:
POST https://webhook.site/<redacted-token>/tavily-after-1784469396989/extract
x-client-source: openclaw
authorization: Bearer redacted-test-key
content: {"urls":["https://example.com/article"]}

captured request count for tavily-after-1784469396989/extract: 1

The after-run shows the whitespace-only URL now fails locally with the intended tavily_extract validation error and creates no outbound request. The padded valid URL still reaches the real extract HTTP path, but the captured request body contains the normalized URL. The final malformed JSON response is expected because the request-capture endpoint is not the Tavily API; it is used only to prove the outbound request payload without any real credentials.

Sibling surfaces checked: this PR only changes the dedicated tavily_extract URL list normalization. tavily_search query/domain filtering, Tavily client request construction, cache keys, response wrapping, provider config, and search provider behavior are unchanged.

Validation:

node scripts/run-vitest.mjs run extensions/tavily/src/tavily-tools.test.ts
# passed: 1 test file, 15 tests

pnpm exec oxfmt --check extensions/tavily/src/tavily-extract-tool.ts extensions/tavily/src/tavily-tools.test.ts
# passed: all matched files use the correct format

git diff --check
# passed

Local limitation:

pnpm check:changed -- extensions/tavily/src/tavily-extract-tool.ts extensions/tavily/src/tavily-tools.test.ts
# blocked locally: check:changed delegated to Blacksmith Testbox via crabbox,
# but the local crabbox binary failed its basic --version/--help sanity checks.

clawsweeper · 2026-07-19T09:17:01Z

Codex review: needs maintainer review before merge. Reviewed July 19, 2026, 10:02 AM ET / 14:02 UTC.

Summary
The branch trims tavily_extract URL entries, rejects blank-only input before the provider call, and adds focused regression coverage for blank and padded URLs.

PR surface: Source 0, Tests +22. Total +22 across 2 files.

Reproducibility: yes. source inspection gives a high-confidence path: call tavily_extract with urls: [" "]; current main retains the value through truthiness filtering and reaches runTavilyExtract. The PR body also supplies a before/after live HTTP-path transcript.

Review metrics: none identified.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Refresh the merge result against current main before landing because the branch is behind.

Risk before merge

[P1] The branch is behind current main; refresh the merge result against 7ea4e6701ed4c1ba4d49107dd7af736c971aa9da before landing, even though GitHub currently reports it mergeable.

Maintainer options:

Decide the mitigation before merge
Keep validation in extensions/tavily/src/tavily-extract-tool.ts, retain the focused regression test, refresh the branch against current main, and merge once the exact head remains clean.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

[P2] No concrete repair remains: the patch has no supported code finding, and the remaining action is ordinary maintainer review of the refreshed merge result.

Security
Cleared: The two-file diff only normalizes locally supplied URL strings before an existing provider request and adds tests; it introduces no dependency, permission, secret, workflow, or supply-chain change.

Review details

Best possible solution:

Keep validation in extensions/tavily/src/tavily-extract-tool.ts, retain the focused regression test, refresh the branch against current main, and merge once the exact head remains clean.

Do we have a high-confidence way to reproduce the issue?

Yes, source inspection gives a high-confidence path: call tavily_extract with urls: [" "]; current main retains the value through truthiness filtering and reaches runTavilyExtract. The PR body also supplies a before/after live HTTP-path transcript.

Is this the best way to solve the issue?

Yes. Normalizing at the dedicated extract-tool boundary is the narrowest maintainable fix because it rejects blank input before cache-key construction and the provider request, without changing Tavily search or shared plugin SDK behavior.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 7ea4e6701ed4.

Label changes

Label changes:

add proof: sufficient: Contributor real behavior proof is sufficient. The PR body contains a redacted before/after live-runtime transcript through the actual tool and HTTP path, showing blank input no longer sends a request and padded valid input is sent trimmed.
add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body contains a redacted before/after live-runtime transcript through the actual tool and HTTP path, showing blank input no longer sends a request and padded valid input is sent trimmed.
remove rating: 🦪 silver shellfish: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.
remove status: 📣 needs proof: Current PR status label is status: 👀 ready for maintainer look.

Label justifications:

P2: This is a bounded provider-tool validation bug that prevents avoidable invalid outbound requests without evidence of data loss, security impact, or core-runtime unavailability.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body contains a redacted before/after live-runtime transcript through the actual tool and HTTP path, showing blank input no longer sends a request and padded valid input is sent trimmed.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body contains a redacted before/after live-runtime transcript through the actual tool and HTTP path, showing blank input no longer sends a request and padded valid input is sent trimmed.

Evidence reviewed

PR surface:

Source 0, Tests +22. Total +22 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	1	1	0
Tests	1	22	0	+22
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	23	1	+22

What I checked:

Current behavior remains unfixed on main: Current main filters extract URLs only by truthiness, so a whitespace-only string remains in the outbound request list; the central bug is still necessary to fix. (github.com) (extensions/tavily/src/tavily-extract-tool.ts:50, 7ea4e6701ed4)
Correct ownership boundary: The Tavily plugin entrypoint creates and registers the dedicated tavily_extract tool, so normalizing at that tool boundary avoids changing search, shared SDK, or HTTP-client behavior. (raw.githubusercontent.com) (extensions/tavily/index.ts:1, 7ea4e6701ed4)
Provider and cache boundary: runTavilyExtract uses the supplied URL array for both its cache key and outbound body, making pre-dispatch normalization the appropriate place to prevent blank payloads and make padded URLs canonical. (raw.githubusercontent.com) (extensions/tavily/src/tavily-client.ts:4, 7ea4e6701ed4)
Sibling behavior inspected: tavily_search independently filters optional domain arrays, while the PR changes only the required extract URL list; no shared helper or plugin API contract is modified. (raw.githubusercontent.com) (extensions/tavily/src/tavily-search-tool.ts:2, 7ea4e6701ed4)
Patch and regression coverage: The PR maps string entries through trim() before filtering and adds assertions that blank input does not call runTavilyExtract while a padded URL is forwarded in canonical form. (extensions/tavily/src/tavily-extract-tool.ts:53, cfe24e7fb687)
After-fix real behavior proof: The PR body records an actual createTavilyExtractTool() invocation through the Tavily HTTP path: whitespace-only input now fails locally without a captured request, and a padded URL produces one captured request with the trimmed value. (extensions/tavily/src/tavily-tools.test.ts:354, cfe24e7fb687)

Likely related people:

VectorPeak: Authored the current focused Tavily repair and its supplied before/after runtime proof; source-history attribution for the existing Tavily files could not be established in this read-only review environment. (role: proposed repair author; confidence: low; commits: cfe24e7fb687; files: extensions/tavily/src/tavily-extract-tool.ts, extensions/tavily/src/tavily-tools.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Review history (1 earlier review cycle)

reviewed 2026-07-19T09:13:36.485Z sha 03fad64 :: needs real behavior proof before merge. :: none

Co-authored-by: chatgpt-codex-connector[bot] <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>

Co-authored-by: VectorPeak <[email protected]>

steipete · 2026-07-19T16:18:37Z

Merged via squash.

Prepared head SHA: b938e94869c59c5dd1133b45abde651a9f0b3d2e
Landed commit: fe9e018045d4b332896d7d8160d1753eedc21fcd

* fix(tavily): reject blank extract URLs Co-authored-by: chatgpt-codex-connector[bot] <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com> * fix(tavily): normalize tool string arrays Co-authored-by: VectorPeak <[email protected]> --------- Co-authored-by: chatgpt-codex-connector[bot] <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com> Co-authored-by: Peter Steinberger <[email protected]>

openclaw-barnacle Bot added extensions: tavily size: XS labels Jul 19, 2026

VectorPeak force-pushed the fix/tavily-extract-trim-urls branch from 03fad64 to cfe24e7 Compare July 19, 2026 13:14

steipete self-assigned this Jul 19, 2026

VectorPeak and others added 2 commits July 19, 2026 17:04

fix(tavily): reject blank extract URLs

700e1d1

Co-authored-by: chatgpt-codex-connector[bot] <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>

fix(tavily): normalize tool string arrays

b938e94

Co-authored-by: VectorPeak <[email protected]>

steipete force-pushed the fix/tavily-extract-trim-urls branch from cfe24e7 to b938e94 Compare July 19, 2026 16:11

steipete merged commit fe9e018 into openclaw:main Jul 19, 2026
83 checks passed

github-actions Bot mentioned this pull request Jul 19, 2026

📡 Upstream Digest — 2026-07-19 16:47 UTC curtismercier/openclaw-mods#1347

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(tavily): reject blank extract URLs#111333

fix(tavily): reject blank extract URLs#111333
steipete merged 2 commits into
openclaw:mainfrom
VectorPeak:fix/tavily-extract-trim-urls

VectorPeak commented Jul 19, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 19, 2026 •

edited

Loading

Uh oh!

Uh oh!

steipete commented Jul 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

VectorPeak commented Jul 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

clawsweeper Bot commented Jul 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

steipete commented Jul 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

VectorPeak commented Jul 19, 2026 •

edited

Loading

clawsweeper Bot commented Jul 19, 2026 •

edited

Loading