feat(firecrawl): keyless firecrawl-free web search + scrape (opt-in)#93590
feat(firecrawl): keyless firecrawl-free web search + scrape (opt-in)#93590developersdigest wants to merge 4 commits into
Conversation
Adds a keyless firecrawl-free provider for both web_search and web_fetch via
Firecrawl's keyless tier (no API key, no account). Selectable opt-in
(tools.web.{search,fetch}.provider); not the default. Keyless providers carry
no credential path (requiresCredential: false), so secret resolution never
targets the paid Firecrawl key; adds genuine keyless support to the web-fetch
provider contract, mirroring web-search. Keyed firecrawl providers unchanged.
|
Thanks for the context here. I swept through the related work, and this is now duplicate or superseded. Close as superseded: the useful Firecrawl keyless scrape/fetch work is now tracked by the narrower maintainer PR, while this branch adds a broader parallel Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Canonical path: Use #94551 as the canonical Firecrawl keyless scrape/fetch path, and handle keyless Firecrawl search only with explicit maintainer sponsorship. So I’m closing this here and keeping the remaining discussion on #94551. Review detailsBest possible solution: Use #94551 as the canonical Firecrawl keyless scrape/fetch path, and handle keyless Firecrawl search only with explicit maintainer sponsorship. Do we have a high-confidence way to reproduce the issue? Not applicable as a bug reproduction: this is a feature PR. The external Firecrawl premise is high-confidence because live unauthenticated Is this the best way to solve the issue? No; the broader parallel Security review: Security review needs attention: The diff intentionally adds anonymous Firecrawl network paths; the superseding maintainer PR narrows that boundary to selected scrape/fetch.
AGENTS.md: found and applied where relevant. What I checked:
Likely related people:
Codex review notes: model internal, reasoning high; reviewed against b0375f25b31b. |
mirror the web_search opt-in policy so the keyless firecrawl-free fetch provider is never auto-detected for no-key installs and is only used when explicitly selected via tools.web.fetch.provider. addresses clawsweeper P1 security-boundary review on openclaw#93590.
…eyless # Conflicts: # src/secrets/runtime-web-tools.ts
|
The canonical scoped implementation has landed in #94551 at beea31a. It keeps the existing Closing this broader parallel |

Summary
firecrawl-freeprovider for bothweb_searchandweb_fetch— clean Firecrawl results and page scrapes with no API key, no account. Previously Firecrawl was keyed-only./v2/searchand/v2/scrapeanswer anonymously).tools.web.{search,fetch}.provider: firecrawl-free(or onboarding).autoDetectOrderkeeps it behind the existing keyless defaults, so a fresh install's behavior is unchanged. No default/auto-selection change.credentialPathandrequiresCredential: false, so secret resolution never targets the paid Firecrawl key (this adds genuine keyless support to the web-fetch provider contract, matching web-search).firecrawlproviders andfirecrawl_search/firecrawl_scrapetools are unchanged; any configured keyed provider still wins; no new config keys.What should reviewers focus on?
firecrawl-client.ts(noAuthorization,openclaw-firecrawlUA, key optional); keyless credential handling (emptycredentialPath,requiresCredential: false); and provider registration in the public-artifact barrels + manifest.Linked context
Was this requested by a maintainer or owner? No — external contribution.
Real behavior proof (required for external PRs)
web_searchandweb_fetchreturn live results with no API key.api.firecrawl.dev/v2/searchand/v2/scrapewith noAuthorizationheader.pnpm build, then exercised the real provider path keyless —runFirecrawlSearch({ keyless: true, query: "openclaw github" })andrunFirecrawlScrape({ keyless: true, url: "https://example.com" })— plus thefirecrawl-toolstests and the web-fetch provider contract suite.provider: firecrawl-free, 200, no key# Example Domainmarkdown, 200, no keyfirecrawl-freefor both search and fetch401; the providers threw "needs a Firecrawl API key".Tests and validation
firecrawl-toolsunit tests (35 passed), web-fetch provider contract suite (3 passed incl.firecrawl-free),pnpm build, live keyless search/scrape.Authorization+ theopenclaw-firecrawlUA; registration contract updated for the newfirecrawl-freesearch + fetch ids.Risk checklist
firecrawl-freeis opt-in (selectable), not auto-selected; the existing keyless default is unchanged.tools.web.{search,fetch}.provider; registered as non-migrated in doctor.api.firecrawl.dev(already an allowed host), via existing web-tools guards; no credentials sent; keyless providers never target the paid credential path.Current review state