feat: upgrade browser skill to Puppeteer + Stealth for WAF bypass#11
Merged
feat: upgrade browser skill to Puppeteer + Stealth for WAF bypass#11
Conversation
Replace Playwright-based browser-navigation skill with Puppeteer + puppeteer-extra-plugin-stealth. The stealth plugin applies ~12 evasion techniques that pass WAFs (Imperva, Cloudflare, DataDome) which block vanilla headless Chrome and Playwright. Dockerfile changes: - Add OPENCLAW_INSTALL_STEALTH_BROWSER build arg - Installs Chrome for Testing at /opt/chrome/ - Installs puppeteer-core, puppeteer-extra, puppeteer-extra-plugin-stealth - Installs required system libs for headless Chrome Skill changes: - Rewrite all examples from Playwright to Puppeteer + Stealth - Document why stealth is needed (WAF evasion techniques) - Update troubleshooting for common container issues - Use ES module syntax (.mjs) throughout Tested on F1 Fantasy (Imperva WAF): - Playwright: blocked (CORS/network error on all XHR) - agent-browser: blocked (same WAF) - Puppeteer + Stealth: login + full navigation successful
- Copy docker/agent/ build files into fork (Option A: single build context) - Source of truth: backend/docker/agent/Dockerfile, now lives in fork - Remove Playwright/Chromium/Xvfb browser block from root Dockerfile - Add Chrome for Testing + puppeteer-extra-plugin-stealth to agent Dockerfile (always installed) - Agent Dockerfile verified against running container d6b0d06f35de: - butley-api extension: confirmed at dist/extensions/butley-api/ - QMD symlink: /usr/local/lib/node_modules/@tobilu/qmd/bin/qmd - Workspace dir: /root/workspace - Passes Imperva/Cloudflare WAFs that block vanilla headless Chrome
npm install -g puts packages in /usr/local/lib/node_modules/ which Node can't resolve without NODE_PATH. Install in /opt/openclaw/ instead so they're found via the existing node_modules resolution.
- Use cp -rn to merge ALL packages (including transitive deps like debug) - Add NODE_PATH=/opt/openclaw/node_modules so scripts run from /root/ can resolve
- Add references/sample-stealth-navigation.mjs as working template - Document createRequire() + absolute path pattern for ESM compatibility - Update all code examples to use the correct import pattern - Agents can copy the sample and adapt for their use case
- Add references/sample-browserbase-navigation.mjs template - Document when to use local Chrome+Stealth vs Browserbase - Browserbase uses residential IPs — passes banking sites (Chase, etc.) - No stealth plugin needed — real browser, not detectable Tested: Chase.com login succeeds via Browserbase (blocked via datacenter)
- Add @browserbasehq/sdk to Dockerfile (official SDK) - Update sample to use SDK (session.create, session.debug) - Document live debug URL and session replay - Cleaner than raw fetch API calls
- Remove Chrome for Testing + system libs from Dockerfile (~250MB savings) - Remove puppeteer-extra + stealth plugin (not needed with Browserbase) - Keep only puppeteer-core + @browserbasehq/sdk - BROWSERBASE_API_KEY passed as env var from orchestrator - Skill rewritten for Browserbase-only workflow - Cloud browser = residential IPs, no anti-detection needed
- Remove OPENCLAW_INSTALL_STEALTH_BROWSER block from root Dockerfile - Remove Chrome for Testing, puppeteer-extra, stealth plugin (~250MB savings) - Install only puppeteer-core + @browserbasehq/sdk in agent Dockerfile - Rewrite skill for Browserbase-only workflow - API key injected via BROWSERBASE_API_KEY env var (set by orchestrator)
guiramos
added a commit
that referenced
this pull request
Mar 29, 2026
- #1: Fix capabilities query param — use append() for repeated params instead of join() - #4: Fix isConfigured — require hostname instead of registryUrl (default localhost OK) - #5: Add deregister on stopAccount — store registryClient+agentId per account - #7: Heartbeat re-registration — after 3 failures, deregister + re-register - #10: Read pilotPort from config in resolveAccount - #11: Remove unused execFile import from send.ts - #12: Fix ENOENT race condition — use spawn event instead of setTimeout(500ms) - #13: heartbeat checkReady — only require Pilot daemon if pilotPort configured - #17: Verified daemon.ts imports — both spawn and execFile are used (no change) - openclaw#18: Client reuse — store registryClient per account, reuse in gateway methods - Also: Add API key support (registryApiKey config + X-Registry-Key header) - Also: Add ok/error fields to SearchResult type
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces Playwright browser skill (PR #9) with Puppeteer + stealth plugin for WAF bypass.
Why
The original Playwright approach fails on sites with WAF protection (Imperva, Cloudflare, DataDome).
Test results on F1 Fantasy (Imperva WAF):
Changes
Build:
docker build --build-arg OPENCLAW_INSTALL_STEALTH_BROWSER=1 .