Skip to content

feat(web-fetch): add ssrfPolicy.allowCidrs for custom CIDR allowlisting#28657

Open
xiaoyaner0201 wants to merge 3 commits intoopenclaw:mainfrom
xiaoyaner0201:feat/web-fetch-ssrf-allowcidrs
Open

feat(web-fetch): add ssrfPolicy.allowCidrs for custom CIDR allowlisting#28657
xiaoyaner0201 wants to merge 3 commits intoopenclaw:mainfrom
xiaoyaner0201:feat/web-fetch-ssrf-allowcidrs

Conversation

@xiaoyaner0201
Copy link
Copy Markdown
Contributor

Summary

  • Problem: web_fetch is completely broken in fake-IP proxy environments (Surge Enhanced Mode, Clash fake-ip, Quantumult X, etc.) because the SSRF guard blocks RFC 2544 benchmark range (198.18.0.0/15) addresses used for DNS interception. There is no config path to opt out. Internal network fetching (e.g. local docs servers) is also impossible.
  • Why it matters: Every user running Surge, Clash, Shadowrocket, or similar tools in fake-IP/TUN mode cannot use web_fetch at all. Users wanting to access internal documentation servers are also blocked.
  • What changed: Added tools.web.fetch.ssrfPolicy.allowCidrs — an array of CIDR strings that should be allowed through SSRF protection. This is a generic approach: instead of adding specific booleans for each IP range (like allowRfc2544BenchmarkRange), users specify exactly which CIDRs they trust.
  • What did NOT change: Default behavior remains fully restrictive (no CIDRs allowed). The existing dangerouslyAllowPrivateNetwork nuclear option is untouched. Browser SSRF policy is unaffected.

Config example:

{
  "tools": {
    "web": {
      "fetch": {
        "ssrfPolicy": {
          "allowCidrs": ["198.18.0.0/15"]
        }
      }
    }
  }
}

More examples:

// Allow fake-IP proxy range
{ "allowCidrs": ["198.18.0.0/15"] }

// Allow internal docs server
{ "allowCidrs": ["10.0.0.0/8"] }

// Multiple ranges
{ "allowCidrs": ["198.18.0.0/15", "172.16.0.0/12"] }

Change Type (select all)

  • Bug fix
  • Feature

Scope (select all touched areas)

  • Skills / tool execution
  • API / contracts

Linked Issue/PR

User-visible / Behavior Changes

  • New config: tools.web.fetch.ssrfPolicy.allowCidrs (string array, optional, default empty)
  • When set, IP addresses matching any listed CIDR bypass SSRF blocking for web_fetch
  • Invalid CIDR strings are warned and skipped (no crash)
  • Schema labels and help text added for UI discoverability

Security Impact (required)

  • New permissions/capabilities? Yes — opt-in capability to bypass SSRF protection for specific CIDR ranges
  • Secrets/tokens handling changed? No
  • New/changed network calls? No (same fetch surface; destination allowlist is relaxed only when opted in)
  • Command/tool execution surface changed? No
  • Data access scope changed? Yes — when allowCidrs is set, tool can reach specified IP ranges
  • Risk + mitigation: Users could allowlist dangerous ranges. Mitigation: (1) default is empty (fully restrictive), (2) explicit opt-in required, (3) each CIDR must be individually listed — no wildcards, (4) consistent with existing patterns (browser.ssrfPolicy.allowPrivateNetwork, dangerouslyAllowPrivateNetwork)

Repro + Verification

Environment

  • OS: macOS 15.x (Apple Silicon)
  • Runtime: Node.js v22.22.0
  • Proxy: Surge Enhanced Mode (fake-IP to 198.18.0.0/15)
  • Config: { "tools": { "web": { "fetch": { "ssrfPolicy": { "allowCidrs": ["198.18.0.0/15"] } } } } }

Steps

  1. Run OpenClaw behind Surge Enhanced Mode (or any fake-IP proxy)
  2. Use web_fetch to access any public URL (e.g. https://example.com)
  3. DNS resolves to 198.18.x.x (fake-IP range) → SSRF guard blocks it

Expected

With allowCidrs: ["198.18.0.0/15"], web_fetch should succeed.

Actual (before fix)

Blocked: resolves to private/internal/special-use IP address for every URL.

Actual (after fix)

web_fetch succeeds for all public URLs, including cursor.com, baidu.com, etc.

Evidence

  • npx vitest run src/infra/net/fetch-guard.ssrf.test.ts — 8/8 passed (including updated RFC2544 opt-in test)
  • npx vitest run src/shared/net/ip.test.ts — 4/4 passed
  • npm run build — clean, no type errors
  • npx oxfmt --check — clean
  • Live testing: web_fetch https://cursor.com/pricing succeeds with allowCidrs: ["198.18.0.0/15"] behind Surge

Human Verification (required)

  • Verified scenarios: (1) web_fetch blocked without config, (2) web_fetch works with allowCidrs for fake-IP proxy, (3) invalid CIDRs logged as warning and skipped, (4) empty allowCidrs = default restrictive behavior
  • Edge cases checked: IPv6 CIDRs gracefully skipped for IPv4 addresses, malformed CIDR strings, empty array
  • What you did not verify: IPv6-only proxy environments, performance under very large CIDR lists (>100)

Compatibility / Migration

  • Backward compatible? Yes — new config is optional, defaults maintain current restrictive behavior
  • Config/env changes? New optional tools.web.fetch.ssrfPolicy.allowCidrs
  • Migration needed? No

Failure Recovery (if this breaks)

  • Remove ssrfPolicy.allowCidrs from config → behavior reverts to default restrictive SSRF policy
  • No data migration, no schema versioning needed

Risks and Mitigations

  • Risk: Users allowlist overly broad ranges (e.g. 0.0.0.0/0)
    • Mitigation: This is an explicit opt-in; dangerouslyAllowPrivateNetwork already provides full bypass. allowCidrs is strictly more restrictive since each range must be individually listed.
  • Risk: CIDR parsing overhead per request
    • Mitigation: Typical lists are 1-3 entries; ipaddr.parseCIDR is trivial. Could be cached if needed in future.

@openclaw-barnacle openclaw-barnacle bot added agents Agent runtime and tooling size: S labels Feb 27, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0718852bd4

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

}

const allowPrivateNetwork = resolveAllowPrivateNetwork(params.policy);
const parsedCidrs = parseCidrStrings(params.policy?.allowCidrs);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve legacy RFC2544 opt-in compatibility

resolvePinnedHostnameWithPolicy now derives the SSRF override only from params.policy?.allowCidrs, so existing callers that still pass allowRfc2544BenchmarkRange lose their exemption and 198.18.0.0/15 becomes blocked again. I verified this still exists in-tree (for example src/telegram/bot/delivery.ts uses allowRfc2544BenchmarkRange: true), which means fake-IP proxy deployments can regress back to blocked media/download fetches even after this change.

Useful? React with 👍 / 👎.

Comment on lines +265 to +266
if (base.kind() !== "ipv4") {
continue;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor allowCidrs for IPv6 ranges

This loop ignores every non-IPv4 CIDR, but the new policy parsing/type path accepts IPv6 CIDRs as valid inputs, so entries like fd00::/8 are silently ignored and IPv6 special-use addresses remain unconditionally blocked. That makes the new allowCidrs option inconsistent for IPv6-only/internal-network environments because user-provided config appears valid but has no effect.

Useful? React with 👍 / 👎.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Feb 27, 2026

Greptile Summary

Added flexible CIDR-based allowlisting for SSRF protection to fix web_fetch in fake-IP proxy environments.

Key changes:

  • Replaced specific allowRfc2544BenchmarkRange boolean with generic allowCidrs array in SSRF policy
  • Users can now specify exact CIDR ranges (e.g., ["198.18.0.0/15"]) to bypass SSRF blocking
  • Invalid CIDRs are logged and gracefully skipped without crashing
  • Default behavior remains fully restrictive (empty allowlist)

Architecture:

  • Config flows from tools.web.fetch.ssrfPolicy.allowCidrs through web-fetch.tsssrf.tsip.ts
  • CIDR parsing happens once per request using ipaddr.parseCIDR()
  • IPv6 CIDRs are silently skipped when validating IPv4 addresses (correct behavior)
  • Matching logic: if address matches any allowed CIDR → allow; else check blocked ranges

Validation:

  • Zod schema enforces allowCidrs must be string array
  • Runtime parsing catches malformed CIDRs with try-catch
  • RFC2544 range (198.18.0.0/15) now blocked by default, allowable via explicit CIDR

Confidence Score: 5/5

  • Safe to merge - well-implemented security feature with proper defaults and validation
  • Clean implementation with no logic errors, maintains security-by-default, comprehensive testing, and proper error handling. The feature is opt-in, backward compatible, and follows existing patterns (similar to allowPrivateNetwork). CIDR validation is defensive with graceful fallbacks.
  • No files require special attention

Last reviewed commit: 0718852

@openclaw-barnacle
Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle bot added the stale Marked as stale due to inactivity label Mar 5, 2026
@xiaoyaner0201

This comment was marked as spam.

@openclaw-barnacle openclaw-barnacle bot removed the stale Marked as stale due to inactivity label Mar 6, 2026
@xiaoyaner0201

This comment was marked as spam.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c62411cb40

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

url: params.url,
maxRedirects: params.maxRedirects,
timeoutSeconds: params.timeoutSeconds,
policy: params.ssrfPolicy,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Block Firecrawl fallback for allowlisted private targets

Passing params.ssrfPolicy into fetchWithWebToolsNetworkGuard lets web_fetch reach private/internal URLs when allowCidrs (or allowPrivateNetwork) is configured, but the existing fallback path still calls maybeFetchFirecrawlWebFetchPayload on fetch errors/non-OK responses and forwards the same URL to Firecrawl (runWebFetch in this file). In environments where Firecrawl is enabled, this newly allows internal hostnames/paths/query strings to be sent to an external service, which is a security/privacy regression compared to prior behavior where those targets were blocked before any fallback.

Useful? React with 👍 / 👎.

treblam added a commit to treblam/openclaw that referenced this pull request Mar 7, 2026
千乘妍 (Xiaoyaner) added 3 commits March 13, 2026 17:02
Add tools.web.fetch.ssrfPolicy config with:
- allowCidrs: string[] — CIDR ranges to allow through SSRF protection
  (e.g. '198.18.0.0/15' for fake-IP proxies, '10.0.0.0/8' for internal nets)

This is a generic alternative to the specific allowRfc2544BenchmarkRange
approach. CIDR strings are parsed once per request with ipaddr.parseCIDR;
invalid CIDRs are warned and skipped. IPv6 CIDRs are gracefully handled
when checking IPv4 addresses.

Changes:
- Config: ssrfPolicy.allowCidrs in zod schema, types, labels, help
- Runtime: thread allowCidrs from config → resolveSsrfPolicy → runWebFetch
  → fetchWithSsrFGuard → resolvePinnedHostnameWithPolicy
- SSRF: parseCidrStrings() helper, updated type signatures throughout
- IP: isBlockedSpecialUseIpv4Address checks allowCidrs before blocked ranges

All defaults remain restrictive. dangerouslyAllowPrivateNetwork remains
as the nuclear option for full private network bypass.

Closes openclaw#28271, closes openclaw#26945. Related: openclaw#28349, openclaw#25229.
…owRfc2544BenchmarkRange

Tests were still referencing the old allowRfc2544BenchmarkRange property
which was replaced by the generic allowCidrs array. Updated both
ssrf.test.ts and ssrf.pinning.test.ts to use the new API with
ipaddr.parseCIDR('198.18.0.0/15') for equivalent RFC2544 allowlisting.
@openclaw-barnacle openclaw-barnacle bot added channel: discord Channel integration: discord channel: slack Channel integration: slack channel: telegram Channel integration: telegram labels Mar 13, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bd321ac097

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +264 to +265
const [base, prefix] = cidr;
if (base.kind() !== "ipv4") {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Guard allowCidrs entries before dereferencing CIDR base

This loop assumes each allowCidrs entry is a parsed tuple and immediately calls base.kind(), but the exported policy shape uses string CIDRs; if a caller forwards SsrFPolicy.allowCidrs directly to isPrivateIpAddress/isBlockedHostnameOrIp (both exported through plugin-sdk), base becomes a string and this throws TypeError: base.kind is not a function instead of applying SSRF checks. That creates a runtime crash path for JS/untyped integrations using the new allowCidrs field.

Useful? React with 👍 / 👎.

@openclaw-barnacle
Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle bot added the stale Marked as stale due to inactivity label Apr 3, 2026
@Timqt
Copy link
Copy Markdown

Timqt commented Apr 3, 2026

Real-world use case: Telegram channel media download blocked by SSRF behind private proxy

Just confirming this PR is still very much needed, and adding a concrete real-world scenario that is currently blocked.

Our setup

  • OpenClaw version: 2026.4.2
  • Running behind a private HTTP proxy at 192.168.71.222:20170
  • The proxy is used for Telegram channel outbound calls (channels.telegram.proxy: "http://192.168.71.222:20170")

The problem

When the Telegram channel receives a message with inline media (photo/video), OpenClaw attempts to download the media file from api.telegram.org/file/.... This download goes through the configured proxy. The proxy resolves api.telegram.org to a private IP or the connection to the proxy itself involves a private IP hop.

OpenClaw's SSRF protection blocks this with:

[security] blocked URL fetch (url-fetch) target=https://api.telegram.org/file/bot... reason=Blocked hostname (not in allowlist): 192.168.71.222

The blocked hostname is 192.168.71.222 — the private proxy IP.

Current workaround attempts

We tried adding ssrfPolicy.allowCidrs under tools.web.fetch:

{
  "tools": {
    "web": {
      "fetch": {
        "enabled": true,
        "ssrfPolicy": {
          "allowCidrs": ["192.168.0.0/16"]
        }
      }
    }
  }
}

This had no effect — the field appears to not be recognized in v2026.4.2. We also tried 198.18.0.0/15 (RFC 2544 range) without success. The schema in tools.web.fetch does not appear to accept ssrfPolicy as a valid key in this version.

Impact

Telegram media (photos, videos, documents) cannot be received — every inbound media message triggers the SSRF block. This affects every user who runs OpenClaw behind a private proxy (common in China for accessing Telegram, and common for users running Clash/mihomo/Surge in fake-IP mode).

Request

This PR (or a released version containing it) would solve the problem by allowing:

{
  "tools": {
    "web": {
      "fetch": {
        "ssrfPolicy": {
          "allowCidrs": ["192.168.0.0/16", "198.18.0.0/15"]
        }
      }
    }
  }
}

Is there anything blocking this from being merged? Happy to help test. The March 7 rebase update with 13/13 tests passing sounds very close to ready.

@openclaw-barnacle openclaw-barnacle bot removed the stale Marked as stale due to inactivity label Apr 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling channel: discord Channel integration: discord channel: slack Channel integration: slack channel: telegram Channel integration: telegram size: S

Projects

None yet

2 participants