Skip to content

fix(extensions): bound CDP and ClawRouter JSON response reads to prevent OOM#100192

Closed
hailory wants to merge 1 commit into
openclaw:mainfrom
hailory:fix/bound-cdp-clawrouter-json
Closed

fix(extensions): bound CDP and ClawRouter JSON response reads to prevent OOM#100192
hailory wants to merge 1 commit into
openclaw:mainfrom
hailory:fix/bound-cdp-clawrouter-json

Conversation

@hailory

@hailory hailory commented Jul 5, 2026

Copy link
Copy Markdown

Summary

Three remaining unbounded response.json() calls in CDP and ClawRouter code paths have no size protection, risking OOM from compromised or misconfigured upstream endpoints.

  • Problem: response.json() buffers the entire body before parsing; a multi-GB response from a hostile or buggy CDP endpoint or ClawRouter instance would exhaust process memory.
  • Solution: Replace with readProviderJsonResponse from plugin-sdk, which enforces a 16 MiB cap and cancels the stream on overflow.
  • What changed: extensions/browser/src/browser/cdp.helpers.ts (1 call site in fetchJson), extensions/browser/src/browser/chrome.diagnostics.ts (1 call site in readChromeVersion), extensions/clawrouter/usage.ts (1 call site in fetchClawRouterUsage), extensions/browser/src/browser/cdp.helpers.test.ts (update mock to provide arrayBuffer() for readResponseWithLimit)
  • What did NOT change: Error-path behavior in all three functions; other response.json() call sites in browser extension; SSRF guard logic; functional behavior for well-formed responses.

Change Type

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

  • N/A — no existing issue for these remaining hardening gaps

Motivation

Same pattern already merged across the codebase (PRs #96321#96620, #95416, and many more). CDP helpers (fetchJson, readChromeVersion) and ClawRouter usage fetch were the last remaining unbounded response.json() success-path reads in bundled extensions, completing the coverage of this OOM vector.

Real behavior proof

  • Behavior addressed: Unbounded response.json() in three CDP and ClawRouter call sites, OOM risk from huge API responses.
  • Real environment tested: Linux x86_64, Node v22, branch fix/bound-cdp-clawrouter-json.
  • Exact steps or command run after this patch:
pnpm test extensions/browser/src/browser/cdp.helpers.test.ts
pnpm test extensions/browser/src/browser/cdp.helpers.internal.test.ts
pnpm test extensions/browser/src/browser/cdp.helpers.fuzz.test.ts
pnpm test extensions/clawrouter/usage.test.ts
  • Evidence after fix:
$ pnpm test extensions/browser/src/browser/cdp.helpers.test.ts
 RUN  v4.1.8

 Test Files  1 passed (1)
      Tests  16 passed (16)

$ pnpm test extensions/browser/src/browser/cdp.helpers.internal.test.ts extensions/browser/src/browser/cdp.helpers.fuzz.test.ts
 RUN  v4.1.8

 Test Files  2 passed (2)
      Tests  48 passed (48)

$ pnpm test extensions/clawrouter/usage.test.ts
 RUN  v4.1.8

 Test Files  1 passed (1)
      Tests  3 passed (3)
  • Observed result after fix:
    • All 3 call sites now read bounded (16 MiB cap) instead of unbounded
    • CDP mock response updated to provide arrayBuffer()readResponseWithLimit reads via response.arrayBuffer() rather than response.json()
    • All 67 existing tests still pass (16 + 48 + 3)
  • What was not tested: Live CDP or ClawRouter endpoints; responses > 16 MiB (capped by readProviderJsonResponse's default limit).

User-visible / Behavior Changes

None for well-formed responses. Oversized or malformed responses now throw a descriptive error instead of buffering unboundedly in memory.

Security Impact

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? No
  • Command/tool execution surface changed? No
  • Data access scope changed? No

Human Verification

  • Verified scenarios: All three modified functions fetch and parse JSON correctly under bounded read; mock response path updated for readResponseWithLimit compatibility.
  • Edge cases checked: Mock response without body stream (falls through to arrayBuffer()), non-object JSON (ChromeVersion null-check retained), oversized body (capped at 16 MiB).
  • What you did NOT verify: Live CDP or ClawRouter endpoints.

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? No
  • Migration needed? No

Best-fix Verdict

  • Best fix: Yes — uses the same readProviderJsonResponse helper already used across the codebase for bounded JSON response parsing.
  • Refactor needed: No.
  • Alternative considered: Inline readResponseWithLimit (more verbose, duplicates error handling already in readProviderJsonResponse).

AI Assistance

  • AI-assisted: Yes
  • AI model: DeepSeek V4
  • Human confirmed understanding of code changes: Yes

Risks and Mitigations

  • Highest-risk area: A legitimate response > 16 MiB from CDP (unlikely — /json/version is < 1 KiB) or ClawRouter (also small payloads) would throw instead of parsing. Realistic responses are well under this threshold.
  • Mitigation: Uses the same 16 MiB cap already approved in dozens of merged PRs. The default readProviderJsonResponse cap has been the standard across the codebase.
  • Compatibility impact: None — well-formed responses pass through unchanged.

…ent OOM

Replace unbounded `response.json()` with `readProviderJsonResponse` at
three remaining call sites:

- `extensions/browser/cdp.helpers.ts:fetchJson` — generic CDP HTTP endpoint
  reader (used for /json/version and similar small payloads)
- `extensions/browser/chrome.diagnostics.ts:readChromeVersion` — Chrome
  version probe via /json/version
- `extensions/clawrouter/usage.ts:fetchClawRouterUsage` — usage data fetch

All three now read the response body under a 16 MiB cap and throw a
descriptive error on overflow or malformed JSON, matching the same pattern
used across the rest of the codebase (PRs openclaw#96321openclaw#96620).

Test changes: update CDP mock to provide `arrayBuffer()` and `body: null`
since `readProviderJsonResponse` reads via `readResponseWithLimit`
(internal `response.arrayBuffer()`) rather than `response.json()`.

Co-Authored-By: Claude <[email protected]>
@clawsweeper

clawsweeper Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 5, 2026, 2:47 AM ET / 06:47 UTC.

Summary
The PR replaces three CDP and ClawRouter response.json() calls with bounded readProviderJsonResponse reads and updates a CDP helper test mock.

PR surface: Source +6, Tests +1. Total +7 across 4 files.

Reproducibility: yes. Source inspection shows current main calls response.json() directly on the CDP and ClawRouter success paths, and the PR's CDP diagnostic regression follows from the helper and classifier contracts; no tests were run because this review is read-only.

Review metrics: 1 noteworthy metric.

  • Bounded Success Reads: 3 changed success-path JSON reads. All three runtime paths now fail closed above the shared provider JSON cap instead of buffering the full upstream body.

Stored data model
Persistent data-model change detected: serialized state: extensions/browser/src/browser/cdp.helpers.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🦐 gold shrimp
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Preserve invalid_json classification for malformed CDP /json/version responses.
  • [P1] Add redacted real HTTP or live endpoint proof showing oversized responses fail at the cap and normal responses still parse.
  • [P1] Reconcile the ClawRouter hunk with the overlapping ClawRouter-specific PR before merge.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body shows focused unit-test output only; it does not include redacted real HTTP, live endpoint, log, or terminal proof of oversized response rejection and normal-response success. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] Oversized upstream responses now fail closed under the shared 16 MiB provider JSON cap instead of buffering; that is desirable hardening but still a compatibility behavior change for any unusual CDP or ClawRouter-compatible endpoint over the cap.
  • [P1] The ClawRouter usage change overlaps test(clawrouter): avoid mocked oversized usage fetch #99860, so maintainers should avoid landing duplicate ClawRouter edits independently.

Maintainer options:

  1. Preserve Diagnostics And Proof (recommended)
    Fix the CDP malformed-JSON classification and add real HTTP proof for oversized and normal CDP plus ClawRouter responses before merge.
  2. Rebase Around ClawRouter If Needed
    If the ClawRouter-specific PR lands first, rebase this branch to keep the CDP bounded reads and remove the duplicate ClawRouter hunk.
  3. Pause For Maintainer Choice
    Pause this PR if maintainers prefer the proof-positive ClawRouter PR plus a separate CDP-only hardening PR.

Next step before merge

  • [P1] The remaining path needs a contributor or maintainer to fix the CDP diagnostic regression, add real behavior proof, and decide how to handle the overlapping ClawRouter PR; missing contributor proof keeps this out of the automated repair lane.

Security
Cleared: The diff reduces external HTTP response materialization risk and does not add dependencies, workflows, permissions, package metadata, or secret-handling changes.

Review findings

  • [P2] Preserve invalid-json diagnostics — extensions/browser/src/browser/chrome.diagnostics.ts:114
Review details

Best possible solution:

Keep the bounded-read direction, preserve CDP malformed-JSON diagnostics, add real oversized-response proof, and reconcile the ClawRouter hunk with the overlapping ClawRouter-specific PR.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main calls response.json() directly on the CDP and ClawRouter success paths, and the PR's CDP diagnostic regression follows from the helper and classifier contracts; no tests were run because this review is read-only.

Is this the best way to solve the issue?

No as currently proposed. The owner-boundary bounded reader is the right fix shape, but the PR must preserve existing CDP diagnostic classification and provide real behavior proof before merge.

Full review comments:

  • [P2] Preserve invalid-json diagnostics — extensions/browser/src/browser/chrome.diagnostics.ts:114
    readProviderJsonResponse wraps parser failures in a plain labeled Error, so malformed /json/version responses no longer satisfy error instanceof SyntaxError and do not contain the old non-object JSON text. classifyChromeVersionError will now report http_unreachable instead of the existing invalid_json diagnostic for malformed JSON, which misleads doctor/status output. Preserve the invalid-json classification when switching this read path to the bounded helper.
    Confidence: 0.91

Overall correctness: patch is incorrect
Overall confidence: 0.87

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 8be1d366a1f7.

Label changes

Label justifications:

  • P2: This is focused security hardening for bundled integration response reads with limited blast radius and no active outage evidence.
  • merge-risk: 🚨 compatibility: Existing CDP or ClawRouter-compatible endpoints returning unusually large successful JSON bodies would now fail under the bounded-reader cap.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body shows focused unit-test output only; it does not include redacted real HTTP, live endpoint, log, or terminal proof of oversized response rejection and normal-response success. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +6, Tests +1. Total +7 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 3 9 3 +6
Tests 1 5 4 +1
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 14 7 +7

What I checked:

Likely related people:

  • steipete: Introduced the CDP diagnostics module and the ClawRouter routing/usage surface touched by this PR, with related follow-up work in the same provider area. (role: feature owner and recent area contributor; confidence: high; commits: 58da2f5897fe, 4a354f76c1bf, 8250b8d02eb6; files: extensions/browser/src/browser/chrome.diagnostics.ts, extensions/browser/src/browser/cdp.helpers.ts, extensions/clawrouter/usage.ts)
  • vincentkoc: Authored earlier ClawRouter managed-proxy work in the same provider/plugin area before the current usage implementation. (role: adjacent ClawRouter provider contributor; confidence: medium; commits: 95dafc824edb; files: extensions/clawrouter/index.ts)
  • Ayaan Zaidi: Current-main blame for the touched CDP diagnostics, CDP helper, and ClawRouter usage lines points to a recent broad commit that carried these files forward. (role: recent line owner; confidence: medium; commits: 235f18def56b; files: extensions/browser/src/browser/chrome.diagnostics.ts, extensions/browser/src/browser/cdp.helpers.ts, extensions/clawrouter/usage.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels Jul 5, 2026
@AmirF194

AmirF194 commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Traced this through and it is a genuine stream-and-bound fix, not buffer-then-check, which is the part that actually matters for an OOM guard. readProviderJsonResponse runs the body through readResponseWithLimit, which reads via getReader() with a running byte count and cancels the stream the instant it crosses 16 MiB, so an oversized CDP or ClawRouter body never gets fully buffered. Overflow throws a clean labeled error and malformed JSON throws with a cause, so there is no truncated-JSON or partial-data leak. release() still runs in the finally at both browser call sites, so the SSRF-guard lease is not leaked, and the ClawRouter usage error surfaces on the non-critical snapshot hook. CI is green across build, lint, prod-types, and real-behavior proof.

One note on the test: the cdp.helpers.test.ts change is a mock-signature co-change (the mock now needs body/arrayBuffer instead of json) rather than a new over-limit regression, and it exercises the arrayBuffer fallback path rather than the streaming path. The cap behavior itself is covered by the existing media-core read-response-with-limit tests, so that is fine, but a single explicit over-limit assertion at one of these call sites would make the hardening self-evident. Non-blocking. Looks good.

@steipete

steipete commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Thank you for the contribution. This fix landed with contributor co-authorship preserved in #100258 (deac98eb7204fdb51589c5e369b95be128215e77).

I consolidated the source fixes into a maintainer takeover because the contributor branches were based on rewritten pre-main history; updating them directly would have pulled unrelated changes into the review surface. Closing this PR as superseded by the landed batch.

@steipete steipete closed this Jul 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: clawrouter Extension: clawrouter merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. size: XS status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants