fix(parallel): bound successful web-search JSON response reads by Alix-007 · Pull Request #96035 · openclaw/openclaw

Alix-007 · 2026-06-23T08:12:00Z

What Problem This Solves

The Parallel web_search provider parses its /v1/search success response with an unbounded await res.json():

return (await res.json()) as ParallelSearchResponse;

That body comes from an external web-search upstream (api.parallel.ai, or an operator-configured baseUrl proxy) and is explicitly treated as untrusted (externalContent.untrusted: true). With no Content-Length guard and no byte cap, a hostile or malfunctioning endpoint that streams an unbounded or absurdly large JSON body forces the runtime to buffer the entire payload into memory before JSON.parse ever runs. On the provider/plugin path that means memory pressure (potential OOM) or an indefinite hang while the runtime keeps draining bytes — a denial-of-service surface driven entirely by the remote side.

The sibling error-body path was already hardened (readResponseTextLimited, 8 KiB cap, added in "fix(parallel): bound search error bodies"). The success path was the remaining unbounded read. This closes it, mirroring the #95103 / #95108 response-limit campaign and reusing the exact same shared helper landed in #95218.

Changes

parallel-web-search-provider.runtime.ts: replace unbounded await res.json() with the shared readProviderJsonResponse<ParallelSearchResponse>(res, "Parallel API", { maxBytes: 16 MiB }) from openclaw/plugin-sdk/provider-http. This reads through the same bounded reader used for binary/provider JSON responses: on overflow it cancels the underlying stream and throws a bounded error; malformed JSON still surfaces a wrapped error. No new abstraction — reuses the existing helper.
New PARALLEL_SEARCH_RESPONSE_LIMIT_BYTES = 16 * 1024 * 1024 constant (matches readProviderJsonResponse's default / fix(agents): bound provider JSON response reads #95218's PROVIDER_JSON_RESPONSE_MAX_BYTES), exported via testing alongside the existing PARALLEL_ERROR_BODY_LIMIT_BYTES.
parallel-web-search-provider.test.ts: add two regression tests — a multi-chunk streamed oversized body that must throw the bounded error, stop pulling chunks early, and cancel the stream; and a well-formed body that still parses under the cap.

Real behavior proof

Behavior addressed: an unbounded successful-JSON read on the Parallel web-search path lets a remote endpoint force unbounded memory buffering / hang; the fix caps the read at 16 MiB and cancels the stream on overflow.
Real environment tested: a real node:http server bound to 127.0.0.1 streaming a JSON body with no Content-Length in 1 MiB chunks (64 MiB total, 4× the cap), driving the real exported executeParallelWebSearchProviderTool(...) via a real fetch() Response. Only the SSRF-guarded transport wrapper (which by design refuses loopback) was swapped for a plain fetch; the code under test — the new bounded read — ran unmodified.
Exact steps: node --import <tsx+resolve-hook> proof.mts → (1) drive the real exported function against the oversized stream; (2) negative control reading the same body with an unbounded reader; (3) drive it against a small well-formed body.
Evidence after fix:
- Real exported fn threw Parallel API: JSON response exceeds 16777216 bytes.
- Server sent only ~20 MiB of the 64 MiB body before the read stopped, and observed a client disconnect → the underlying stream was cancelled on overflow (no full buffering).
- Negative control (unbounded read) buffered the entire 64 MiB → the 16 MiB cap is load-bearing, not incidental.
- Small well-formed body still parsed end-to-end (provider: "parallel").
Observed result: oversized → bounded error + early stream cancel; unbounded baseline → full buffer (regression reproduced); normal → unaffected.
What was not tested: the SSRF/host-trust transport layer itself (out of scope, already covered upstream) and a truly infinite (never-ending) body — the cap makes the read terminate deterministically, so an infinite stream is bounded by the same code path the finite oversized fixture exercises.

Evidence

PASS: real runtime exports PARALLEL_SEARCH_RESPONSE_LIMIT_BYTES=16777216 (16 MiB)
PASS: oversized stream -> real fn threw bounded error: "Parallel API: JSON response exceeds 16777216 bytes"
PASS: bounded read stopped early: server sent 20971520 bytes of 67108864 (stream cancelled before draining)
PASS: server observed client disconnect -> overflow cancelled the underlying stream
PASS: negative control (unbounded read) buffered the ENTIRE 67108864 bytes (>= 16 MiB cap) -> the byte cap is load-bearing
PASS: normal small well-formed response still parses end-to-end: provider=parallel
ALL PROOF CHECKS PASSED

Regression suite (node scripts/run-vitest.mjs extensions/parallel/src/parallel-web-search-provider.test.ts --run): 29 passed. oxlint on changed files: clean. tsgo -p tsconfig.extensions.json: no errors in extensions/parallel (only a pre-existing unrelated crabline module-resolution issue in extensions/qa-lab).

Label: security

🤖 AI-assisted: implementation and proof harness authored with AI assistance; all code, tests, and the real-environment proof were reviewed and run locally by the author. Symmetric follow-up to the #95103 / #95108 response-limit campaign.

The Parallel web_search provider parsed its /v1/search success body with an unbounded await res.json(). The body comes from an external web-search upstream, so a hostile or malfunctioning endpoint streaming an unbounded JSON payload could force the runtime to buffer the whole response before parsing, creating memory pressure or a hang on the provider path. Read the success body through the shared readProviderJsonResponse helper with a 16 MiB cap (matching the provider JSON cap from openclaw#95218); on overflow the stream is cancelled and a bounded error is thrown. The error-body path was already bounded (readResponseTextLimited, 8 KiB). Symmetric follow-up to the openclaw#95103/openclaw#95108 response-limit campaign.

Replace the PR-specific 'openclaw#95218' annotation with a neutral description of the shared provider JSON cap so the comment stays accurate independent of upstream PR numbering.

Alix-007 · 2026-06-23T10:19:22Z

@clawsweeper review

clawsweeper · 2026-06-23T10:19:26Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

clawsweeper · 2026-06-23T20:56:21Z

Codex review: needs maintainer review before merge. Reviewed June 23, 2026, 5:00 PM ET / 21:00 UTC.

Summary
The PR routes the paid Parallel web_search success response through the shared bounded JSON reader, adds a 16 MiB cap constant, and tests oversized-stream cancellation plus normal parsing.

PR surface: Source +8, Tests +93. Total +101 across 2 files.

Reproducibility: yes. source inspection on current main shows the paid Parallel success path still uses unbounded res.json(), and the PR body gives a concrete streaming-server proof path for oversized and normal responses. I did not run the harness in this read-only review.

Review metrics: 1 noteworthy metric.

External success reads: 1 paid Parallel JSON success path bounded at 16 MiB. This is the review-relevant behavior change: a previously unbounded external response read now uses the shared provider byte cap.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Next step before merge

No ClawSweeper repair lane is needed because the PR has no actionable code findings; it is ready for normal maintainer review and merge gating.

Security
Cleared: The diff reduces an external-response DoS surface and introduces no new dependencies, CI code execution, secrets handling, or broader permissions.

Review details

Best possible solution:

Land the narrow bounded-read change with the added regression tests, keeping the cap in the provider-owned Parallel runtime and the byte-reading mechanics in the shared provider HTTP helper.

Do we have a high-confidence way to reproduce the issue?

Yes, source inspection on current main shows the paid Parallel success path still uses unbounded res.json(), and the PR body gives a concrete streaming-server proof path for oversized and normal responses. I did not run the harness in this read-only review.

Is this the best way to solve the issue?

Yes; this is the best small fix because it reuses the shared provider JSON byte-cap helper from the public plugin SDK seam instead of adding a Parallel-local reader or a weaker Content-Length-only guard.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 252673d5b19e.

Label changes

Label changes:

add P2: This is a normal-priority security hardening fix for an optional bundled web-search provider with limited blast radius.
add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal proof from a real local HTTP streaming setup showing bounded overflow, stream cancellation, a negative control, and normal parsing; no contributor action is needed.
add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes after-fix terminal proof from a real local HTTP streaming setup showing bounded overflow, stream cancellation, a negative control, and normal parsing; no contributor action is needed.

Label justifications:

P2: This is a normal-priority security hardening fix for an optional bundled web-search provider with limited blast radius.
rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes after-fix terminal proof from a real local HTTP streaming setup showing bounded overflow, stream cancellation, a negative control, and normal parsing; no contributor action is needed.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal proof from a real local HTTP streaming setup showing bounded overflow, stream cancellation, a negative control, and normal parsing; no contributor action is needed.

Evidence reviewed

PR surface:

Source +8, Tests +93. Total +101 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	14	6	+8
Tests	1	93	0	+93
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	107	6	+101

What I checked:

Repository policy read: Read the full root review policy and the scoped extensions policy; the relevant guidance is to review beyond the diff, keep plugin code on public SDK seams, and treat untrusted provider responses/security hardening carefully. (AGENTS.md:1, 252673d5b19e)
Scoped extension policy read: The touched bundled plugin stays within the extension boundary by importing openclaw/plugin-sdk/provider-http and does not reach into core internals directly. (extensions/AGENTS.md:1, 252673d5b19e)
Current main has the unbounded read: On current main, the paid Parallel REST provider handles an OK response by calling await res.json() inside runParallelSearch, so the success path is still unbounded before this PR. (extensions/parallel/src/parallel-web-search-provider.runtime.ts:154, 252673d5b19e)
PR replaces the unbounded read with the shared cap: The PR imports readProviderJsonResponse, adds PARALLEL_SEARCH_RESPONSE_LIMIT_BYTES = 16 * 1024 * 1024, and replaces the success-path res.json() call with readProviderJsonResponse<ParallelSearchResponse>(..., { maxBytes }). (extensions/parallel/src/parallel-web-search-provider.runtime.ts:160, ad6c8194f12f)
Shared helper contract checked: readProviderJsonResponse already reads via readResponseWithLimit, throws a labeled overflow error, and wraps malformed JSON with the caller label, matching the PR's intended behavior. (src/agents/provider-http-errors.ts:297, 252673d5b19e)
Byte-limit helper cancels streams on overflow: readResponseWithLimit stops when the next chunk exceeds the max byte count, cancels the reader, and throws via onOverflow, so the PR's dependency is the correct layer for the DoS guard. (packages/media-core/src/read-response-with-limit.ts:95, 252673d5b19e)

Likely related people:

vincentkoc: Blame and log history show the current Parallel paid web-search runtime and its adjacent tests were introduced in commit fa0427347af26565cecb8d39acaff24898ad2459; the same person also authored related response-limit hardening PRs in the campaign. (role: recent area contributor; confidence: high; commits: fa0427347af2, b073d7cc11dc, d6cefe26f499; files: extensions/parallel/src/parallel-web-search-provider.runtime.ts, extensions/parallel/src/parallel-web-search-provider.test.ts, src/gateway/model-pricing-cache.ts)
Alix-007: Merged PR https://github.com/openclaw/openclaw/pull/95218 authored the shared bounded provider JSON helper reused by this PR, so this contributor has recent history on the exact helper contract beyond only opening this PR. (role: adjacent hardening contributor; confidence: medium; commits: a15f8e3aaac5, 2592f8a51a4e; files: src/agents/provider-http-errors.ts, src/agents/provider-http-errors.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

sallyom · 2026-06-24T14:54:16Z

Merge-ready from my review.

This follows the same response-body cap pattern as #96042 and the sibling Exa/Parallel/Ollama PRs: replace one unbounded success JSON read from an untrusted provider endpoint with the existing bounded reader, preserve normal small-response behavior, and add focused overflow/cancel coverage.

I ran local autoreview in the PR worktree against origin/main: clean, no accepted/actionable findings. I also reviewed upgrade/compatibility risk and found no blocking concerns: no config, persisted state, migration, plugin SDK, or provider API compatibility blocker. The only user-visible behavior change is the intended bounded failure path for malformed or oversized JSON.

Best-fix verdict: this is the right narrow fix because it reuses the existing bounded provider JSON helper instead of adding a broader provider refactor.

…law#96035) * fix(parallel): bound successful web-search JSON response reads The Parallel web_search provider parsed its /v1/search success body with an unbounded await res.json(). The body comes from an external web-search upstream, so a hostile or malfunctioning endpoint streaming an unbounded JSON payload could force the runtime to buffer the whole response before parsing, creating memory pressure or a hang on the provider path. Read the success body through the shared readProviderJsonResponse helper with a 16 MiB cap (matching the provider JSON cap from openclaw#95218); on overflow the stream is cancelled and a bounded error is thrown. The error-body path was already bounded (readResponseTextLimited, 8 KiB). Symmetric follow-up to the openclaw#95103/openclaw#95108 response-limit campaign. * docs(parallel): drop upstream PR ref from response-cap comment Replace the PR-specific 'openclaw#95218' annotation with a neutral description of the shared provider JSON cap so the comment stays accurate independent of upstream PR numbering.

openclaw-barnacle Bot added extensions: parallel size: S labels Jun 23, 2026

docs(parallel): drop upstream PR ref from response-cap comment

ad6c819

Replace the PR-specific 'openclaw#95218' annotation with a neutral description of the shared provider JSON cap so the comment stays accurate independent of upstream PR numbering.

sallyom merged commit 6163b19 into openclaw:main Jun 24, 2026
116 of 120 checks passed

This was referenced Jun 24, 2026

fix(chutes-oauth): bound Chutes OAuth JSON response reads #96249

Closed

fix(video-generation): bound OpenAI video submitted response reads #96144

Closed

fix(video-generation): bound dashscope task response reads #96036

Closed

hugenshen mentioned this pull request Jun 24, 2026

fix(search): bound perplexity and tavily web-search JSON response reads #96478

Closed

wangmiao0668000666 mentioned this pull request Jun 24, 2026

fix(googlechat): bound Google Chat API + certs success JSON response reads #96483

Closed

clawsweeper Bot mentioned this pull request Jun 24, 2026

fix(parallel): bound MCP success response body read #96510

Closed

github-actions Bot mentioned this pull request Jun 24, 2026

📡 Upstream Digest — 2026-06-24 17:48 UTC curtismercier/openclaw-mods#1145

Open

This was referenced Jun 25, 2026

fix(anthropic): bound streaming 200 success-body SSE reads via shared plugin-sdk guard #96632

Closed

fix(sdk): promote streaming-byte-guard to plugin-sdk + bound 4 core streaming reads #96688

Closed

clawsweeper Bot mentioned this pull request Jun 29, 2026

fix(pdf): bound AI provider response reads in PDF text extraction #97696

Closed

clawsweeper Bot mentioned this pull request Jul 11, 2026

fix(parallel): resolve env search key refs #103803

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(parallel): bound successful web-search JSON response reads#96035

fix(parallel): bound successful web-search JSON response reads#96035
sallyom merged 2 commits into
openclaw:mainfrom
Alix-007:fix/bound-parallel-response

Alix-007 commented Jun 23, 2026

Uh oh!

Alix-007 commented Jun 23, 2026

Uh oh!

clawsweeper Bot commented Jun 23, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 23, 2026 •

edited

Loading

Uh oh!

sallyom commented Jun 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

Alix-007 commented Jun 23, 2026

What Problem This Solves

Changes

Real behavior proof

Evidence

Uh oh!

Alix-007 commented Jun 23, 2026

Uh oh!

clawsweeper Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clawsweeper Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sallyom commented Jun 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clawsweeper Bot commented Jun 23, 2026 •

edited

Loading

clawsweeper Bot commented Jun 23, 2026 •

edited

Loading