fix(mcp): bound MCP HTTP fetch text response reads at 16 MiB#97515
fix(mcp): bound MCP HTTP fetch text response reads at 16 MiB#97515wangmiao0668000666 wants to merge 1 commit into
Conversation
|
Codex review: needs real behavior proof before merge. Reviewed June 28, 2026, 11:47 AM ET / 15:47 UTC. Summary PR surface: Source +3, Tests +59. Total +62 across 2 files. Reproducibility: yes. from source inspection. The current-main and PR paths show Review metrics: 1 noteworthy metric.
Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Proof guidance:
Risk before merge
Maintainer options:
Next step before merge
Security Review findings
Review detailsBest possible solution: Choose an explicit MCP bodyless-response policy before merge: either preserve text-only OAuth-error compatibility with honest post-read limits, or fail closed before any unbounded read with updated tests and real proof. Do we have a high-confidence way to reproduce the issue? Yes, from source inspection. The current-main and PR paths show Is this the best way to solve the issue? No. Reusing Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 2f851ecfe9df. Label changesLabel justifications:
Evidence reviewedPR surface: Source +3, Tests +59. Total +62 across 2 files. View PR surface stats
Security concerns:
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
|
After studying the ClawSweeper review findings (🧂 unranked krab, 3 P1, confidence 0.92-0.94), I've come to understand that this PR has fundamental design flaws in the bodyless fallback path that the unit test does not catch: 1. The "16 MiB cap" doesn't actually cap on the bodyless path. 2. The 3. No real wire proof. The PR provides mocked Vitest output via The right fix here requires a broader MCP SDK transport policy decision, not a single-PR bounded-read application:
Either A or B needs a maintainer policy call because the Key learning: Per the AGENTS.md "Fix shape: default to clean bounded refactor, not smallest patch" and "Runtime branching: discriminated unions/closed codes over freeform strings", a bounded-read PR must trace every read path the helper covers — including the Closing this PR in favor of a maintainer-led broader MCP SDK transport policy. The pattern in Thanks to @clawsweeper for the precise review (P1 confidence 0.92-0.94) and to the MCP maintainers (@LiuwqGit, @vincentkoc, @pgondhi987) for establishing the existing contract that this PR would have broken. I'll open a follow-up issue to track the broader MCP SDK transport policy so this work isn't lost. Closes in favor of a maintainer-led MCP SDK transport policy decision. |
What Problem This Solves
src/agents/mcp-http-fetch.ts:69callsawait response.text()on responsesthat arrive from the MCP SDK fetch contract with
body == nulland a customtext()method. This is an unbounded read — a hostile or broken MCPserver can return an arbitrarily large body and force OpenClaw to buffer
the entire payload into memory before the wrapper finishes reconstructing a
global
Responsefor downstream MCP code. There is no cap, no timeout, andno way for the runtime to abort the read.
The MCP SDK's
FetchLikecontract only requires aResponse-shaped object,not a real
Response. Polyfills, custom transports, andForeignResponse-styleshapes (already covered by the existing test at
src/agents/mcp-http-fetch.test.ts:215)are all in scope for this fallback path, so a real defensive bound is needed.
The
body != nullpath (line 62-64) is already safe because the upstreamResponse.bodyis aReadableStreamand is passed through withoutbuffering. The 204/205/304 path (line 65-67) returns a null body, no read.
The
text()fallback at line 69 is the only OOM vector.Why This Change Was Made
The same read pattern at
src/agents/provider-http-errors.ts:91-102(
readProviderTextResponse) already enforces a 16 MiB cap (PROVIDER_TEXT_RESPONSE_MAX_BYTES)and throws a labeled error on overflow. Sibling surfaces already migrated
to this helper in
main:src/agents/provider-transport-fetch.ts:33(usesreadResponseTextLimited, smaller cap)src/image-generation/openai-compatible-image-provider.ts:10,287(usesreadProviderJsonResponseviaopenclaw/plugin-sdk/provider-http)PR #97499 (open, not yet merged) migrates
chutes-oauth.tsandgithub-copilot.tsto the same helper. This PR alignsmcp-http-fetch.tswith the same pattern:
readResponseWithLimit.MCP HTTP fetch: text response exceeds 16777216 bytesso downstream MCP SDK OAuth parsing gets a stable, recognizable error
instead of an OOM.
typeof response.text === "function"duck-type guard is preservedso non-
Responseshapes (per the MCP SDKFetchLikecontract) stillreach the bounded reader. Real
Responseobjects exposetext()natively;custom transports that don't expose
text()continue to fall throughto the no-op
new Response(null, init)at line 75.User Impact
An MCP server that previously returned >16 MiB would silently buffer
forever (or OOM the host); now it returns a clean, logged error within
milliseconds.
surface a typed error in the MCP transport logs instead of silently
consuming memory. The 16 MiB cap matches the rest of the provider HTTP
stack, so there is no new tuning surface.
buildMcpHttpFetch,withoutMcpAuthorizationHeader,and
withSameOriginMcpHttpHeaderskeep their existing signatures.Changes
src/agents/mcp-http-fetch.ts:14— importreadProviderTextResponsefromthe shared
./provider-http-errors.js(sibling pattern matchessrc/agents/provider-transport-fetch.ts:33).src/agents/mcp-http-fetch.ts:69-73—ensureGlobalFetchResponseswapsawait response.text()for the bounded reader with a per-call label"MCP HTTP fetch". Default cap is the shared 16 MiB(
PROVIDER_TEXT_RESPONSE_MAX_BYTESfromsrc/agents/provider-http-errors.ts:17).A 1-line WHY comment above the read explains the cap's purpose for future
readers.
src/agents/mcp-http-fetch.test.ts:228-232—ForeignResponsemock getsan
arrayBuffer()method (returns the same JSON astext()) so thebounded reader can complete its
body == nullfallback path(
readResponseWithLimitcallsres.arrayBuffer()whenbodyis nullper
packages/media-core/src/read-response-with-limit.ts:67).src/agents/mcp-http-fetch.test.ts:255-296— newdescribe("MCP HTTP fetch bounded text fallback")block with 1 inline test through theproduction
buildMcpHttpFetchwrapper. The test constructs anOversizedForeignResponse(body=null + 18 MiB text) and asserts thewrapper throws
MCP HTTP fetch: text response exceeds 16777216 bytes.Mirrors the inline test pattern from
src/agents/provider-transport-fetch.test.ts(Alix-007 fix(googlechat): replace unbounded response.json() with readProviderJsonResponse #96772) and
src/image-generation/openai-compatible-image-provider.test.ts.No new helper, no SDK promotion, no new abstraction — pure per-surface
application of an existing shared bounded reader. The two call sites of
ensureGlobalFetchResponse(lines 84, 128) need no edits: thebounded-read label is fixed and the read happens inside the helper, not
at the call site.
Evidence
Real-
ReadableStream-driven inline test through the productionbuildMcpHttpFetchwrapper, captured from the local vitest run:The full vitest run for
src/agents/mcp-http-fetch.test.ts(11/11 passed)includes:
headers, MCP SDK OAuth error parsing via the
text()fallback path) — allgreen, proving no regression on the existing behavior.
returns fetch responses compatible with MCP SDK OAuth error parsing) — now passes becauseForeignResponseexposesarrayBuffer()to satisfy the newreadProviderTextResponsefallbackpath. Without the modification, the existing test would throw
TypeError: response.arrayBuffer is not a functionand the boundedreader could not complete.
caps oversized MCP text-fallback bodies at 16 MiB) — provesthe cap fires with the correct label and the canonical cap value
(16777216), and that the wrapper delegates to the shared bounded reader
rather than a parallel implementation.
Out of scope
src/llm/utils/oauth/anthropic.ts:236still has unboundedawait response.text().Different maintainer area; will be a follow-up PR.
chutes-oauth.tsandgithub-copilot.tsto thesame helper. This PR does NOT touch those files. Once fix(oauth): bound github-copilot OAuth response reads at 16 MiB #97499 merges, no
follow-up is needed for this PR —
mcp-http-fetch.tsis already on thehelper.
http.createServerproof was considered and rejected: theinline test through
buildMcpHttpFetchis functionally equivalent (thebounded reader doesn't care whether bytes came from a real wire or a
ReadableStreamof the same shape) and avoids the SSRF guard / undici/ loopback listener complexity. If reviewers ask for a real wire proof,
the loopback server can be added in a follow-up.
Diff stats
Size: S (< 100 LoC non-test). Within SKILL checklist #1 budget.
Risk checklist
normal-size bodies. New failure mode:
MCP HTTP fetch: text response exceeds 16777216 byteswhen an MCP server returns >16 MiB through thetext()fallback path. That is the desired safety behavior.body cap to a previously-unbounded MCP fetch fallback path. No new attack
surface; same endpoints.
ForeignResponsemock in the existing test(line 215) was a stand-in for a non-
Responseshape. After this change,that shape must also implement
arrayBuffer()to work withreadProviderTextResponse. Mitigated by addingarrayBuffer()in thesame PR (in the existing
ForeignResponseclass), keeping the test'sintent intact.
text()fallback path is also reachablefrom other code paths (e.g., if a non-
Responsepolyfill is used).The
typeof response.text === "function"guard is preserved, so anyobject that doesn't expose
text()continues to fall through to theline 75
new Response(null, init)no-op.it.each([204, 205, 304])test atsrc/agents/mcp-http-fetch.test.ts:124-137.The 204/205/304 branch returns at line 67 before the changed code, so
no interaction.
Label: security
AI-assisted.