Bound proxy error response JSON parsing to avoid loading huge bodies#97351
Conversation
|
Codex review: needs maintainer review before merge. Reviewed June 28, 2026, 12:42 AM ET / 04:42 UTC. Summary PR surface: Source +2, Tests +51. Total +53 across 2 files. Reproducibility: yes. Current main and v2026.6.10 still call Review metrics: 1 noteworthy metric.
Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Risk before merge
Maintainer options:
Next step before merge
Security Review detailsBest possible solution: Land this narrow bounded error-body fix as a discrete proxy hardening step, or deliberately fold it into one broader proxy guard PR before merging. Do we have a high-confidence way to reproduce the issue? Yes. Current main and v2026.6.10 still call Is this the best way to solve the issue? Yes. Reusing AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against c6ade83a5ccb. Label changesLabel justifications:
Evidence reviewedPR surface: Source +2, Tests +51. Total +53 across 2 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
|
Merge-ready for b2f0b70. Local review and ClawSweeper review clean, CI green, best narrow fix, and follows the established bounded provider JSON response-read pattern. This is the right small precursor to #97235: it only bounds the non-OK proxy error JSON branch, preserves the existing status/statusText fallback for malformed or oversized bodies, and leaves the broader proxy SSE/body/idle-timeout policy decisions for #97235 to consider next. No maintainer edits needed. |
Summary
What problem does this PR solve?
/api/streamproxy response path calledResponse.json()directly, so the parser had to read and decode the complete error body before the proxy client could fall back to the HTTP status message.readProviderJsonResponse, the existing bounded JSON response reader backed byreadResponseWithLimit, instead of adding a downstream fallback after the unbounded read already happened.Why does this matter now?
What is the intended outcome?
What is intentionally out of scope?
{ "error": "..." }error extraction.What does success look like?
The proxy client reports the existing
Proxy error: 502 Bad Gatewayfallback for an oversized malformed JSON error response, and the real local HTTP proxy observes the client closing the response before the full oversized body is sent.What should reviewers focus on?
readProviderJsonResponseis already the source-of-truth helper for bounded provider JSON parsing, so this proxy path now follows the same response-size invariant instead of maintaining a separate limit.check-upstream-fixedfound origin/main has not covered this PR and has no overlapping files with the PR diff; newer main changes are unrelated media/plugin/state work.Linked context
Which issue does this close?
No linked issue number; this PR is based on the reported proxy error-response path.
Which issues, PRs, or discussions are related?
Related: ClawSweeper comment requiring real proxy/gateway proof for PR #97351.
Was this requested by a maintainer or owner?
No maintainer request was linked in the local task context.
Real behavior proof (required for external PRs)
Behavior or issue addressed: A non-OK
/api/streamproxy error response with a very large JSON body should return the proxy error result without reading the whole error stream into memory.Real environment tested: Local OpenClaw agent runtime loaded from this PR head, using the real
streamProxyimplementation and a real local HTTP proxy server listening on127.0.0.1for/api/stream.Exact steps or command run after this patch:
node --import tsx <local real-proxy proof script>Evidence after fix (screenshot, recording, terminal capture, console output, redacted runtime log, linked artifact, or copied live output):
Observed result after fix: The real local proxy planned to stream 24 MiB but observed the client connection close after 18 MiB, while the client surfaced
Proxy error: 502 Bad Gateway; this shows the bounded reader stopped the oversized non-OK body before the full response was consumed.What was not tested: No external hosted proxy service was called; the proof uses a real local HTTP proxy endpoint to avoid credentials and non-public infrastructure while exercising the real client fetch path.
Proof limitations or environment constraints: The proof is a localhost proxy/gateway simulation for the client-side parser boundary, not a full deployed OpenClaw gateway run; it directly covers the failure ClawSweeper requested: oversized non-OK proxy body, bounded fallback, and early connection close.
Before evidence (optional but encouraged): The regression test failed before the fix with
expected 20 to be less than 20, showing the old path read all twenty 1 MiB chunks in the controlled streaming Response case.Tests and validation
Which commands did you run?
src/agents/runtime/proxy.test.tsviapnpm exec vitest run src/agents/runtime/proxy.test.ts.What regression coverage was added or updated?
502 Bad Gatewayproxy response withContent-Type: application/jsonstreams oversized chunks; the proxy must return the fallback error and stop reading before the complete body is consumed.What failed before this fix, if known?
streamProxynon-OK response branch and response-reader boundary directly, while the new real local proxy proof exercises the same branch through a real HTTP server and Node fetch path.If no test was added, why not?
A regression test was added in
src/agents/runtime/proxy.test.ts; the current update adds PR-visible proof only and does not change production code.Risk checklist
Did user-visible behavior change? (
Yes/No)No for normal proxy errors; oversized or malformed JSON error bodies now stop at the bounded reader and keep the existing status/statusText fallback.
Did config, environment, or migration behavior change? (
Yes/No)No.
Did security, auth, secrets, network, or tool execution behavior change? (
Yes/No)No; the touched code is proxy/provider-adjacent, but it only changes how non-OK error response bodies are parsed after fetch returns.
What is the highest-risk area?
How is that risk mitigated?
src/agents/runtime/proxy.tsandsrc/agents/runtime/proxy.test.ts, does not touch config/schema/session/auth storage, preserves existing fallback messages, and now includes a real local proxy proof showing early close before full oversized body consumption.Current review state
What is the next action?
Ready for ClawSweeper/maintainer re-review after this PR body update with real proxy proof.
What is still waiting on author, maintainer, CI, or external proof?
RF-001, RF-002, and RF-003 are addressed by the real local HTTP proxy proof above; RF-004 is addressed because no code repair job is needed and this update only supplies contributor proof.
Which bot or reviewer comments were addressed?
Addressed ClawSweeper comment #97351 (comment): added redacted real local proxy output showing bounded fallback and early connection close for an oversized non-OK response body.