fix(proxy): bound SSE parser buffer to prevent OOM by wangmiao0668000666 · Pull Request #96993 · openclaw/openclaw

wangmiao0668000666 · 2026-06-26T08:44:51Z

What Problem This Solves

The streamProxy SSE reader in src/agents/runtime/proxy.ts:231 accumulates decoded chunks into an internal buffer string that only shrinks when a \n line boundary is found. If a hostile or misconfigured proxy endpoint sends data without newline delimiters, the buffer grows unboundedly — the same OOM vector fixed for the OpenAI parseSSE in #96972.

This affects every LLM call routed through the OpenClaw runtime proxy gateway.

Changes

src/agents/runtime/proxy.ts (+10 LoC): Added PROXY_SSE_BUFFER_MAX_BYTES = 64 KiB constant. After each chunk decode, the buffer check throws an error if the accumulated buffer exceeds the cap — preventing unbounded memory growth. Mirrors the parseSSE fix in fix(openai): bound SSE parser buffer to prevent OOM #96972 exactly.
src/agents/runtime/proxy.test.ts (+27 LoC): Added inline test "rejects oversized SSE response without line boundary in proxy reader" — 65 KiB proxy SSE response without \n → buffer cap error.

Real behavior proof

Behavior addressed: Unbounded SSE buffer growth in runtime proxy reader when data arrives without \n delimiters — buffer capped at 64 KiB.
Real environment tested: Linux x86_64, Node 22.19, OpenClaw main@ec737ee74d
Exact steps or command run after this patch:
```
node --import tsx _proof_proxy_cap.mts
```

Evidence after fix:

PASS  oversized body: buffer cap error (server sent 65.0 KiB, cap=64.0 KiB)
PASS  valid SSE: data passes through (server sent 39 B, 1 line(s))
PASS  negative control: unbounded read buffer = 65.0 KiB (server sent 65.0 KiB, cap=64.0 KiB)

Results: 3/3 passed

5/5 unit tests passed: pnpm test src/agents/runtime/proxy.test.ts --run

Observed result after fix:
- Real node:http server delivering 65 KiB SSE without \n → reader throws buffer cap error at 64 KiB. Server-side bytes-sent: 65.0 KiB.
- Same server with valid SSE data (39 B, data: {"type":"done","reason":"stop"}\n\n) → passes through correctly.
- Negative control: same 65 KiB body without the cap → full 65 KiB accumulated in-memory (OOM vector confirmed).
What was not tested: The proxy WebSocket path (uses a different parser); non-streaming proxy calls.

Out of scope

This is the parser-buffer counterpart to the stream-level cap in fix(runtime/proxy): bound streaming success-body SSE reads at 16 MiB #96768 (16 MiB via createSseByteGuard). That PR and this one are complementary — fix(runtime/proxy): bound streaming success-body SSE reads at 16 MiB #96768 caps total bytes, this PR caps the per-frame accumulation.

Risk

Low. The cap (64 KiB) is well above any reasonable proxy event payload. If triggered, the stream errors with a clear message rather than silently terminating. The change is additive — no existing behavior is modified, only a guard added.

AI-assisted; reviewed before submission.

clawsweeper · 2026-06-26T08:51:02Z

Codex review: needs maintainer review before merge. Reviewed June 26, 2026, 5:11 AM ET / 09:11 UTC.

Summary
Adds a 64 KiB tail cap to streamProxy's SSE parser and tests both delimiter-free oversized bodies and large coalesced valid newline-delimited chunks.

PR surface: Source +10, Tests +56. Total +66 across 2 files.

Reproducibility: yes. Current main has a clear source path where streamProxy keeps appending an unterminated decoded SSE tail, and the PR body includes copied live output from a real node:http proof for the delimiter-free overflow case.

Review metrics: 1 noteworthy metric.

Proxy SSE tail cap: 1 added at 64 KiB. This is a new fail-closed parser boundary, so the cap value matters independently of ordinary diff size.

Stored data model
Persistent data-model change detected: serialized state: src/agents/runtime/proxy.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P2] Maintainers should explicitly accept, tune, or coordinate the 64 KiB single-line proxy SSE cap before merge.

Risk before merge

[P1] The 64 KiB unterminated-line cap is a new fail-closed runtime boundary: existing proxy streams with unusually large single-line SSE events will now return an error instead of continuing.
[P1] This PR only caps the parser tail; total streaming response size remains separately covered by the related open proxy stream-byte-cap work.

Maintainer options:

Accept The Tail Cap
Maintainers can accept 64 KiB as the maximum proxy SSE line tail because the PR now caps only unterminated data and preserves coalesced valid events.
Tune The Boundary Before Merge
If proxy deployments may emit larger single-line JSON events, raise the cap or align it with the broader stream-byte policy before landing.
Coordinate With Total-Byte Guard
Maintainers can pause this until the related proxy total-body guard lands so both limits are reviewed as one stream policy.

Next step before merge

No automated repair is needed now; the remaining action is maintainer judgment on the fail-closed cap and normal merge gating.

Security
Cleared: No dependency, workflow, secret, or supply-chain concern was found; the remaining fail-closed behavior is tracked as merge risk rather than a security finding.

Review details

Best possible solution:

Land this tail-buffer guard if maintainers accept the 64 KiB single-line boundary, and handle total proxy stream size through the related shared byte-guard path.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main has a clear source path where streamProxy keeps appending an unterminated decoded SSE tail, and the PR body includes copied live output from a real node:http proof for the delimiter-free overflow case.

Is this the best way to solve the issue?

Yes for the parser-tail OOM vector. The latest PR head applies the cap after draining complete lines, which is the narrow maintainable fix; the separate total-body byte cap remains related follow-up work.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against ec737ee74d9b.

Label changes

Label changes:

add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes copied live output from a real node:http proof showing oversized delimiter-free proxy SSE rejection, a valid small SSE pass, and a negative control.
remove rating: 🧂 unranked krab: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.
remove status: ⏳ waiting on author: Current PR status label is status: 👀 ready for maintainer look.

Label justifications:

P2: This is a focused runtime availability hardening fix with limited blast radius and a clear review path.
merge-risk: 🚨 compatibility: Existing proxy streams with a single valid SSE line over 64 KiB can now fail closed where they previously continued.
merge-risk: 🚨 availability: The new guard prevents OOM-style growth but can also turn an unusually large proxied streaming response into an error result.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes copied live output from a real node:http proof showing oversized delimiter-free proxy SSE rejection, a valid small SSE pass, and a negative control.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied live output from a real node:http proof showing oversized delimiter-free proxy SSE rejection, a valid small SSE pass, and a negative control.

Evidence reviewed

PR surface:

Source +10, Tests +56. Total +66 across 2 files.

View PR surface stats

Area	Files	Added	Net
Source	1	10	+10
Tests	1	56	+56
Docs	0	0	0
Config	0	0	0
Generated	0	0	0
Other	0	0	0
Total	2	66	+66

What I checked:

Repository policy applied: Root AGENTS.md was read fully and its core-runtime guidance treats fail-closed runtime stream behavior as compatibility-sensitive; scoped src/agents/AGENTS.md was also read and is relevant to focused agent runtime tests. (AGENTS.md:21, ec737ee74d9b)
Current main parser behavior: Current streamProxy appends decoded chunks to buffer, then splits on newline; without a newline boundary, the tail remains unbounded in current main. (src/agents/runtime/proxy.ts:226, ec737ee74d9b)
Latest release behavior: The latest release tag still has the same unbounded proxy SSE tail-buffer behavior, so this is not already shipped fixed behavior. (src/agents/runtime/proxy.ts:226, aa69b12d0086)
PR guard placement: The PR head now splits complete newline-delimited lines first, stores only the leftover tail in buffer, and enforces the cap on that tail before processing complete lines. (src/agents/runtime/proxy.ts:231, 501c07a4a6f9)
Regression coverage: The PR adds one oversized no-newline proxy SSE test and one coalesced valid newline-delimited chunk test, directly covering the prior review blocker. (src/agents/runtime/proxy.test.ts:193, 501c07a4a6f9)
Sibling SSE precedent: The Signal SSE client drains complete lines and then recomputes buffered byte length, which supports reviewing this as a tail-buffer boundary rather than a transport-chunk boundary. (extensions/signal/src/client.ts:397, ec737ee74d9b)

Likely related people:

zhangguiping-xydt: Current-main blame and git log -S point the streamProxy file and buffer loop to commit f12ade0082ad; GitHub associates that commit history with this handle. (role: current source provenance signal; confidence: medium; commits: f12ade0082ad; files: src/agents/runtime/proxy.ts, src/agents/runtime/proxy.test.ts)
obviyus: git show lists Ayaan Zaidi as committer for the current-source provenance commit, making this a routing signal for recent source movement rather than fault assignment. (role: committer for current source snapshot; confidence: medium; commits: f12ade0082ad; files: src/agents/runtime/proxy.ts, src/agents/runtime/proxy.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

wangmiao0668000666 · 2026-06-26T09:03:56Z

@clawsweeper re-review

P1 finding addressed:

Moved SSE cap after line draining (split by \n first, then cap only the unterminated tail).
Added coalesced-valid-chunk regression test: 2000 proxy SSE lines in one chunk (no cap false-positive).

Commit: 501c07a

clawsweeper · 2026-06-26T09:03:58Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

vincentkoc · 2026-06-26T23:13:46Z

closing this as not mergeable in its current shape. the 64 KiB guard is not a valid proxy protocol limit: ProxyAssistantMessageEvent has no event-size contract, and a valid 70 KiB text_delta line fails or succeeds depending on ReadableStream chunking. the implementation also measures JavaScript string length while calling the value bytes, so multibyte payloads are inconsistent. sibling SSE handling uses an explicit byte-accurate 1 MiB bound; this PR has no equivalent contract or complete-event proof. please reopen a narrow replacement only after the proxy framing/size contract is established.

Extend the byte-accurate 1 MiB policy to the unterminated tail (data after last \n) and the EOF final frame, closing the bypass paths identified by ClawSweeper review. The 1 MiB tail cap is deterministic unlike the prior 64 KiB attempt (openclaw#96993) because any hostile payload exceeding 1 MiB total bytes triggers the cap regardless of TCP chunking. - Add tail buffer cap after draining complete lines (proxy.ts:237) - Add EOF final frame cap before processSseLine (proxy.ts:246) - 2 new tests: unterminated tail across chunks, no-newline EOF frame Co-Authored-By: Claude <[email protected]>

fix(proxy): bound SSE parser buffer to prevent OOM

60149e4

openclaw-barnacle Bot added agents Agent runtime and tooling size: XS labels Jun 26, 2026

fix(proxy): move SSE cap after line draining + coalesced-chunk test

501c07a

openclaw-barnacle Bot added size: S and removed size: XS labels Jun 26, 2026

vincentkoc self-assigned this Jun 26, 2026

vincentkoc closed this Jun 26, 2026

wangmiao0668000666 mentioned this pull request Jun 27, 2026

fix(proxy): bound SSE parser via complete-line cap at 1 MiB #97191

Closed

clawsweeper Bot mentioned this pull request Jun 27, 2026

fix(runtime/proxy): bound streaming success-body SSE reads at 16 MiB #96768

Closed

This was referenced Jun 29, 2026

fix(discord): bound gateway metadata response reads at 16 MiB #97721

Closed

fix(tlon): bound auth response body at 16 MiB #97730

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(proxy): bound SSE parser buffer to prevent OOM#96993

fix(proxy): bound SSE parser buffer to prevent OOM#96993
wangmiao0668000666 wants to merge 2 commits into
openclaw:mainfrom
wangmiao0668000666:fix/bound-proxy-parser-buffer

wangmiao0668000666 commented Jun 26, 2026

Uh oh!

clawsweeper Bot commented Jun 26, 2026 •

edited

Loading

Uh oh!

wangmiao0668000666 commented Jun 26, 2026

Uh oh!

clawsweeper Bot commented Jun 26, 2026 •

edited

Loading

Uh oh!

vincentkoc commented Jun 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

wangmiao0668000666 commented Jun 26, 2026

What Problem This Solves

Changes

Real behavior proof

Out of scope

Risk

Uh oh!

clawsweeper Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wangmiao0668000666 commented Jun 26, 2026

Uh oh!

clawsweeper Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincentkoc commented Jun 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clawsweeper Bot commented Jun 26, 2026 •

edited

Loading

clawsweeper Bot commented Jun 26, 2026 •

edited

Loading