Skip to content

fix(extensions): bound JSON response reads to prevent OOM#97558

Closed
NarahariRaghava wants to merge 1 commit into
openclaw:mainfrom
NarahariRaghava:fix/bound-unbounded-json-response-reads
Closed

fix(extensions): bound JSON response reads to prevent OOM#97558
NarahariRaghava wants to merge 1 commit into
openclaw:mainfrom
NarahariRaghava:fix/bound-unbounded-json-response-reads

Conversation

@NarahariRaghava

@NarahariRaghava NarahariRaghava commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Fixes an issue where a malformed or unexpectedly large API response from external services (Feishu, Discord, Google Meet, Tlon) could cause the gateway process to buffer an arbitrarily large payload into memory, leading to an out-of-memory crash.

Why This Change Was Made

Recent work bounded response reads across most extensions to prevent OOM from oversized API responses. Several extensions still used raw response.json() without size limits. This change replaces those calls with the shared readProviderJsonResponse helper, which caps reads at a safe limit. Existing error handling patterns (try/catch, .catch() fallbacks) are preserved.

User Impact

Gateway operators running Feishu, Discord, Google Meet, or Tlon channel plugins are protected from OOM crashes caused by unexpectedly large external API responses.

Evidence

All 1136 tests pass across 84 test files. Build succeeds. Terminal proof shows the bounded reader rejecting a 20MB response with "Discord PluralKit response: JSON response exceeds 16777216 bytes" while normal responses parse correctly.

@openclaw-barnacle openclaw-barnacle Bot added channel: discord Channel integration: discord channel: msteams Channel integration: msteams channel: signal Channel integration: signal channel: telegram Channel integration: telegram channel: tlon Channel integration: tlon channel: feishu Channel integration: feishu plugin: google-meet size: S labels Jun 28, 2026
@clawsweeper

clawsweeper Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Thanks for the context here. I did a careful shell check against current main, and this is already implemented.

Close: current main now has same-or-better bounded JSON response handling across the touched extension paths, and the proof commit is current-main-only rather than shipped in v2026.6.11.

So I’m closing this as already implemented rather than keeping a duplicate issue open.

Review details

Best possible solution:

Keep current main as the canonical implementation and use future narrow per-plugin PRs only if a new unbounded response path is found against current main.

Do we have a high-confidence way to reproduce the issue?

Yes. The PR diff shows the original source-level unbounded response.json() reads, and current main shows bounded replacements plus focused oversized-response tests; this read-only review did not run live external API repros.

Is this the best way to solve the issue?

No, this branch is no longer the best merge path. Current main has same-or-better bounded-reader coverage, stronger focused tests, and the branch is now conflicting with already-landed work.

Security review:

Security review cleared: The diff narrows external response-body handling and adds no new dependencies, downloaded code, secret handling, package-resolution, or workflow execution surface.

AGENTS.md: found and applied where relevant.

What I checked:

Likely related people:

  • vincentkoc: Merged refactor(deadcode): localize release tooling declarations #101904, whose commit is the current-main blame source for the bounded-reader lines across the touched files in this shallow checkout. (role: current-main proof author and merger; confidence: high; commits: 3dbe98a1f737; files: extensions/discord/src/pluralkit.ts, extensions/discord/src/send.webhook.ts, extensions/discord/src/voice-message.ts)
  • hugenshen: Authored the merged focused Tlon scry bounded JSON response fix that overlaps the Tlon slice of this branch. (role: recent adjacent fix author; confidence: medium; commits: 19f5a4a0bb03; files: extensions/tlon/src/urbit/channel-ops.ts, extensions/tlon/src/urbit/error-body-boundary.test.ts)
  • Alix-007: Authored the merged Discord response-bounds sibling PR for requestDiscord, which is part of the same broader response-body hardening campaign. (role: recent adjacent contributor; confidence: medium; commits: fca15641dba5; files: extensions/discord/src/api.ts, extensions/discord/src/api.test.ts)

Codex review notes: model internal, reasoning high; reviewed against 4ae8d735bf54; fix evidence: commit 3dbe98a1f737, main fix timestamp 2026-07-07T23:13:55Z.

@NarahariRaghava
NarahariRaghava force-pushed the fix/bound-unbounded-json-response-reads branch from d85e77d to 66e9df3 Compare June 28, 2026 20:01
@openclaw-barnacle openclaw-barnacle Bot removed channel: signal Channel integration: signal channel: telegram Channel integration: telegram labels Jun 28, 2026
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 28, 2026
@harjothkhara

Copy link
Copy Markdown
Contributor

Current CI triage for this head (66e9df36e13d8194030f8d2b8c505bc3ed49a231): check-lint is red, and one failure is PR-local.

  • extensions/feishu/src/streaming-card.test.ts:68:13 reports no-unused-vars for ok. That looks introduced by the test mock conversion to real Response objects: Response.ok is now derived from status, so the local ok variable and ok = false assignments can be removed while keeping the existing status handling.
  • The same lint job also reports extensions/line/src/message-cards.test.ts:333:9 (loneHighSurrogate shadowing), but that file is not in this PR's changed-file list, so I would treat it as unrelated current-main lint noise unless a rebase changes the diff.

Other relevant early checks I saw were green, including Real behavior proof, check-prod-types, check-test-types, check-additional-extension-channels, and check-dependencies; several matrix jobs were still running when I checked.

@clawsweeper clawsweeper Bot added P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. labels Jun 28, 2026
@NarahariRaghava
NarahariRaghava force-pushed the fix/bound-unbounded-json-response-reads branch from 66e9df3 to 26788c8 Compare June 28, 2026 20:33
@clawsweeper clawsweeper Bot added the merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. label Jun 28, 2026
@NarahariRaghava
NarahariRaghava force-pushed the fix/bound-unbounded-json-response-reads branch 2 times, most recently from 98eb6ce to 8f29e22 Compare June 28, 2026 21:06
@NarahariRaghava

Copy link
Copy Markdown
Contributor Author

Addressed both findings:

  1. Feishu guard cleanup - Both token and card-create paths now wrap readProviderJsonResponse in try/finally so release() is always called, even on oversized or malformed responses.

  2. Overflow proof - Added a focused test that sends a 20MB response and verifies:

    • The bounded reader rejects with a cap-exceeded error
    • The SSRF guard release() is still called via try/finally

@clawsweeper clawsweeper Bot removed the merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. label Jun 28, 2026
@NarahariRaghava
NarahariRaghava force-pushed the fix/bound-unbounded-json-response-reads branch from 8f29e22 to 57d5968 Compare June 28, 2026 21:46
@NarahariRaghava

Copy link
Copy Markdown
Contributor Author

Addressed all findings:

  1. PluralKit fixture — Updated success test to return a real Response object.
  2. Feishu guard cleanup — Both paths use try/finally for release().
  3. Real behavior proof — Terminal output below shows the bounded reader rejecting a 20MB response with "JSON response exceeds 16777216 bytes" while normal responses parse correctly.

@clawsweeper re-review
Screenshot 2026-06-28 at 4 47 07 PM

@clawsweeper

clawsweeper Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot added proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Jun 28, 2026
@clawsweeper clawsweeper Bot added the rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. label Jun 28, 2026
@NarahariRaghava
NarahariRaghava force-pushed the fix/bound-unbounded-json-response-reads branch from 57d5968 to 245504c Compare June 28, 2026 23:55
@NarahariRaghava

Copy link
Copy Markdown
Contributor Author

Addressed remaining findings:

  1. MS Teams sibling reads — Bounded response.json() in attachments/graph.ts (2 call sites).
  2. Tlon scry reader — Bounded response.json() in urbit/channel-ops.ts.
  3. PluralKit fixture — Updated to real Response object.

Changed-path proof — Terminal output shows readProviderJsonResponse with the "Discord PluralKit response" label (the exact label used in the changed plugin path):

  • 20MB response: REJECTED: Discord PluralKit response: JSON response exceeds 16777216 bytes
  • Normal response: PARSED: {"id":"456","member":{"id":"m2","name":"Alex"}}

@clawsweeper re-review
Screenshot 2026-06-28 at 6 58 19 PM

@clawsweeper

clawsweeper Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Jun 29, 2026
@NarahariRaghava
NarahariRaghava force-pushed the fix/bound-unbounded-json-response-reads branch from 245504c to 3f52a5a Compare June 29, 2026 00:05
@clawsweeper clawsweeper Bot added the merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. label Jun 29, 2026
@NarahariRaghava
NarahariRaghava force-pushed the fix/bound-unbounded-json-response-reads branch from 3f52a5a to bda6c3d Compare June 29, 2026 00:16
@openclaw-barnacle openclaw-barnacle Bot added the scripts Repository scripts label Jun 29, 2026
Replace unbounded response.json() calls with readProviderJsonResponse
in Feishu, Discord, Google Meet, and Tlon.
@NarahariRaghava
NarahariRaghava force-pushed the fix/bound-unbounded-json-response-reads branch from bda6c3d to 366573a Compare June 29, 2026 00:20
@openclaw-barnacle openclaw-barnacle Bot removed the channel: msteams Channel integration: msteams label Jun 29, 2026
@NarahariRaghava

Copy link
Copy Markdown
Contributor Author

Narrowed PR scope: removed MS Teams to keep the PR clean and focused. MS Teams has additional sibling paths (Bot Framework attachment reader, Graph error visibility) that deserve their own dedicated PR.

This PR now covers: Feishu, Discord, Google Meet, and Tlon only.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@steipete

steipete commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

The focused Tlon scry slice has landed via #100376 after bounded loopback proof, sanitized AWS focused tests, exact-head CI, and fresh autoreview.

Please remove these overlapping files from this broader PR before further review:

  • extensions/tlon/src/urbit/channel-ops.ts
  • extensions/tlon/src/urbit/channel-ops.test.ts

The Discord, Feishu, Google Meet, Tlon API, and SDK-surface work remains distinct and should stay independently reviewable. Thanks for the broader response-bounds work.

@clawsweeper

clawsweeper Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper applied the proposed close for this PR.

@clawsweeper clawsweeper Bot closed this Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: discord Channel integration: discord channel: feishu Channel integration: feishu channel: tlon Channel integration: tlon merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. P2 Normal backlog priority with limited blast radius. plugin: google-meet proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. scripts Repository scripts size: M status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants