Skip to content

fix(matrix): bound non-raw JSON response body in transport#95240

Merged
steipete merged 3 commits into
openclaw:mainfrom
Alix-007:fix/bound-matrix-transport-json
Jun 23, 2026
Merged

fix(matrix): bound non-raw JSON response body in transport#95240
steipete merged 3 commits into
openclaw:mainfrom
Alix-007:fix/bound-matrix-transport-json

Conversation

@Alix-007

@Alix-007 Alix-007 commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Summary

performMatrixRequest already bounds the raw media path (transport.ts): it prechecks content-length and streams through readResponseWithLimit(response, params.maxBytes, …) so a malicious/huge media body is cancelled at the cap. The non-raw JSON path, however, fell through to a plain const text = await response.text();, which buffers the entire body with no ceiling and no idle timeout.

That JSON path serves control-plane calls against the homeserver — whoami, read receipts, room key-backup status, directory search, and the generic doRequest — and the Matrix homeserver is an untrusted source. A hostile or buggy homeserver can stream an unbounded JSON body and exhaust memory before JSON.parse is ever reached.

This is the symmetric counterpart to #95108 (which hardened the Anthropic error-stream read with a byte/idle cap): same class of unbounded response.text() on an untrusted upstream, fixed by reusing the existing bounded reader.

Changes

  • extensions/matrix/src/matrix/sdk/transport.ts: bound the non-raw JSON read using the same in-file readResponseWithLimit helper and parseMediaContentLength precheck already used by the raw media path — no new abstraction.
    • Added MATRIX_JSON_RESPONSE_MAX_BYTES = 8 * 1024 * 1024 as the default ceiling for control-plane JSON responses (generous for whoami/receipts/directory/key-backup, far below an OOM).
    • Honors an explicit params.maxBytes when a caller provides one (falls back to the default), and threads params.readIdleTimeoutMs as chunkTimeoutMs so a stalled stream is cancelled, mirroring the raw path exactly.
    • Overflow / content-length excess now throws a clear Matrix JSON response exceeds configured size limit (…) error instead of using the media-specific MatrixMediaSizeLimitError (semantically a JSON response, not media).
    • The JSON read now passes a JSON-specific onIdleTimeout so a stalled stream rejects with Matrix JSON response stalled: no data received for {ms}ms rather than the raw path's Matrix media download stalled: … default — the timeout diagnostic now distinguishes a stalled JSON read from a stalled raw/media read.
  • extensions/matrix/src/matrix/sdk/transport.test.ts: added regression coverage for the JSON path — content-length precheck rejection, streamed-overflow rejection without content-length, configured idle-timeout on a stalled stream (asserting the JSON-specific stalled message), and an under-cap body returned intact.

Real behavior proof

Label: bounded non-raw JSON transport read

  • Behavior addressed: performMatrixRequest non-raw JSON path buffered an unbounded response body via await response.text(); an untrusted homeserver could OOM the process. Now the body is read under a byte cap with stream cancellation and an optional idle timeout.
  • Real environment tested: Node v22.22.0 on Linux, driving the actual performMatrixRequest JSON path (real undici fetch + SSRF-pinned dispatcher) against a real node:http server on 127.0.0.1.
  • Exact steps or command run after this patch:
    • node --import tsx proof-bound-json.mts — a tsx harness importing performMatrixRequest from source and hitting a live localhost HTTP server.
    • Case 1: server streams 4 KiB JSON chunks forever (no content-length), client cap maxBytes: 8 KiB.
    • Case 2: small JSON body under the cap.
    • Case 3: server sends a 64 KiB body advertising content-length: 65536, client cap maxBytes: 1 KiB.
    • node scripts/run-vitest.mjs extensions/matrix/src/matrix/sdk/transport.test.ts --run (11/11 pass, incl. 4 new).
    • node scripts/run-vitest.mjs extensions/matrix/src/matrix/sdk/http-client.test.ts --run (caller, 4/4 pass).
  • Evidence after fix:
    • Case 1: request rejected with Matrix JSON response exceeds configured size limit (12288 bytes > 8192 bytes); the server observed the connection close (stream cancelled, socket released) after ~16 KiB instead of streaming forever.
    • Case 2: full JSON body returned intact (result.text and result.buffer match the payload).
    • Case 3: rejected on the content-length precheck with Matrix JSON response exceeds configured size limit (65536 bytes > 1024 bytes) — body not buffered.
  • Observed result after fix: ALL PROOF CASES PASSED. Negative control: temporarily reverting the source fix to plain await response.text() and re-running the infinite-stream case showed the bug — within a 1.5 s window the unbounded path did NOT reject; the server pushed ~5 MB and the read kept buffering (would continue to OOM/timeout). Restoring the fix returned to ALL PROOF CASES PASSED.
  • What was not tested: the matrix-js-sdk /sync long-poll path is unaffected (it goes through createMatrixGuardedFetcharrayBuffer(), a separate buffering site not in scope here); no live remote homeserver was exercised (localhost server used as the untrusted upstream).

Reviewer notes (for clawsweeper findings)

  • 8 MiB default cap (MATRIX_JSON_RESPONSE_MAX_BYTES) — maintainer decision requested: This is a fail-closed default, not a value I want to set unilaterally. It is sized to sit far above any legitimate Matrix control-plane JSON response (whoami, read receipts, key-backup status, directory search, generic doRequest — all kilobyte-scale), while staying far below a memory-exhaustion threshold, and it lines up with the order of magnitude used by the other bound-stream caps in this repo. Callers can already override it per-request via params.maxBytes; the constant is only the floor for callers that pass nothing. If maintainers prefer a different ceiling (or want it sourced from config), it is a one-line change to the constant — please accept or tune as you see fit.
  • Sibling createMatrixGuardedFetch (matrix-js-sdk /sync path) — explicitly out of scope here: It buffers via arrayBuffer() and is a separate, distinct buffering site. As clawsweeper noted, that should be tracked separately; this PR intentionally does not touch it to keep the change focused on the performMatrixRequest JSON path. Happy to open a follow-up for the guarded-fetch path once this lands.

This change is AI-assisted.

@openclaw-barnacle openclaw-barnacle Bot added channel: matrix Channel integration: matrix size: S labels Jun 20, 2026
@clawsweeper

clawsweeper Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 23, 2026, 10:28 AM ET / 14:28 UTC.

Summary
This PR adds a default 8 MiB bounded reader for non-raw Matrix JSON responses in performMatrixRequest, plus regression tests for content-length rejection, streamed overflow, idle timeout, and under-cap success.

PR surface: Source +24, Tests +136. Total +160 across 2 files.

Reproducibility: yes. by source inspection and contributor proof: current main and v2026.6.9 still use await response.text() for non-raw Matrix responses, and the PR body reports a live localhost negative control where the old path kept buffering. I did not rerun the harness because this review is read-only.

Review metrics: 1 noteworthy metric.

  • New default response limit: 1 added: 8 MiB Matrix JSON cap. The new default changes upgrade behavior for oversized Matrix control-plane JSON responses, so maintainers should notice it before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] Have maintainers explicitly accept or tune the 8 MiB Matrix JSON response cap before merge.

Risk before merge

  • [P1] The new default can make Matrix control-plane JSON responses above 8 MiB fail where they previously buffered successfully, so maintainers should explicitly accept or tune that fail-closed behavior.
  • [P1] This PR does not harden the separate matrix-js-sdk createMatrixGuardedFetch path, which still buffers with arrayBuffer()/full-body JSON handling and should be tracked separately if broader Matrix response hardening is desired.

Maintainer options:

  1. Accept the 8 MiB fail-closed cap
    Maintainers can merge as-is if Matrix control-plane responses above 8 MiB should fail closed to avoid unbounded memory use.
  2. Tune the cap before merge
    Raise, lower, or endpoint-scope the default if maintainers know legitimate Matrix homeservers can return larger control-plane JSON payloads.
  3. Keep sibling buffering separate
    Treat this PR as the narrow performMatrixRequest fix and open a follow-up for createMatrixGuardedFetch if full Matrix transport hardening is required.

Next step before merge

  • [P1] Keep this in human maintainer review because the remaining action is acceptance or tuning of the new fail-closed Matrix JSON response cap, not a narrow code repair.

Security
Cleared: Security review cleared: the diff narrows an untrusted Matrix homeserver memory-DoS surface and adds no dependency, workflow, secret, or code-execution changes.

Review details

Best possible solution:

Land this focused performMatrixRequest JSON hardening after maintainers accept or tune the 8 MiB cap, and track createMatrixGuardedFetch as separate Matrix transport hardening if needed.

Do we have a high-confidence way to reproduce the issue?

Yes by source inspection and contributor proof: current main and v2026.6.9 still use await response.text() for non-raw Matrix responses, and the PR body reports a live localhost negative control where the old path kept buffering. I did not rerun the harness because this review is read-only.

Is this the best way to solve the issue?

Yes for the narrow performMatrixRequest bug: reusing the existing bounded reader and content-length parser is the owner-local fix. The exact 8 MiB default remains a maintainer compatibility decision, and matrix-js-sdk response buffering is separate follow-up work if broader hardening is required.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 8e6624cb6cb8.

Label changes

Label justifications:

  • P2: This is a normal-priority Matrix transport hardening with limited channel-specific blast radius and a clear maintainer review path.
  • merge-risk: 🚨 compatibility: The PR changes existing Matrix JSON transport behavior from unbounded buffering to failing responses above the new 8 MiB default cap.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes after-fix terminal proof against the actual Matrix request path with a live localhost HTTP server plus a negative control for the old unbounded read.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal proof against the actual Matrix request path with a live localhost HTTP server plus a negative control for the old unbounded read.
Evidence reviewed

PR surface:

Source +24, Tests +136. Total +160 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 27 3 +24
Tests 1 136 0 +136
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 163 3 +160

What I checked:

  • Current main behavior: Current main bounds raw Matrix media responses but still reads non-raw responses with await response.text(), so the central bug is not already fixed on main. (extensions/matrix/src/matrix/sdk/transport.ts:320, 8e6624cb6cb8)
  • PR implementation: The PR head adds MATRIX_JSON_RESPONSE_MAX_BYTES, content-length precheck rejection, readResponseWithLimit, and a JSON-specific idle-timeout error while preserving the returned text and buffer shape. (extensions/matrix/src/matrix/sdk/transport.ts:17, 6277cb34ddd8)
  • Regression coverage: The PR adds tests for JSON content-length overflow, streamed overflow without content-length, JSON idle timeout, and successful under-limit JSON reads. (extensions/matrix/src/matrix/sdk/transport.test.ts:199, 6277cb34ddd8)
  • Caller reachability: MatrixAuthedHttpClient.requestJson calls performMatrixRequest, and Matrix client flows such as doRequest, read receipts, whoami, and room key backup reach that JSON path. (extensions/matrix/src/matrix/sdk/http-client.ts:35, 8e6624cb6cb8)
  • Shared bounded-reader contract: The shared reader accumulates only until the byte cap, cancels on overflow, and supports custom idle-timeout errors, matching the PR's intended JSON behavior. (packages/media-core/src/read-response-with-limit.ts:129, 8e6624cb6cb8)
  • Sibling Matrix path: createMatrixGuardedFetch still buffers with response.arrayBuffer(), and matrix-js-sdk 41.6.0 also uses full-body res.json()/res.text() for its fetch API, so this PR should be treated as focused performMatrixRequest hardening rather than complete Matrix transport hardening. (extensions/matrix/src/matrix/sdk/transport.ts:237)

Likely related people:

  • gumadeiras: Relevant history shows Gustavo Madeira Santana with the largest shortlog share across the Matrix transport/client files and several Matrix runtime fixes around sync, E2EE, and private-network homeserver handling. (role: Matrix runtime feature contributor; confidence: high; commits: f62be0ddcf7c, 0c00c3c2306f, 94081d886336; files: extensions/matrix/src/matrix/sdk/transport.ts, extensions/matrix/src/matrix/sdk.ts)
  • steipete: Peter Steinberger appears in recent Matrix test-boundary and plugin-SDK/media boundary refactors that shape the shared bounded-reader and extension-boundary context used here. (role: recent shared-boundary and Matrix test contributor; confidence: medium; commits: 605cb60586eb, 418056f7a0c8, 62ddc9d9e0d9; files: extensions/matrix/src/matrix/sdk.ts, packages/media-core/src/read-response-with-limit.ts, src/plugin-sdk/response-limit-runtime.ts)
  • vincentkoc: Local blame on the shallow current checkout points the Matrix transport lines at a recent Vincent Koc commit, and the linked merged Anthropic PR by vincentkoc fixed the same unbounded upstream-response class in another transport. (role: recent area and adjacent stream-boundary contributor; confidence: medium; commits: 306f0ec37f5a, d6cefe26f499, c645ec4555c0; files: extensions/matrix/src/matrix/sdk/transport.ts, src/agents/anthropic-transport-stream.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels Jun 20, 2026
@Alix-007

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review — Addressed the P3 finding: the JSON read path now passes a JSON-specific onIdleTimeout ("Matrix JSON response stalled: …") instead of falling back to the media default message; the raw path is unchanged. Added reviewer notes in the body covering the two P1s: the 8 MiB cap rationale (left for maintainer accept/tune — unchanged here, overridable via params.maxBytes) and that createMatrixGuardedFetch buffering is tracked as a separate follow-up, out of scope here. Tests: transport 11/11, http-client 4/4, oxlint + tsgo clean.

@clawsweeper

clawsweeper Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Alix-007 added 2 commits June 23, 2026 15:40
…N read

The non-raw JSON read in performMatrixRequest fell back to the bound
reader's default media idle-timeout message ('Matrix media download
stalled: ...'), which is misleading for a JSON control-plane read. Pass
a JSON-specific onIdleTimeout so a stalled JSON stream now rejects with
'Matrix JSON response stalled: no data received for {ms}ms', letting the
timeout diagnostic distinguish a stalled JSON read from a stalled
raw/media read. Update the regression assertion accordingly.
@Alix-007
Alix-007 force-pushed the fix/bound-matrix-transport-json branch from 8b6d491 to 6277cb3 Compare June 23, 2026 07:45
@Alix-007

Copy link
Copy Markdown
Contributor Author

Maintenance update: rebased this branch linearly onto the latest upstream/main. The previous prod-types/lint failures pointed at src/config/sessions/session-accessor.ts, which is outside this PR's Matrix diff and is already updated on current main. The PR diff remains limited to Matrix transport/test files, with author/committer email set to [email protected].

@steipete steipete self-assigned this Jun 23, 2026
@steipete
steipete requested a review from a team as a code owner June 23, 2026 20:12
@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation channel: discord Channel integration: discord channel: googlechat Channel integration: googlechat channel: imessage Channel integration: imessage channel: line Channel integration: line channel: mattermost Channel integration: mattermost channel: nextcloud-talk Channel integration: nextcloud-talk channel: signal Channel integration: signal channel: telegram Channel integration: telegram channel: whatsapp-web Channel integration: whatsapp-web channel: zalo Channel integration: zalo channel: zalouser Channel integration: zalouser app: android App: android labels Jun 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Dependency Guard

This PR changes dependency-related files. Maintainers should confirm these changes are intentional.

Changed files:

  • extensions/acpx/npm-shrinkwrap.json
  • extensions/acpx/package.json
  • extensions/qa-lab/package.json
  • pnpm-lock.yaml
  • pnpm-workspace.yaml

Maintainer follow-up:

  • Review whether the dependency changes are intentional.
  • Inspect resolved package deltas when lockfile, shrinkwrap, or workspace dependency policy changes are present.
  • Treat package-lock.json and npm-shrinkwrap.json diffs as security-review surfaces.
  • Run pnpm deps:changes:report -- --base-ref origin/main --markdown /tmp/dependency-changes.md --json /tmp/dependency-changes.json locally for detailed release-style evidence.

@github-actions

Copy link
Copy Markdown
Contributor

Dependency graph changes are blocked

OpenClaw does not accept dependency graph changes through PRs unless a repository admin or security explicitly authorizes the current head SHA. Dependency updates are generated internally by maintainers so external PRs cannot change the resolved graph.

Detected dependency graph changes:

  • extensions/acpx/npm-shrinkwrap.json changed.
  • pnpm-lock.yaml changed.
  • extensions/acpx/package.json changed dependencies.
  • extensions/qa-lab/package.json changed devDependencies.

Auto-scrub was not attempted because this PR changes package manifest dependency graph fields:

  • extensions/acpx/package.json changed dependencies.
  • extensions/qa-lab/package.json changed devDependencies.

Dependency graph changes must be reviewed by security or handled by maintainers internally. Please remove lockfile changes manually if they are not needed.

To remove lockfile changes, restore them from the target branch:

git fetch origin
git checkout 'origin/main' -- 'extensions/acpx/npm-shrinkwrap.json' 'pnpm-lock.yaml'
git commit -m 'chore: remove dependency lockfile change'
git push

If this PR intentionally needs a dependency graph change, ask a repository admin or member of @openclaw/openclaw-secops to comment:

/allow-dependencies-change

The action will approve the current head SHA (6ea80c43ec75024cba24367726169457be6b7099) when it reruns. A later push requires a fresh approval.

@steipete

Copy link
Copy Markdown
Contributor

Maintainer verification complete on exact head 6ea80c43ec75024cba24367726169457be6b7099.

  • Reviewed direct Matrix raw/JSON requests, the matrix-js-sdk injected fetch path, redirects, timeout ownership, and media download routing.
  • Direct dependency inspection used matrix-js-sdk 41.6.0: lib/http-api/fetch.js consumes injected responses through text(), json(), or blob(); lib/rust-crypto/rust-crypto.js also has a raw encrypted key-bundle path. That contract is why the final stack uses an 8 MiB direct JSON cap and a compatibility-sized 64 MiB SDK envelope cap.
  • node scripts/run-vitest.mjs extensions/matrix/src/matrix/sdk/transport.test.ts passed 12 tests, including declared-size cancellation and rebuilt SDK response consumption.
  • Blacksmith Testbox tbx_01kvv1h4864q3w2tzg0wafw67w / workflow run 28053690579 passed remote pnpm check:changed.
  • Fresh Codex autoreview completed with no accepted/actionable findings (0.94 confidence).

One unrelated agent session-lock timing test failed in hosted CI while 1,530 sibling tests passed. node scripts/run-vitest.mjs src/agents/embedded-agent-runner/run/attempt.session-lock.test.ts then passed all 105 tests locally, and the exact failed hosted job was rerun at the same SHA. No Matrix proof gaps.

@steipete

Copy link
Copy Markdown
Contributor

Merged via rebase.

@Alix-007

Copy link
Copy Markdown
Contributor Author

Thanks @steipete for the detailed Matrix verification and rebase merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling app: android App: android app: ios App: ios app: web-ui App: web-ui channel: discord Channel integration: discord channel: googlechat Channel integration: googlechat channel: imessage Channel integration: imessage channel: line Channel integration: line channel: matrix Channel integration: matrix channel: mattermost Channel integration: mattermost channel: nextcloud-talk Channel integration: nextcloud-talk channel: qqbot channel: signal Channel integration: signal channel: synology-chat channel: telegram Channel integration: telegram channel: twitch Channel integration: twitch channel: whatsapp-web Channel integration: whatsapp-web channel: zalo Channel integration: zalo channel: zalouser Channel integration: zalouser cli CLI command changes commands Command implementations dependencies-changed PR changes dependency-related files docker Docker and sandbox tooling docs Improvements or additions to documentation extensions: acpx extensions: anthropic extensions: anthropic-vertex extensions: brave extensions: chutes extensions: cloudflare-ai-gateway extensions: codex extensions: copilot extensions: copilot-proxy Extension: copilot-proxy extensions: deepinfra extensions: deepseek extensions: duckduckgo extensions: elevenlabs extensions: github-copilot extensions: google extensions: huggingface extensions: kilocode extensions: lmstudio extensions: memory-core Extension: memory-core extensions: memory-wiki extensions: minimax extensions: mistral extensions: nvidia extensions: ollama extensions: opencode extensions: opencode-go extensions: openrouter extensions: perplexity extensions: qa-lab extensions: qianfan extensions: vercel-ai-gateway extensions: xai extensions: xiaomi gateway Gateway runtime merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. plugin: google-meet proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. scripts Repository scripts size: XL status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants