fix(byteplus): bound video-generation success response reads by Alix-007 · Pull Request #96606 · openclaw/openclaw

Alix-007 · 2026-06-25T00:33:55Z

What Problem This Solves

The BytePlus video provider read two untrusted success responses with an unbounded await response.json(), exposing the create + poll loop to OOM / hang.

readBytePlusJsonResponse in extensions/byteplus/video-generation-provider.ts parses both the submit task (/contents/generations/tasks) and poll status (/contents/generations/tasks/{id}) success bodies. The BytePlus baseUrl is user-supplied and the endpoint is untrusted external content, so a hostile or misconfigured host can stream an arbitrarily large (or Content-Length-less, never-ending) JSON body on either path. response.json() buffers the whole payload before parsing, driving the video-generation create + poll loop into memory pressure / OOM or a hang. The sibling video buffer download in the same file was already bounded with readResponseWithLimit; this closes the two remaining unbounded JSON reads.

Changes

Route both success reads through the shared byte-bounded reader readProviderJsonResponse (16 MiB cap = PROVIDER_JSON_RESPONSE_MAX_BYTES, re-exported via openclaw/plugin-sdk/provider-http) instead of await response.json() — it streams the body, cancels the underlying stream on overflow, and JSON.parses the decoded prefix. No new abstraction: this reuses the readResponseWithLimit-backed reader already merged in fix(agents): bound provider JSON response reads #95218 / fix(ollama): bound model-discovery JSON response reads #96027 / fix(exa): bound untrusted search JSON response reads #96038.
Applies to the submit read (label: "BytePlus video generation failed") and the poll read (label: "BytePlus video status request failed").
Error semantics preserved: malformed JSON still surfaces ${label}: malformed JSON response; overflow throws ${label}: JSON response exceeds 16777216 bytes. Plugin-scoped — only extensions/byteplus changes.

Real behavior proof

Behavior addressed: An untrusted BytePlus submit or poll success response with no
Content-Length and an oversized/never-ending body must not be buffered whole; the
read must stop at the 16 MiB cap, cancel the stream (tear down the socket), and throw a
bounded error — while valid small submit/poll bodies still parse unchanged.
Real environment tested: A real local node:http server (createServer) bound to
127.0.0.1, returning a Content-Length-less body that streams 1 MiB chunks up to a
~32 MiB advertised payload (2× the cap). The real exported readProviderJsonResponse
— the exact byte-bounded reader readBytePlusJsonResponse calls for both the submit
and poll success-JSON paths — was driven against it over a real TCP socket via the
real fetch, using the two real provider labels verbatim. Node v22.22.0.
Exact steps: node --import tsx <paths-loader> proof.mts — start server → drive the
real reader against /oversized with the submit label → repeat with the poll
label → read back the server's observed bytes-on-wire / socket-abort per request → run a
negative control (the OLD unbounded response.json()-style whole-body read of the same
endpoint) → drive the real reader against small valid /small-submit and /small-poll
bodies.
Evidence after fix: For both submit and poll, readProviderJsonResponse threw
... JSON response exceeds 16777216 bytes; the server observed the socket aborted
after ~17–20 MiB (submit bytesSent=19922944, poll bytesSent=17825792, both ≪ the
~32 MiB it would have sent), confirming the stream was cancelled, not drained. The
negative control's old unbounded read pulled the full 33554432 bytes (proving the cap
is load-bearing). The small submit body parsed to { id: "task_real_submit" } and the
small poll body parsed to status: "succeeded" with the video_url intact.
What was not tested: Did not hit a live byteplus/ark endpoint (no key; would not
reproduce a hostile oversized body); the untrusted-body behavior is fully reproduced with
a local streaming server over the same transport. The proof script is not committed.

Evidence

Real node:http terminal proof (real socket, not a vitest in-process fixture):

[proof] real node:http server on http://127.0.0.1:37321, cap=16777216 bytes, oversized-body≈33554432 bytes (no content-length)

PASS  submit: oversized JSON throws bounded error :: threw=true msg="BytePlus video generation failed: JSON response exceeds 16777216 bytes"
PASS  submit: stream cancelled — server saw socket abort :: aborted=true
PASS  submit: bytes on wire stayed under the full advertised body (cancelled, not drained) :: bytesSent=19922944 (< 33554432)
PASS  poll: oversized JSON throws bounded error :: threw=true msg="BytePlus video status request failed: JSON response exceeds 16777216 bytes"
PASS  poll: stream cancelled — server saw socket abort :: aborted=true
PASS  poll: bytes on wire stayed under the full advertised body (cancelled, not drained) :: bytesSent=17825792 (< 33554432)
PASS  negative control: OLD unbounded read buffers the WHOLE body past the cap (cap is load-bearing) :: buffered=33554432 bytes serverSent=33554432 (>= 33554432, > cap 16777216)
PASS  happy path: small submit JSON still parsed :: id="task_real_submit"
PASS  happy path: small poll JSON still parsed :: status="succeeded" video_url="https://example.com/byteplus.mp4"

[proof] 9 PASS / 0 FAIL
[proof] ALL PASS

The in-repo Vitest suite for this provider still passes (it keeps readProviderJsonResponse
real via importActual, so the byte-bounded reader actually streams and cancels under
test rather than being stubbed — two real-ReadableStream overflow tests for submit and
poll assert the bounded error + that the body is not fully pulled):

 Test Files  1 passed (1)
      Tests  13 passed (13)

oxlint (exit 0) and tsgo -p tsconfig.extensions.json (no errors in extensions/byteplus)
are clean on the changed files.

This is the BytePlus video-generation counterpart to the #96027 / #96038 response-limit
campaign, applying the same readResponseWithLimit-backed bounded reader to the two
provider-discovery success-JSON reads.

Label: security

AI-assisted.

clawsweeper · 2026-06-25T00:37:21Z

Codex review: needs maintainer review before merge. Reviewed June 24, 2026, 11:20 PM ET / 03:20 UTC.

Summary
The PR changes BytePlus video-generation submit and poll success JSON parsing to use the shared bounded provider JSON reader and adds regression tests for oversized stream cancellation.

PR surface: Source -2, Tests +160. Total +158 across 2 files.

Reproducibility: yes. Current main clearly calls await response.json() for both BytePlus submit and poll success bodies, and the PR body gives a real TCP node:http proof path showing oversized streams are canceled after the cap.

Review metrics: 1 noteworthy metric.

Bounded BytePlus success JSON reads: 2 changed from unbounded to 16 MiB capped. Both submit task and poll status success bodies now fail closed on oversized JSON, which is the compatibility-sensitive behavior maintainers need to accept before merge.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

[P1] Merging will make BytePlus submit and poll success JSON over the shared 16 MiB cap fail with a bounded error instead of attempting an unbounded parse, so maintainers should accept that upgrade behavior for custom or misconfigured baseUrl setups.

Maintainer options:

Accept the shared BytePlus JSON cap (recommended)
Treat 16 MiB as the intended provider success-JSON safety cap for BytePlus submit and poll responses, matching the shared helper and sibling provider hardening.
Choose a BytePlus-specific cap first
If maintainers know legitimate BytePlus task JSON can exceed 16 MiB, require a provider-local cap or documented exception before landing.
Pause until response-size policy is explicit
Keep the branch open but do not merge until the provider response-size policy is clear enough to avoid surprising upgrades.

Next step before merge

[P1] The only blocker is maintainer acceptance of the compatibility-sensitive 16 MiB fail-closed behavior; there is no narrow automated repair to queue.

Security
Cleared: Cleared: the diff adds no dependency, workflow, secret-handling, or code-execution surface and reduces an untrusted provider response memory/OOM risk.

Review details

Best possible solution:

Land the plugin-local bounded-read fix after maintainer acceptance of the shared 16 MiB BytePlus success-JSON cap and normal required merge checks.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main clearly calls await response.json() for both BytePlus submit and poll success bodies, and the PR body gives a real TCP node:http proof path showing oversized streams are canceled after the cap.

Is this the best way to solve the issue?

Yes. This is the best code shape I found: BytePlus response validation remains local while body reading reuses the SDK-exported bounded JSON helper instead of duplicating provider-local limit logic.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 643410c1f3c0.

Label changes

Label justifications:

P2: This is focused BytePlus provider hardening with limited blast radius and no evidence of an active production outage.
merge-risk: 🚨 compatibility: Existing BytePlus setups that return submit or poll success JSON over the shared 16 MiB cap will now fail instead of attempting an unbounded parse.
rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): Sufficient terminal proof: the PR body includes real node:http TCP output for submit and poll overflow errors, early socket aborts, a negative control for the old unbounded read, and unchanged small-response parsing.
proof: sufficient: Contributor real behavior proof is sufficient. Sufficient terminal proof: the PR body includes real node:http TCP output for submit and poll overflow errors, early socket aborts, a negative control for the old unbounded read, and unchanged small-response parsing.

Evidence reviewed

PR surface:

Source -2, Tests +160. Total +158 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	8	10	-2
Tests	1	207	47	+160
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	215	57	+158

What I checked:

Root policy read and applied: Root AGENTS.md was read fully; its review policy treats provider routing and fail-closed behavior as compatibility-sensitive and requires checking caller, callee, sibling surfaces, tests, current main, and history before verdict. (AGENTS.md:1, 643410c1f3c0)
Extension boundary read and applied: extensions/AGENTS.md was read fully; the PR stays inside the plugin boundary by importing the helper from openclaw/plugin-sdk/provider-http rather than core internals. (extensions/AGENTS.md:1, 643410c1f3c0)
Current main still has unbounded BytePlus JSON reads: On current main, readBytePlusJsonResponse wraps await response.json(), and the helper is used by both the poll status path and submit task path. (extensions/byteplus/video-generation-provider.ts:58, 643410c1f3c0)
PR head uses the shared bounded reader: The PR head changes readBytePlusJsonResponse to accept a Response and call readProviderJsonResponse while preserving BytePlus-specific object-shape validation. (extensions/byteplus/video-generation-provider.ts:59, cbd0aefb9722)
Shared helper contract: readProviderJsonResponse defaults to PROVIDER_JSON_RESPONSE_MAX_BYTES, reads through readResponseWithLimit, throws ': JSON response exceeds bytes' on overflow, and wraps malformed JSON with ': malformed JSON response'. (src/agents/provider-http-errors.ts:312, 643410c1f3c0)
Bounded reader cancels overflow streams: readResponseWithLimit cancels the body reader when the next chunk would exceed maxBytes, then throws the caller-provided overflow error. (packages/media-core/src/read-response-with-limit.ts:96, 643410c1f3c0)

Likely related people:

shakkernerd: git blame on current main attributes the BytePlus video-generation helper and submit/poll call sites to commit 9d82906. (role: recent area contributor; confidence: high; commits: 9d82906f792b; files: extensions/byteplus/video-generation-provider.ts)
steipete: git history shows Peter Steinberger authored earlier BytePlus/video-generation provider support and related media-provider follow-ups in these files. (role: feature owner; confidence: high; commits: 932194b7d58c, cd5b1653f600, a88c6f0fe713; files: extensions/byteplus/video-generation-provider.ts, extensions/byteplus/video-generation-provider.test.ts)
xieyongliang: The merged video_generate providerOptions/input role work changed the BytePlus provider and tests in this area. (role: adjacent feature contributor; confidence: medium; commits: 2c57ec7b5ff1; files: extensions/byteplus/video-generation-provider.ts, extensions/byteplus/video-generation-provider.test.ts)
Alix-007: The author also landed the shared bounded provider JSON reader and sibling Ollama/Exa response-limit PRs that this PR builds on. (role: response-limit campaign contributor; confidence: medium; commits: 2592f8a51a4e, d1c2934d0d17, 605aede38c10; files: src/agents/provider-http-errors.ts, extensions/ollama/src/provider-models.ts, extensions/exa/src/exa-web-search-provider.runtime.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Alix-007 · 2026-06-25T01:04:14Z

@clawsweeper review

clawsweeper · 2026-06-25T01:04:17Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Alix-007 · 2026-06-25T02:38:30Z

@clawsweeper review

sallyom · 2026-06-25T17:46:45Z

Maintainer local review: merge-ready, reviewed as a batch:

fix(qwen): bound video description success response reads #96604 - Qwen video description
fix(google): bound Veo video operation response reads #96605 - Google Veo video operation
fix(byteplus): bound video-generation success response reads #96606 - BytePlus video submit/poll
fix(github-copilot): bound usage response reads #96607 - Copilot usage
fix(voyage): bound embedding-batch status, error, and non-OK responses #96608 - Voyage embedding batch

Current CI red check is unrelated to this PR change

Autoreview is clean, the 16 MiB cap is acceptable for these provider JSON/status/error/usage bodies, and the changes follow the established shared bounded-reader pattern. This is the best narrow fix for these call sites: no new SDK/API/config surface, no upgrade/backward-compatibility concern beyond the intended oversized-body failure mode, and no expected breaking behavior for normal provider responses.

The fal music generation provider in extensions/fal/music-generation-provider.ts parsed its HTTP success response with an unbounded await response.json(). A hostile or buggy fal.ai endpoint — the base URL is user-configurable via models.providers.fal.baseUrl (resolved by resolveFalHttpRequestConfig in http-config.ts, defaulting to https://fal.run) — could stream an arbitrarily large JSON body into memory before parsing, forcing the music generation handler into OOM. Route the read through the shared readProviderJsonResponse (from openclaw/plugin-sdk/provider-http), which enforces the 16 MiB byte cap (PROVIDER_JSON_RESPONSE_MAX_BYTES), cancels the stream on overflow, and wraps malformed JSON with the caller label. The change is minimal — a single .json() call replaced by readProviderJsonResponse<unknown> — and preserves the existing downstream logic unchanged. Update the test suite: replace fake { json: async () => ... } response stubs with proper Response objects carrying JSON body streams via a new jsonBodyResponse() helper, so the real readProviderJsonResponse implementation actually exercises the byte-bounded reader under test. Preserve the real readProviderJsonResponse in the provider-http mock (via importOriginal) so the bounded reader streams and cancels oversized bodies. Add a focused regression test: when the music generation stream exceeds the JSON byte cap (32 MiB body, double the 16 MiB cap), generateMusic rejects with a "fal-music-generation: JSON response exceeds" error and the reader cancels the body mid-flight (ReadableStream.cancel fires, bytesPulled < 32 MiB). Existing parse/HTTP-error cases keep passing. Symmetric counterpart to the openclaw#96027/openclaw#96038/openclaw#96042/openclaw#96606/openclaw#96607 response-limit campaign.

fix(byteplus): bound video-generation success response

cbd0aef

openclaw-barnacle Bot added extensions: byteplus size: M labels Jun 25, 2026

This was referenced Jun 25, 2026

fix(qwen): bound video description success response reads #96604

Merged

fix(google): bound Veo video operation response reads #96605

Merged

sallyom merged commit 7b5ee73 into openclaw:main Jun 25, 2026
167 of 184 checks passed

This was referenced Jun 25, 2026

fix(github-copilot): bound usage response reads #96607

Merged

fix(voyage): bound embedding-batch status, error, and non-OK responses #96608

Merged

github-actions Bot mentioned this pull request Jun 25, 2026

📡 Upstream Digest — 2026-06-25 18:04 UTC curtismercier/openclaw-mods#1151

Open

This was referenced Jun 26, 2026

fix(pixverse): bound video-generation success JSON response reads at 16 MiB #96885

Closed

fix(together, pixverse): bound video response reads #96904

Merged

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 26, 2026

fix(byteplus): bound video-generation success response (openclaw#96606)

12cf6eb

QiuYuang pushed a commit to QiuYuang/openclaw that referenced this pull request Jul 1, 2026

fix(byteplus): bound video-generation success response (openclaw#96606)

b4b0298

hugenshen mentioned this pull request Jul 7, 2026

fix(mattermost): truncate inbound preview on code-point boundary #101630

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(byteplus): bound video-generation success response reads#96606

fix(byteplus): bound video-generation success response reads#96606
sallyom merged 1 commit into
openclaw:mainfrom
Alix-007:fix/bound-byteplus-video

Alix-007 commented Jun 25, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

Uh oh!

Alix-007 commented Jun 25, 2026

Uh oh!

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

Uh oh!

Alix-007 commented Jun 25, 2026

Uh oh!

sallyom commented Jun 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

Alix-007 commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Changes

Real behavior proof

Evidence

Uh oh!

clawsweeper Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Alix-007 commented Jun 25, 2026

Uh oh!

clawsweeper Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Alix-007 commented Jun 25, 2026

Uh oh!

sallyom commented Jun 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Alix-007 commented Jun 25, 2026 •

edited

Loading

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading