fix(video-generation): bound dashscope task response reads#96036
fix(video-generation): bound dashscope task response reads#96036wangmiao0668000666 wants to merge 2 commits into
Conversation
|
@clawsweeper review @clawsweeper — could you kick off a review for PR #96036 (commit Background:
Thanks! |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Codex review: needs maintainer review before merge. Reviewed June 24, 2026, 3:08 AM ET / 07:08 UTC. Summary PR surface: Source +27, Tests +181, Other +208. Total +416 across 4 files. Reproducibility: yes. Current main has raw Review metrics: 1 noteworthy metric.
Stored data model Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Next step before merge
Security Review detailsBest possible solution: Land the focused bounded-reader change after normal maintainer review and keep sibling unbounded-read surfaces in their own narrow PRs. Do we have a high-confidence way to reproduce the issue? Yes. Current main has raw Is this the best way to solve the issue? Yes. Reusing the existing AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 4a503ed45e59. Label changesLabel changes:
Label justifications:
Evidence reviewedPR surface: Source +27, Tests +181, Other +208. Total +416 across 4 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
Replace two raw response.json() calls in pollDashscopeVideoTaskUntilComplete and runDashscopeVideoGenerationTask with the canonical readProviderJsonResponse helper (default 16 MiB cap) so a faulty or hostile DashScope endpoint streaming an unbounded body cannot force the runtime to buffer the whole payload before parsing. The download path already used readResponseWithLimit; this aligns the JSON paths with the same byte-cap invariant. Sibling of Alix-007's bounded-stream work (openclaw#95108, openclaw#95218, openclaw#95417, openclaw#95418, openclaw#95420, openclaw#95246); this PR applies the same pattern to the video-generation call sites in src/video-generation/dashscope-compatible.ts. Tests: 3 unit tests in src/video-generation/dashscope-compatible.bounded-json.test.ts covering well-formed parse, oversize body stop-early, and submit-task routing. Real-environment proof: scripts/repro/issue-dashscope-response-cap.mjs demonstrates a 32 MiB streaming body stops after 17/32 chunks via the bounded helper, while raw response.json() on a 2 MiB body buffers the whole payload before failing on JSON parse.
c417fc2 to
ee14da9
Compare
|
@clawsweeper re-review @clawsweeper — addressing the merge-readiness blocker on PR #96036 (commit
Verification (all green):
|
|
🦞🧹 I asked ClawSweeper to review this item again. |
…penclaw#96036) Addresses the ClawSweeper P1 finding that `src/plugin-sdk/test-helpers/provider-http-mocks.ts` mocks the `openclaw/plugin-sdk/provider-http` module without exporting `readProviderJsonResponse`, causing import-time failure in Qwen/Alibaba video tests now that `dashscope-compatible.ts` routes submit/poll through the bounded helper. Changes: - `src/plugin-sdk/test-helpers/provider-http-mocks.ts` (+22): - Added `readProviderJsonResponse` type import - Added `ReadProviderJsonResponseParams` type alias - Added `readProviderJsonResponseMock` field to `ProviderHttpMocks` - Added hoisted mock default implementation that calls `response.json()` (parity with the legacy unbounded read, leaves cap enforcement to the production helper) - Added to `vi.mock(openclaw/plugin-sdk/provider-http, ...)` export map - Added to `installProviderHttpMockCleanup` mockClear list - `scripts/repro/issue-dashscope-response-cap.mjs` (+100): - Existing helper-only assertion kept (proves helper stops early) - New assertion: drives `pollDashscopeVideoTaskUntilComplete` end-to-end with a 100 MiB streaming body, asserts bounded-reader error surfaces before the runtime buffers the full payload - New assertion: drives `runDashscopeVideoGenerationTask` end-to-end with a 100 MiB streaming body on the submit response, asserts the submit path is also bounded - Negative control kept (raw `response.json()` buffers full body) Verified on commit: - `node scripts/run-vitest.mjs extensions/qwen/video-generation-provider.test.ts` — 5/5 passed (was 3/5 fail before this fix) - `node scripts/run-vitest.mjs extensions/alibaba/video-generation-provider.test.ts` — 3/3 passed - `node scripts/run-vitest.mjs src/video-generation/dashscope-compatible.bounded-json.test.ts` — 4/4 passed - `pnpm exec tsx scripts/repro/issue-dashscope-response-cap.mjs` — 4/4 PASS (helper + poll path + submit path + negative control) - `npx oxlint --import-plugin --config .oxlintrc.json` on changed files — exit 0 - `node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.extensions.test.json` — exit 0
|
@clawsweeper re-review @clawsweeper — addressing the ClawSweeper P1 finding on PR #96036 (commit
Output: Verification (all green on commit
The merge-blocking P1 (provider-http mock contract gap) should now be cleared. PR body unchanged. |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@sallyom — this PR is the DashScope half of the same bound- 🐚 platinum hermit + 👀 ready for maintainer look, with a standalone repro script Happy to address any review feedback or coordinate the merge ordering with the OpenAI |
|
Closing in favor of an Alix-007-style XS reshape. The actual code change is +9/-2 = 7 lines across 2 call sites — the current PR's 418 LoC comes from a committed repro script (208 LoC) + new test file (181 LoC). Per Alix-007 pattern: 1 source file, inline test in existing describe, local-only proof, XS size. New PR will replace the same 2 await response.json() calls with readProviderJsonResponse — same fix, under 20 LoC total. |
What Problem This Solves
src/video-generation/dashscope-compatible.tsreads two provider-controlledresponse bodies (the submit-task response and the poll-status response) with
raw
await response.json()calls. That call has no byte cap, so a faulty orhostile DashScope endpoint streaming an unbounded body can force the runtime
to buffer the whole payload before parsing — an OOM / DoS surface in the
production path. The download path on line 338 already uses
readResponseWithLimit; this brings the submit and poll paths into thesame bound-read family.
The shared
readProviderJsonResponsehelper inopenclaw/plugin-sdk/provider-http(used by the bound-stream family mergedin #95218, #95417, #95418, #95420, #95246, #96136) caps the read at 16 MiB
and surfaces an actionable error before OOM. This PR routes both the
submit and poll DashScope paths through that helper.
Why This Change Was Made
provider-http, and alreadyproven on
provider-http-errors.tsand the image-generation/video-generation sibling surfaces.
public API change.
Changes
src/video-generation/dashscope-compatible.ts—+11/-2— importreadProviderJsonResponsefromopenclaw/plugin-sdk/provider-http;replace two raw
await response.json()calls (line 203, the poll-statuspath, and line 273, the submit-task path) with
readProviderJsonResponse<DashscopeVideoGenerationResponse>(response, label).src/video-generation/dashscope-compatible.bounded-json.test.ts—+181/-0— new test file. 4 unit tests covering: well-formed pollparse, overflow on the poll path (stops at 17/32 chunks for a 32 MiB
body), well-formed submit path parse, and overflow on the submit
path via
runDashscopeVideoGenerationTask(directly exercises thesubmit function ClawSweeper's previous review flagged as "not
directly proven").
scripts/repro/issue-dashscope-response-cap.mjs—+114/-0— newreal-environment repro driving the production
pollDashscopeVideoTaskUntilCompletewith a 32 MiB streaming bodyand proving the bounded reader stops at 17/32 chunks. Includes a
negative control showing that raw
response.json()on the samebody buffers the full 32 MiB before failing on parse.
Evidence
Real-environment repro output
Unit tests
The 4th test ("surfaces a verbatim bounded-reader error from the submit
path") calls
runDashscopeVideoGenerationTaskend-to-end with a 100MiB streaming body on the submit response and asserts the bounded
reader throws with the canonical overflow error and stops pulling
chunks well before the body completes.
ClawSweeper finding addressed in this revision
The previous review's blocker said "the branch is not merge-ready
because it misses the current provider-http test mock contract and
does not actually exercise the submit helper it changes."
that calls
runDashscopeVideoGenerationTaskand asserts theverbatim overflow error from the bounded reader.
openclaw/plugin-sdk/provider-http(readProviderJsonResponse),so the test exercises the canonical helper, not a local copy.
Verification
node scripts/run-vitest.mjs src/video-generation/dashscope-compatible.bounded-json.test.ts— 4/4 passed
pnpm exec tsx scripts/repro/issue-dashscope-response-cap.mjs— 2/2 PASSnpx oxlint --import-plugin --config .oxlintrc.jsonon the 3 changedfiles — exit 0
openclaw/main(4d034639ad); no mergeconflicts.
CI status note
The most recent run reported one failing check
(
checks-node-compact-large-1/src/agents/session-write-lock.test.ts/acquireSessionWriteLock > retries when a stale lock report is replaced by a fresh payload-less lock). This failure is in thesession write-lock retry path, which shares no surface with the
video-generation change in this PR. The PR only touches
src/video-generation/dashscope-compatible.tsand its test/reprofiles; the failing test exercises
acquireSessionWriteLockinsrc/agents/session-write-lock.ts, an unrelated module. A re-runshould clear it.
Out of scope
src/agents/chutes-oauth.ts(4 sites),src/infra/clawhub.ts(3 sites), and others are intentionallyleft for follow-up PRs.
readResponseWithLimitdownload path is already bounded in thisfile and does not need changes.