Skip to content

fix(runway): bound video create/poll response reads#96907

Merged
vincentkoc merged 3 commits into
openclaw:mainfrom
Alix-007:fix/bound-runway
Jun 29, 2026
Merged

fix(runway): bound video create/poll response reads#96907
vincentkoc merged 3 commits into
openclaw:mainfrom
Alix-007:fix/bound-runway

Conversation

@Alix-007

Copy link
Copy Markdown
Contributor

What Problem This Solves

The Runway video success paths read control JSON with an unbounded await response.json() inside the shared readRunwayJsonResponse helper, which is used by both the create-submit path and the task-poll path. A hostile, misconfigured, or SSRF-reachable endpoint could stream a no-Content-Length body indefinitely and force OpenClaw to buffer the whole payload before parsing, causing OOM pressure or a hang on the video generation path.

The download path already uses readResponseWithLimit, so this shared helper was the one remaining unbounded success-path read in the provider. This is a defense-in-depth fix and a sibling-consistency cleanup: in practice the Runway submit/poll responses are small control JSON ({ id }, { id, status, output }), but the read was not bounded the way the rest of this provider and the already-merged byteplus/google/qwen video bounds are. Fixing the shared helper covers both create and poll in one place.

Changes

  • Rewrite readRunwayJsonResponse to delegate to the shared readProviderJsonResponse helper from openclaw/plugin-sdk/provider-http, reusing the existing 16 MiB provider JSON cap; no new abstraction. Its signature changes from Pick<Response, "json"> to Response so the bounded streaming reader can run.
  • Preserve the existing isRecord() top-level object guard and the ${label}: malformed JSON response wrapping, and keep both call sites (create-submit and task-poll), the status handling, the output-URL extraction, the download path, and release() cleanup unchanged.
  • Keep the shared provider-http test mock's readProviderJsonResponse reader real (via importActual) so the streaming size cap is exercised under test instead of stubbed, and update the focused Runway video tests to stream their submit/poll JSON through real Response bodies.

Real behavior proof

  • Behavior addressed: Untrusted Runway create-submit and task-poll JSON bodies must not be buffered whole on the success path. An oversized no-Content-Length body must stop at the 16 MiB cap, cancel the stream, and throw a bounded ...exceeds 16777216 bytes error with the provider's label. Small valid JSON must continue to parse.
  • Real environment tested: A real node:http server (createServer) bound to 127.0.0.1, streaming JSON in 64 KiB chunks with no Content-Length header and a would-stream size of about 64 MiB. Runway's generateVideo entry does not expose a request-level SSRF opt-in (unlike the OpenAI video provider), so it cannot fetch a loopback origin end-to-end. The proof therefore drives the real exported readProviderJsonResponse — the exact reader that the rewritten readRunwayJsonResponse now calls — over a real loopback socket, replicating Runway's thin isRecord() wrapper and using Runway's actual create and poll labels, and contrasts it against the old unbounded response.json() the helper used before. Node v22.22.0.
  • Exact steps: node --import tsx <scratchpad>/proof.mts — start the streaming server → fetch the streaming create route and run the real bounded reader with label "Runway video generation failed" and assert a bounded throw plus a server-side socket abort near the cap → fetch the streaming poll route and run the reader with label "Runway video status request failed" and assert the same → run a negative control showing the old unbounded response.json() (arrayBuffer() + JSON.parse) buffers the whole body past the cap → run the reader against small valid JSON and assert it still parses.
  • Observed result: the create read threw Runway video generation failed: JSON response exceeds 16777216 bytes and the server aborted after 19,398,663 bytes; the poll read threw Runway video status request failed: JSON response exceeds 16777216 bytes and the server aborted after 20,054,023 bytes (both just past the cap, far below the ~64 MiB they would have streamed). The negative control buffered the full 67,108,871 bytes, proving the cap is load-bearing. The small happy path parsed and returned the valid JSON through the same reader.
  • What was not tested: I did not hit the live Runway endpoint; a live endpoint would not reproduce a hostile oversized body. The Runway submit/poll responses are normally small control JSON, so this is a defense-in-depth bound rather than a fix for an observed live OOM. Because Runway's generateVideo does not expose a loopback SSRF opt-in, the TCP proof drives the exact shared reader the helper now uses with Runway's real labels rather than the full generateVideo entry; the provider-wiring (both call sites going through the rewritten helper) is covered by the focused in-repo Vitest regressions using the same reader and error text. The proof script is not committed.

Evidence

Real node:http terminal proof (real loopback socket, real exported bounded reader, real Runway labels):

[proof] real node:http server on http://127.0.0.1:33891, cap=16777216 bytes, would-stream≈67108864 bytes per overflow route

PASS  create submit JSON: overflow throws bounded error :: threw=true msg="Runway video generation failed: JSON response exceeds 16777216 bytes"
PASS  create submit JSON: stream cancelled near 16MiB :: aborted=true bytesSent=19398663 (<33554432)
PASS  poll status JSON: overflow throws bounded error :: threw=true msg="Runway video status request failed: JSON response exceeds 16777216 bytes"
PASS  poll status JSON: stream cancelled near 16MiB :: aborted=true bytesSent=20054023 (<33554432)
PASS  negative control: OLD unbounded read buffers PAST the 16MiB cap :: buffered=67108871 bytes (> 16777216)
PASS  negative control consumed far more than the bounded read :: unbounded buffered=67108871 vs bounded create sent≈19398663 (ctrlParse=threw-after-buffering)
PASS  happy path: small valid create/poll JSON still parses :: err=undefined id=task-happy status=SUCCEEDED

[proof] 7 PASS / 0 FAIL
[proof] ALL PASS

Focused Vitest:

node scripts/test-projects.mjs extensions/runway/video-generation-provider.test.ts

Test Files  1 passed (1)
Tests  9 passed (9)

Type checks:

node scripts/run-tsgo.mjs -p tsconfig.extensions.json ...                       # extensions source: 0 errors
node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.extensions.test.json ...    # extensions tests: 0 errors

Lint and format:

oxlint extensions/runway/video-generation-provider.ts extensions/runway/video-generation-provider.test.ts src/plugin-sdk/test-helpers/provider-http-mocks.ts   # 0 problems
oxfmt --write ...   # formatted
git diff --check     # clean

Note: the repo autoreview step could not run because its review LLM gateway returned 503 Service Unavailable during this session; the checks above were run locally instead.

Label: security

AI-assisted.

Rewrite the shared `readRunwayJsonResponse` helper (used by both the
create-submit and task-poll success paths) to delegate to the shared
byte-bounded `readProviderJsonResponse` (16 MiB cap) instead of an
unbounded `await response.json()`, matching the already-merged
byteplus/google/qwen video bounds. The download path already uses
`readResponseWithLimit`; this closes the remaining unbounded success-path
reads so a hostile or buggy endpoint cannot stream an unbounded body and
force OpenClaw to buffer it before parsing. The existing isRecord() object
guard and the `malformed JSON response` wrapping are preserved.

Keep the shared provider-http test mock's JSON reader real so the
streaming size cap is exercised under test.
@clawsweeper

clawsweeper Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 28, 2026, 10:45 AM ET / 14:45 UTC.

Summary
The PR changes Runway video create-submit and task-poll JSON parsing to use the shared bounded provider JSON reader and updates focused tests to use real Response bodies.

PR surface: Source -2, Tests -4. Total -6 across 2 files.

Reproducibility: yes. at source and proof level. Current main calls response.json() for Runway create and poll success bodies, and the PR body supplies loopback terminal proof contrasting bounded reads with the old unbounded path.

Review metrics: 1 noteworthy metric.

  • Runway JSON Success Reads: 2 reads bounded through 1 helper. Create-submit and task-poll metadata reads now share the same fail-closed 16 MiB provider JSON cap, which maintainers should notice before merge.

Root-cause cluster
Relationship: canonical
Canonical: #96907
Summary: This PR is the viable canonical landing path for the Runway create/poll bounded JSON reader work; the older same-root Runway PR is closed unmerged, while adjacent merged PRs only provide the shared helper or sibling-provider pattern.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Maintainer should explicitly accept the 16 MiB fail-closed compatibility boundary before merge.

Risk before merge

  • [P1] Successful Runway create or poll metadata responses larger than 16 MiB now fail closed; maintainers should accept that compatibility behavior for custom or proxy endpoints.
  • [P1] The PR body has strong loopback terminal proof, but the combined diff does not add committed Runway-specific oversized create/poll regression tests; maintainers can decide whether shared-reader coverage is enough.

Maintainer options:

  1. Accept The Shared Cap (recommended)
    Land the PR if maintainers accept that successful Runway create/poll control JSON over 16 MiB should fail closed under the shared provider cap.
  2. Request Runway Overflow Tests
    Ask for focused committed create and poll oversized-Response tests if maintainers want this provider-specific overflow proof locked into the repo before merge.
  3. Pause For Provider-Specific Limit
    Pause this PR if maintainers want Runway control JSON to use a different provider-specific size policy instead of the shared default.

Next step before merge

  • [P1] Human maintainer review should accept the fail-closed compatibility boundary; there is no narrow automated code repair from this review.

Security
Cleared: The diff reduces unbounded external-provider response buffering and introduces no dependency, workflow, lockfile, permission, package-resolution, secret-handling, or other supply-chain risk.

Review details

Best possible solution:

Land the shared-reader Runway fix after maintainer acceptance of the 16 MiB control-JSON cap; add committed oversized create/poll tests only if maintainers want provider-level regression coverage before merge.

Do we have a high-confidence way to reproduce the issue?

Yes, at source and proof level. Current main calls response.json() for Runway create and poll success bodies, and the PR body supplies loopback terminal proof contrasting bounded reads with the old unbounded path.

Is this the best way to solve the issue?

Yes, with maintainer compatibility sign-off. Reusing readProviderJsonResponse is the narrowest maintainable fix because it uses the established SDK helper and keeps Runway-specific object validation local; direct use of readProviderJsonObjectResponse would also be plausible but would not materially improve this patch.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 9241b9701d9c.

Label changes

Label justifications:

  • P2: This is normal-priority provider response hardening with limited Runway video-generation blast radius and no evidence of an active production outage.
  • merge-risk: 🚨 compatibility: The PR intentionally changes oversized successful Runway metadata responses from unbounded native parsing to a hard shared-provider-reader failure.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes sufficient terminal proof from a real loopback node:http server showing bounded create/poll overflow, stream cancellation, a negative control, and a small happy path.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes sufficient terminal proof from a real loopback node:http server showing bounded create/poll overflow, stream cancellation, a negative control, and a small happy path.
Evidence reviewed

PR surface:

Source -2, Tests -4. Total -6 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 8 10 -2
Tests 1 45 49 -4
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 53 59 -6

What I checked:

Likely related people:

  • steipete: Git history shows Peter Steinberger introduced the Runway video provider and made several later Runway/media-provider updates before this bounded-read follow-up. (role: feature introducer and recent area contributor; confidence: high; commits: f92ac83d889c, 7184200c17a0, a88c6f0fe713; files: extensions/runway/video-generation-provider.ts, extensions/runway/video-generation-provider.test.ts)
  • Alix-007: Authored the merged shared readProviderJsonResponse bounded-reader PR that this Runway change reuses, so the connection is to current-main helper history as well as this PR. (role: shared-reader contributor and current PR author with prior merged history; confidence: high; commits: 2592f8a51a4e, 2f684dfccc13, 094ab0e9002f; files: src/agents/provider-http-errors.ts, extensions/runway/video-generation-provider.ts, extensions/runway/video-generation-provider.test.ts)
  • vincentkoc: Recent provider HTTP history includes centralized provider request and transport policy work adjacent to the shared helper and provider response boundary used here. (role: adjacent provider transport contributor; confidence: medium; commits: b0f94a227b3e, c405bcfa9836, f28f0f29ba90; files: src/agents/provider-http-errors.ts, src/plugin-sdk/provider-http.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels Jun 26, 2026
@Alix-007

Copy link
Copy Markdown
Contributor Author

Updated this branch against latest upstream main to resolve the post-merge drift in the video bounded-read series. The provider HTTP test helper conflict was resolved by keeping the current main helper, which still exercises bounded streamed JSON behavior.

Verified current head locally:

  • node scripts/run-vitest.mjs extensions/runway/video-generation-provider.test.ts -- --run
  • git diff --check

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@vincentkoc vincentkoc self-assigned this Jun 29, 2026
@vincentkoc
vincentkoc merged commit 63b0893 into openclaw:main Jun 29, 2026
86 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants