Skip to content

fix(producer): graded video renders split into chunks that finish inside the worker limit - #5883

Merged
miguel-heygen merged 2 commits into
mainfrom
cloudrender/plan-frame-cost
Oct 19, 2026
Merged

miguel-heygen merged 2 commits into
mainfrom
cloudrender/plan-frame-cost

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

What changes for users

A distributed render of a composition with a color-graded <video> now gets chunks small enough to finish inside the worker time limit with default flags. Before, the planner priced every frame at a fixed cost per pixel (about 0.23 s for a 1080x1920 frame). A graded frame costs about 10 s on software GL, so a 45 s graded render was planned as 4 chunks of 270 frames, each needing about 2,800 s against a 900 s limit. Every chunk timed out and was retried until the execution gave up.

Compositions without color grading are planned exactly as before and pay nothing extra.

How

  • resolveMaxParallelChunks takes an optional measured seconds per frame and prices frames at the larger of it and the per-pixel estimate. Everything downstream (the one-wave width, the fit-the-limit split, the CHUNK_EXCEEDS_WORKER_TIMEOUT refusal) uses that price.
  • The planner measures only when the composition has data-color-grading and the plan sizes chunks from the worker limit or concurrency (no explicit chunkSize, targetChunkFrames or maxParallelChunks).
  • The measurement is the in-process renderer's own calibration (measureCaptureCostFromSession: 5 frames spread over the timeline, p95), run in a session set up like a chunk worker's: same capture options (buildChunkCaptureOptions, now shared with renderChunk), same software-GL engine config, and the same video frames injected. planV2 defers video extraction to chunks, so the planner extracts just the sampled frames first; without them a graded video has no frame to draw and would measure cheap.
  • It runs beside the audio mix and pre-hash, so it adds wall time only when it is the slowest of the three.
  • chunkFrames.ts is a move, not new logic: the chunk's frame and capture-option helpers left renderChunk.ts so the planner can use them without an import cycle (renderChunk imports planV2, which imports plan). renderChunk.ts re-exports what it exported before.

Interaction with the chunk cap

With default flags the measured price flows into the fit-the-limit split, which may exceed the one-wave width (it warns and runs in waves, as today). An explicit --chunk-size still pins chunking and is still capped at one wave (16 when the worker concurrency is unknown); this PR does not change that.

Measured

Devbox (Linux, software GL, same flags as a chunk worker, a shared and loaded box), planV2 with a 900 s worker limit and 8 workers, 1080x1920 at 24 fps. The graded and ungraded variants differ only by the data-color-grading attribute on the video. Chunks rendered one after another with renderChunkV2.

variant frames measured cost per frame (plan) plan time chunks, per-pixel price (main) chunks, this PR predicted per chunk measured per chunk
ungraded 5 s 120 not measured 0.9 s 1 x 120 1 x 120 87 s 20.4 s
graded 5 s 120 10.54 s (p95 of 5) 73.6 s 1 x 120 8 x 15 218 s 160-178 s (8 chunks, all under the prediction)
ungraded 45 s (plan only) 1,080 not measured 1.4 s 4 x 270 4 x 270
  • On main the graded 5 s render is one 120-frame chunk predicted at 87 s. The same 120 frames took 1,276 s in one local render on this box, so that chunk cannot finish inside 900 s.
  • The measurement costs about 72 s of plan time on the graded render: browser start and the grading shader compile (~17 s), then 5 frames at ~10.5 s. Ungraded renders skip it, so their plan time does not change.
  • For 45 s graded (1,080 frames) the same price gives 19 chunks of about 57 frames, about 660 s each, in 3 waves on 8 workers (from the sizing test; not rendered here).

Tests

  • plan.test.ts: a measured 10.5 s frame turns a 1,080-frame 1080x1920 plan whose chunks would each need far over 900 s into chunks that fit 75% of the limit, and a cheap measured frame keeps the per-pixel plan; plan() measures a graded fixture (mocked measurement) and splits it, and never measures the ungraded fixture. Both fail without the change (checked by removing the measured price from the sizing and by skipping the re-size in the planner).
  • renderChunk.test.ts: the deferred-frame extraction tests now pass frame lists.

Not in this PR

The grading shader itself is the cost: it compiles every effect into one program and software GL pays for the disabled ones. Compiling only the enabled effects is a separate change.

…ide the worker limit

The distributed planner priced every frame at a fixed cost per pixel, so a
color-graded video, which costs about 10 s a frame on software GL, was
planned into chunks that could never finish before the worker time limit.

When the composition has color grading and the plan sizes chunks from the
worker limit, the planner now captures the renderer's calibration frames in a
session set up like a chunk worker's and prices frames at the measured p95
when it is higher than the estimate. Compositions without grading never pay
for the measurement and keep their current plan.
@github-actions

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 192 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (3)

  • resize-size-px-r0-nested-z100-on: tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey 0.02, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey 0.02, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey 0.02, undo true, teleport true
  • resize-idle-px-r0-nested-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true
  • seqheldtwice-tlhold-px-r0-nested-z100: tracking 120.35, pressJump 312.3, drop 0.07, reload 0.07, render 0.03, renderKey -, undo true, teleport false / tracking 0.09, pressJump 0.11, drop 0.39, reload 0.39, render 0.02, renderKey -, undo true, teleport true / tracking 0.09, pressJump 0.11, drop 0.39, reload 0.39, render 0.02, renderKey -, undo true, teleport true

@vanceingalls vanceingalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at cf4e772a.

Strengths

  • plan.ts:427-429: frames are priced at max(per-pixel estimate, measured), so the measured cost can only add chunks. It never lowers the price below the existing estimate, and an ungraded plan is unchanged by construction.
  • chunkSizingUsesFrameCost (plan.ts:448) lifts the old pinsChunking condition into one predicate. resolveMaxParallelChunks and the measure-or-not decision both use it, so they can't disagree.
  • The measuring session mirrors a chunk worker. It uses the same buildChunkCaptureOptions, browserGpuMode: "software" (plan.ts:1086), and the injected video frames. It extracts the deferred frames for exactly the indices selectCaptureCalibrationFrames(totalFrames) returns, which is the function measureCaptureCostFromSession calls too, so no sampled frame is drawn without its video.

Review focus

  • Sample size and outliers: 5 frames (0, 25%, 50%, 75%, last). The p95 of 5 is the slowest sample (captureCost.ts:222), so a cold or contended outlier biases the price up. That's the safe direction: more chunks, never a chunk that runs too long. Browser start and shader compile happen in initializeSession, before the timed samples, and the body's 10.54 s next to the ~17 s setup is consistent with that. Static-frame dedup is bypassed during calibration (captureCost.ts:272-273), so a sample can't read near 0 ms.
  • Chunk bounds: no zero-frame or runaway chunks. wide is clamped to ≥1 (:515). If even one frame doesn't fit the margin, maxFramesPerChunk < 1 makes needed Infinity, which raises the typed ChunkExceedsWorkerTimeoutError (:522-529). The 256-chunk ceiling also raises that error rather than producing an oversized chunk, and resolveChunkPlan still asserts positive integers. At 10.5 s/frame with a 900 s limit, a chunk holds 58 frames, so the refusal starts at about 14.8k frames (about 10 min at 24 fps).
  • Timeout margin: unchanged at 0.75 (:424), and the measured chunks (160-178 s against a 218 s prediction) sit comfortably inside it. Chunk workers run several capture workers (calculateOptimalWorkers) while the planner measures one session, so the prediction is conservative.
  • Non-graded renders: with measureFrameCost false, the third allSettled slot is undefined and the chunk plan is the same resolveChunkPlan(..., maxParallel, ...) as before, just computed later. Also, chunkFrames.ts is a faithful move. I diffed the removed renderChunk.ts lines against the new file with comments stripped. The only changes are slice → frames: Iterable<number> (the caller expands the slice) and the capture-options object turned into buildChunkCaptureOptions. That reads encoder.quality, which is what job.config.quality was (buildSyntheticRenderJob({ quality: encoder.quality })).

Important (non-blocking)

  1. When measurement fails (plan.ts:1431), a rejected measurement now fails the whole plan. That covers Chrome launch, ffmpeg extraction of the sampled frames, or a capture timeout under the shortened calibration protocolTimeout. The measurement is sizing advice, and the per-pixel price is still available. Falling back to it, or to a conservative floor, with a log.warn would keep a transient plan-side flake from failing a render the chunks might have finished. Failing fast is defensible, since an under-sized graded plan times out anyway, so I'm not blocking. If it stays, a sentence in the code saying it's deliberate would help.

Nits
2. planNeedsFrameCostMeasurement (:444) is a static regex on the compiled HTML. Grading added at runtime by script isn't seen, and that render keeps the old per-pixel plan. The attribute appearing in text or comments only costs an unnecessary measurement. Both are fine for now.
3. The measurement runs alongside the audio mix and prepublish on the same box. On a small plan instance that contention inflates the p95. That's the safe direction, but it's worth knowing when reading a surprisingly high [plan] measured frame cost.

CI at cf4e772a: all 8 required checks passed. The first CI run on this head (run 39264487869, 13:56Z) ran Build, Test, Test: runtime contract and Typecheck, plus Producer unit and integration tests, and all succeeded. Windows render verification run 39264487857 passed Render on windows-latest and Tests on windows-latest. A later re-trigger at 14:18Z on the same SHA saw no changed paths and skipped them. regression and Semantic PR title are green.

Verdict: APPROVE
Reasoning: The measured price can only make chunks smaller, chunk-count bounds and the timeout refusal are intact, non-graded plans are unchanged, and the extracted module is a faithful move. Required CI ran and passed at this exact head.

— Via

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 19, 2026
Merged via the queue into main with commit 6c3cc7c Oct 19, 2026
118 checks passed
@miguel-heygen
miguel-heygen deleted the cloudrender/plan-frame-cost branch October 19, 2026 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants