Skip to content

perf(aws-lambda): cloud renders plan without reading video bytes and start chunks sooner - #6194

Merged
miguel-heygen merged 25 commits into
mainfrom
perf/tail-probe-slots
Oct 24, 2026
Merged

miguel-heygen merged 25 commits into
mainfrom
perf/tail-probe-slots

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

What changed

A cloud render's plan no longer touches video bytes, and each chunk starts capturing sooner. Six changes, each in the code it names:

  1. Stored probe facts. sites create stores each video's ffprobe JSON, its final-frame time and its last packet length beside the video, keyed by content hash (like the audio sidecars). The plan reads that instead of probing the file over the network.
  2. Frame counts without decoding. A window that ends a second or more inside its source takes the expected frame count. A window that reaches the source's end takes it from the stored final frame (constant frame rate; a late-starting video stream only when its last packet is a full frame, otherwise the old tail decode runs). Remaining tail decodes run 16 wide (HF_TAIL_PROBE_SLOTS).
  3. Chunk encode overlaps capture for closed-GOP chunks (canEncodeWhileCapturing no longer excludes lockGopForChunkConcat); output is byte-identical to encoding after capture (new test).
  4. Locked warmup ticks stop sleeping once the page is ready. The tick count stays 60, so frame times are unchanged; each Chrome start saves ~2 s, twice per chunk.
  5. Logs: [plan] extract phases, [audio] phases (the plan never passed its logger to the audio stage).
  6. Engine package size budget raised to 456,055 packed / 1,526,873 unpacked bytes (5% over measured): the facts reader and tail-count code live in the engine.

What I measured (93-cut 4K edit, 6 sources, 22 GB, 840 s, cold functions, real sites create)

0.8.190 this PR
wall 389 / 387 / 303 s 66 / 49 / 51 s
cost $0.92-0.97 $0.93 / 0.90 / 0.93
plan 118-198 s 2.2 s (14 s on the first plan after a redeploy)
plan memory 5.4 GB 0.55 GB
  • Frames and audio are identical to 0.8.190 on every run (21,000 frames, same md5 per run).
  • 30-minute 1080p edit, 204 chunks, real sites create (15 s): wall 38.5 / 40.7 / 38.9 s, $0.547 / 0.531 / 0.550, plan 2.5-2.9 s; the 54,000 frames and the audio are identical across the three runs. The released 0.8.190 renders it in 41.9 s / $0.580 (one cold run, the first after a redeploy) with the same frames and audio, so on a single long clip this PR is about 6% faster and 7% cheaper; the big win is the cut-heavy multi-source edit above.
  • Warmup change: 4 distributed-simulated fixtures (CSS keyframes from t=0, a video at frame 0, a web font, GSAP) rendered with and without the sleeps: 0 differing frames, identical final mp4s.
  • End-window count: 8,037 randomized windows against real ffmpeg: 0 wrong, 3 declined (fall back to the decode).
  • Red then green: remoteTail, tailCount, encodeWhileCapturing, warmupTicks, plan audio-log tests each fail without their change.

What I did NOT exercise

  • The stored final frame is used only when the plan's ffprobe has the same major version as the one that stored it (otherwise the plan probes the tail itself). The stream metadata JSON is still trusted, and the md5 runs above used a sites create ffprobe of the same build as the Lambda's, so they do not exercise a version mismatch.
  • Sites made before this change have facts without the last packet length: their end windows still decode the tail (no regression, no speedup) until sites create runs again.
  • A late-starting video whose last packet is short, VFR sources, HDR and color-converted windows: all fall back to the old decode.
  • GPU hosts and non-Lambda targets for the warmup change; the frame check ran on a 4-vCPU SwiftShader host and on Lambda.
  • Cost is flat to slightly lower per render (Lambda-seconds), not a large drop; the chunk time (~27 s x 254 chunks) is the remaining lever.

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 168 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

@miguel-heygen miguel-heygen changed the title perf(engine): a remote source's tail decodes run many at a time in the plan perf(aws-lambda): cloud renders plan without reading video bytes and start chunks sooner Oct 24, 2026

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 blocker · 2 important · 0 nits · head f26b0633

No blockers. The two important items below are follow-ups: both have a narrow trigger, and neither affects the measured paths.

Scope

Audited: ffprobe.ts (stored facts, final-packet probe), videoFrameExtractor.ts (tailFramesFromFinalFrame and the deferred-window count), renderChunk.ts (overlap lifecycle across the capture .catch, screenshot-fallback reset, and finally), encodeStage.ts, frameCapture.ts warmup, deploySite/siteManifest/s3Transport/contentStore, plan and audio logging.
Checked: chunk landed-frame indices are 0-based within the range (parallelCoordinator outputFrameOffset), which matches encodeWhileCapturing's 0..framesInChunk. Adaptive capture retries only recapture missing frames, so frames already fed stay valid. A screenshot-mode retry aborts the encoder before resetForScreenshotRetry wipes workDir.
Trusting: the new tests, which I could not run locally (no bun in the sandbox); CI on this head was still running when I posted. CodeQL's two "network data written to file" alerts on probeFacts.ts are false positives: the file is local ffprobe JSON written to a temp path.

Comment thread packages/engine/src/utils/ffprobe.ts Resolved
One line per audio stage: prepare, groups, mix, master, encode and the peak check, in ms.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 0 resolved, 2 open, 0 new · f26b063 → a8855b8. Rebase only: the patch matches the previous head apart from main's renameWithRetry context lines in audioMixer.ts and the import list. The two important follow-ups (warmup hot loop, ffprobe version skew) are unchanged and still non-blocking.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 2 resolved, 0 open, 0 new · a8855b8 → 901dad9. Warmup now skips the sleep only after a successful tick, and the new test pins a dead page. Stored final frames are ignored on an ffprobe major-version mismatch. Legacy facts and the stored ffprobe JSON are still trusted, as the thread reply states. Accepted as partial.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 0 resolved, 0 open, 0 new · 901dad9 → b7371c2. Test-only: the ffprobeMajor assertion now compares against the build's own banner, using the same regex as the code. Product code is unchanged.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 0 resolved, 0 open, 0 new · b7371c2 → 8e281d8. getFfprobeBinary() moves into a named local in the test and in ffprobe.ts's ffprobeMajor. The ffprobe.ts edit is a product line, but it does not change behavior.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 24, 2026
Merged via the queue into main with commit da4386f Oct 24, 2026
111 of 113 checks passed
@miguel-heygen
miguel-heygen deleted the perf/tail-probe-slots branch October 24, 2026 14:47
diogochaves added a commit to diogochaves/hyperframes that referenced this pull request Oct 24, 2026
…v adds

The increase sits on top of main's engine budget, 456055 after heygen-com#6194.
Main packs the engine at 435883 bytes and this branch at 436205.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@heygengenesis heygengenesis Bot mentioned this pull request Oct 24, 2026
DanielMcSheehy pushed a commit to DanielMcSheehy/hyperframes that referenced this pull request Oct 25, 2026
…pins a GPU (heygen-com#5976)

* fix(engine): software renders reach SwiftShader when VK_DRIVER_FILES pins a GPU

ANGLE (and Dawn) load SwiftShader by pointing the Vulkan loader at its ICD
through VK_ICD_FILENAMES. The loader ignores VK_ICD_FILENAMES while
VK_DRIVER_FILES is set, and still applies VK_LOADER_DRIVERS_SELECT/DISABLE,
so on a machine that pins its Vulkan driver that way a --no-browser-gpu
render gets ANGLE on the real GPU and assertSwiftShader refuses it.

Launch SwiftShader browsers without those three variables. Hardware
launches, and environments that set none of them, are unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* chore(engine): raise the packed size budget by the 342 bytes launchEnv adds

The increase sits on top of main's engine budget, 456055 after heygen-com#6194.
Main packs the engine at 435883 bytes and this branch at 436205.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants