Skip to content

fix(cli): check finishes on video-heavy projects, its geometry seeks skip only video-frame waits - #5715

Merged
miguel-heygen merged 4 commits into
mainfrom
fix/check-geometry-seek-no-media-wait
Oct 17, 2026
Merged

miguel-heygen merged 4 commits into
mainfrom
fix/check-geometry-seek-no-media-wait

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

What

hyperframes check finishes on video-heavy projects again. Its dense overlap grid no longer waits for each video's frame to land before it reads element boxes, so a 79 s project of 1080p H.264 clips goes from "Check did not finish within its deadline" (122 s) to a full report in 12 s, with the same findings.

Why

A user's check on a 79 s project (7 H.264 1080p clips at the root, a captions sub-composition with 58 timed groups, a graphics sub-composition) returned browser: failed every time with "Check did not finish within its deadline", while snapshots at any time worked.

The dense content_overlap grid (collectMotionOverlapSamples, utils/checkPipeline.ts) seeks up to 600 times, 8 per second of video. DENSE_GEOMETRY_SEEK_OPTIONS already drops every post-seek settle, because getBoundingClientRect is valid synchronously after the seek. But seekCompositionTimeline still awaited the runtime's seek barrier (__hfWaitForSeekCompletion) on every one of those seeks, and that barrier holds until the on-screen video has seeked and decoded its frame (holdSeekBarrierUntilVideoLands, core/src/runtime/media.ts, up to 5 s). The geometry pass never reads those pixels.

Related work

None open.

How

  • The seek barrier (registerSeekCompletion, core/src/runtime/adapters/seek-dispatch.ts) now knows which of its holds are only a video frame landing: holdSeekBarrierUntilVideoLands (media.ts) and the proxy swap (mediaProxy.ts) register as "video-frame".
  • waitForSeekCompletion({ videoFrames: false }) waits for every other hold: frame sources' asynchronous draws (frameSources.ts), hf-seek waitUntil work, VFX paints, image decodes. Called with no argument, it waits for all of them, as before.
  • DENSE_GEOMETRY_SEEK_OPTIONS sets waitForVideoFrames: false, and seekCompositionTimeline passes it to the barrier. Every other seek, the audit grid's included, still waits for video frames. A page whose runtime predates the option ignores the argument and waits for all holds, which is slow but correct.
  • The geometry pass reads element boxes, not pixels, so it never needed the video frame. It does need a frame source that moves DOM to finish its draw (review finding), and it now waits for that.

Test plan

  • Unit tests:

    • captureCompositionFrame.test.ts, "skips only the seek barrier's video-frame holds, while the audit seek waits for all": runs the page script the seek sends against a fake window, and checks the barrier is asked with { videoFrames: false } for dense seeks and { videoFrames: true } for audit seeks.
    • media.test.ts, "does not hold a geometry read for a seeking video's frame": a seeking video holds the default barrier, not the videoFrames: false one.
    • frameSources.test.ts, "holds a geometry read that skips video frames until its asynchronous draw lands": the review's case, a frame source whose draw is still pending holds the videoFrames: false barrier until it lands.
    • core seek-dispatch, media, frameSources and mediaProxy tests: 229 passed; CLI src/capture/: 466 passed, 4 skipped; each 3 runs in a row. tsc --noEmit for core and the CLI, oxfmt, oxlint and the comment ratchet clean.
    • fails without the fix: with media.ts not tagging its hold, the media test fails (expected false to be true); with the CLI's seek from main, the CLI test fails (expected [] to deeply equal [ [ { videoFrames: false } ] ]). The frame-source test guards the review's case: it fails if a frame source's draw stops holding the videoFrames: false barrier.
  • Manual: a fixture of the report's shape, made with ffmpeg testsrc2 clips (hyperframes check --json, Linux, software GL):

    Time Result
    0.8.165 as published 122 s check_runtime_failure: "Check did not finish within its deadline."
    this branch (built from source) 14 s completes, with the same findings as an unchanged 0.8.165 run that happened to finish inside its deadline: 4 contrast_aa_failure, 1 escaped_container, 1 caption_track_kind_missing, 58 timed_element_missing_clip_class

    Inside the browser pass of the first version of this PR (barrier skipped outright), the 595 dense geometry seeks took 109.6 s with the barrier wait and 1.6 s without it. The 14 s run is this version, which still drains the barrier minus its video-frame holds; the fixture has no frame sources, so nothing else holds it.

  • Documentation updated (not applicable)

  • Comments follow CONTRIBUTING.md "Comments"

Not exercised: a macOS run; the user's own project.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 187 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

@terencecho terencecho left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed HEAD 4bad251c0feb610c29a1e5731a94f6974347c732 against effective base d05bc4215d5757da86956217f4a8a4b2ab11b1fd.

[P2] Keep asynchronous DOM-source completion in dense geometry seeks

packages/cli/src/capture/captureCompositionFrame.ts:42,344–351

The opt-out skips the whole shared seek barrier, not only video decoding. The public registerFrameSource contract accepts any Element and an asynchronous render callback. The real adapter registers source.seek(sourceTime) with this same barrier (frameSources.ts:130–143); its queue serializes asynchronous draws and coalesces later requests. renderSeek dispatches adapters synchronously but does not await those DOM mutations. The dense pass now reads text geometry before that work finishes.

Concrete source-derived scenario: a 2s, 30fps, data-no-timeline composition has two opaque, absolutely positioned text blocks. A deterministic registered source waits 500ms, then puts them together only for source times 0.45–1.2s. With sparse --at 0,1.875, both ordinary audit samples are separate. BASE waits for each dense source draw and observes overlap at 0.5, 0.625, 0.75, 0.875, 1 and 1.125s: six observations spanning 625ms, which the existing persistence rule promotes to an error. On HEAD, a legal 150ms dense traversal finishes before the first asynchronous draw; reads see the previous non-overlapping pose, intermediate seeks coalesce, and all six findings disappear. Later completion cannot repair samples already collected. Restoring only the completion wait restores the observations; a synchronous DOM renderer also remains correct with the opt-out.

Please retain synchronization for asynchronous DOM-producing frame sources while bypassing unnecessary media-frame completion. The new fake-page test proves that the barrier call is skipped and the default still waits, but it never executes the runtime/source queue or reads geometry, so it does not protect this contract.

Validation and scope: source-only review, including both changed files in full (BASE before HEAD), the actual source queue/barrier/seek dispatch, dense driver and overlap reader, and persistence/report paths; an independent adversarial pass corroborated this finding. The schedule above is a manually authored finite model, not a browser reproduction or measured benchmark. No repository tests, browser, native tools or CI were run. Unrelated adapters/platform execution remain outside this semantic audit; the author's performance measurements were not independently reproduced.

Verdict: REQUEST CHANGES
Reasoning: Avoiding video decoding during text-geometry sampling is reasonable, but bypassing all completion work also introduces a reachable false-negative path for supported asynchronous DOM sources. Preserve that geometry contract without reinstating unnecessary video-decode waits.

— tai

@miguel-heygen
miguel-heygen force-pushed the fix/check-geometry-seek-no-media-wait branch from 4bad251 to 4048562 Compare October 16, 2026 20:24
@miguel-heygen miguel-heygen changed the title fix(cli): check finishes on video-heavy projects, its geometry seeks no longer wait for frames fix(cli): check finishes on video-heavy projects, its geometry seeks skip only video-frame waits Oct 16, 2026
@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

Agreed, the opt-out skipped the whole barrier, frame sources included. Fixed at the barrier in 4048562:

  • registerSeekCompletion(promise, "video-frame") tags only the video-landing holds: holdSeekBarrierUntilVideoLands (media.ts) and the proxy swap (mediaProxy.ts).
  • waitForSeekCompletion({ videoFrames: false }) still waits for everything else: frame-source ready/seek draws, hf-seek waitUntil work, VFX paints, image decodes. With no argument it waits for all, as before.
  • The dense geometry seek always calls the barrier now, with { videoFrames: false }. Audit and every other seek pass { videoFrames: true }. A runtime without the option ignores the argument and waits for all.

Tests:

  • frameSources.test.ts covers your case: a frame source whose asynchronous draw is pending holds the videoFrames: false barrier until it lands.
  • media.test.ts: a seeking video holds the default barrier, not the geometry one.
  • The CLI test now runs the real page function against a stubbed window and checks the barrier is called with { videoFrames: false } for dense seeks and { videoFrames: true } for audit seeks.

The report-shaped fixture still finishes in 14 s with the same findings, since it has no frame sources.

@terencecho terencecho left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed HEAD 404856287b4c409ff2e5406bc8879561706463b6 against effective base 41486b9725ab2aae4f457d75dcd625931d103bb3.

The previous asynchronous DOM-source finding is repaired: dense seeks now invoke the shared completion barrier, excluding only classified native-video/proxy-frame holds. Frame-source readiness/draws, synchronous hf-seek registrations, VFX capture and image-held seeks remain generic completions; ordinary audit and inspected export capture retain full waits. Skipping video holds does not remove them from a later full barrier.

Audited: all eight changed files in full on both effective BASE and HEAD (BASE first), the actual fixing patch and public review history, plus bounded production caller/producer/consumer integration. Traced CLI option serialization to the installed runtime hook, dense geometry sampling to its text-rectangle reader, frame-source queue/abort ownership, GPU heartbeat interaction, retained failures/concurrent drains/reset, native/proxy cleanup and default engine capture. Both independent bounded source-audit lanes completed; no concrete introduced reachable blocker survived.

Trusting / limits: untouched repository behavior outside that bounded integration audit, actual browser/native decoder behavior and platform performance are not independently validated. Read the regression tests, including the real frame-source adapter/barrier case and the serialized CLI callback case, but did not execute tests, CI, browsers or benchmarks. The author's timing/platform results are not independently reproduced.

Verdict: APPROVE
Reasoning: The selective video-frame opt-out preserves the asynchronous DOM/GPU completion contract that the earlier whole-barrier bypass broke, while default capture behavior remains wait-all in the inspected production paths. No blocking introduced defect found in this source-only review.

— tai

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 16, 2026
Merged via the queue into main with commit b1fbc16 Oct 17, 2026
123 of 124 checks passed
@miguel-heygen
miguel-heygen deleted the fix/check-geometry-seek-no-media-wait branch October 17, 2026 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants