Repository navigation
feat(cli): info --media lists every clip the render hears, on the film's clock - #7235
Merged
Merged
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
Contributor
There was a problem hiding this comment.
Approving at db86286c. No blockers.
What I checked:
- Fails without the fix: I removed the probe cap in
endedClip(returnedheardbefore the probe).projectMedia.test.ts > ends a speaking clip with no set length where its media runs out, not where its scene endsfails withend: 30instead of17. It passes at this head. - Suites:
projectMedia.test.ts+info.test.ts16/16.tscreports nothing inprojectMedia.tsorinfo.ts; the only errors locally come from the unbuilt inspector package. - Real CLI run (
tsx src/cli.ts, pinned ffmpeg 8): a project with a real 5 s mp4 (sine audio) used as a root<audio>, a scene at 10 s holding a wrapped clip, an unbounded clip withdata-media-start="1", and a muted clip.info --media --jsongivesbed0–3 (volume 0.5),a-audio10–15 (capped by the media fromdata-duration=9),b-audio22–26 (5 s − 1 s media start). The muted clip is correctly absent. Text mode prints threeHeardlines, andinfo --jsonwithout--mediahas noheardkey.
Non-blocking:
heardClipsrepeatscollectProjectMedia. It reimplements the same bundle +loadProducer()sequence in the same file, but without that function'scollectRenderMediaguard. If the loaded producer lacks the collector, you getis not a functioninstead ofRender media collector unavailable.(await collectProjectMedia(projectDir)).audioswould give one path and the clearer error.- A clip that starts past its media's end is still listed. When
seconds <= mediaStart, the clip keeps the collector's end and shows up as heard, even though the render plays nothing from it. For a captions consumer, dropping it (or ending it atstart) would be more accurate. - The probes aren't deduplicated.
Promise.allprobes every clip at once, once per clip even for the samesrc(three probes of one file in my run). Caching by path would help projects with many cuts of one recording.
Contributor
Edit accuracy: accurate 267 (base branch 267), smooth 100 of thoseThe gate passes. Quarantined, measured but not gated (8)
Unstable (1)
|
Collaborator
Author
|
All three notes taken at c8da1fb:
|
Contributor
There was a problem hiding this comment.
Approving at 30d0eb60. One correction to the handoff first: this head is not "otherwise unchanged". Compared with db86286c, it changes behaviour in heardClips and adds a test, answering all three of my earlier notes. I reviewed it as a code change:
- Duplicate media collection: fixed.
heardClipsnow goes throughcollectProjectMedia, so it keeps that function'scollectRenderMediaguard and error. - Clip starting past its media: fixed. A clip whose
mediaStartis at or beyond the probed length is now dropped (return null, then filtered out). A failed probe still keeps the clip at the collector's end, which is the right fallback. - Repeated probes: fixed. Lengths are memoised per resolved path as a shared promise, so concurrent clips of one file trigger a single probe.
Checked:
projectMedia.test.ts+info.test.ts: 17/17, including the newleaves out a clip that starts past its media's end, and reads each file's length once.tscis clean for the changed files, andoxfmt --checkpasses.- Real CLI run on my earlier project (real 5 s mp4): the same three
Heardlines as before (0–3, 10–15, 22–26).
miguel-heygen
enabled auto-merge
November 5, 2026 13:53
SanHsien
added a commit
to SanHsien/hyperframes
that referenced
this pull request
Nov 6, 2026
…eview Adopts or partially adopts 11 groups: multi-worker tween replay behind the timeline stand-in (heygen-com#7231), template entries in the shared compile stage (heygen-com#7217), HYPERFRAMES_CACHE_DIR and HYPERFRAMES_CONFIG_DIR for every per-user cache and settings file (heygen-com#7224, heygen-com#7229; HeyGen credentials stay under HEYGEN_CONFIG_DIR), WebKit audio clock (heygen-com#7225), packet-based video length (heygen-com#7233), HLG tone mapping (heygen-com#7234), fetch network cause (heygen-com#7236), .hyperframesignore in lint (heygen-com#7239), info --media (heygen-com#7235), lint priority (heygen-com#7244), check --budget (heygen-com#7245) and the unknown-command answer (heygen-com#7247). bun.lock gains only the lint `ignore` edge from heygen-com#7239, already resolved. Watermark advances to 0a6746e. Co-authored-by: Jerrai <[email protected]> Co-authored-by: Miguel Ángel <[email protected]> Co-authored-by: WaterrrForever <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
hyperframes info --media --jsonaddsheard: every clip the render mixes into the film, scenes included, on the film's clock. Each entry hassrc,start,end,mediaStart,playbackRate,volumeandtype. Text mode prints one "Heard" line per clip. Without--media,infois unchanged.Why
A tool that needs to know which clips the viewer hears (captions, for one) had to walk the HTML itself, and a second walker drifts from the render. Two shapes it got wrong:
data-duration, which runs until its media ends.The render already knows the answer. This exposes it from the CLI, so callers read the same list the mix uses instead of rebuilding it.
How
heardClips(projectDir)inutils/projectMedia.ts(the filecheckalready uses to load the producer) bundles the project with core'sbundleToSingleHtml(scenes inlined, nestedsrcrebased) and runs the producer'scollectRenderMediaover it: the same parser the render uses, so the same rule for what is audible (<audio>, and a<video data-has-audio="true">that is not muted) and the same clock.The collector ends an unbounded clip at its scene's end. The mix stops earlier if the media runs out, so each clip's end becomes
min(end, start + (mediaSeconds - mediaStart) / rate), with the length probed by the engine'sextractAudioMetadata. A clip whose media start is past its file's end is left out. Each file is probed once. A looping clip, a ramped rate or a file that cannot be probed keeps the collector's end.Test plan
projectMedia.test.ts, four tests on a real bundled project with a fixed probe:projectMedia.test.ts"ends a speaking clip with no set length where its media runs out, not where its scene ends": 12 s to 17 s, not to the scene's end at 30 s. It fails with the probe cap removed.projectMedia.test.ts"leaves out a clip that starts past its media's end, and reads each file's length once": a clip whose media start is past its file's end adds nothing to the mix, so it is not listed, and two clips of one file probe it once. It fails with either change taken out.info.test.tspassesmedia: falsein its args. The touched files' tests (17) pass three runs in a row.info --media --jsonprints only JSON, withheardholding the clip at 10 s to 15 s and the one at 22 s to 27 s;info --jsonhas noheardkey; text mode prints both "Heard" lines.infosection ofdocs/packages/cli.mdx.HeardClipand one onheardClips.Size
About 55 lines of source in two files, about 85 of tests and three doc lines. One change:
infocan print the render's audible clips.