Skip to content

fix(core): media-treatment --analyze finishes on long HDR clips - #5792

Closed
miguel-heygen wants to merge 3 commits into
mainfrom
fix/media-treatment-analyze-long-hdr
Closed

miguel-heygen wants to merge 3 commits into
mainfrom
fix/media-treatment-analyze-long-hdr

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator

What changes for users

hyperframes media-treatment --analyze now finishes on several-minute HDR clips instead of failing with spawnSync ffmpeg ETIMEDOUT, and its grade reflects the whole clip rather than only the first 40 seconds.

Root cause

analyzeMediaGrade (packages/core/src/mediaGradeAnalyzer.ts, and its vendored copy packages/cli/src/media-use/lib/grade-analyzer.mjs) ran one ffmpeg pass from the start of the file with an fps filter set to 5 / duration, floored at 0.1 fps. For any clip longer than 50 s the floor wins, so ffmpeg decoded every frame of the first ~40 s, at full resolution and frame rate, to emit its five samples. The cost grows with resolution × frame rate, and a 4K60 10-bit HEVC phone clip goes past the fixed 15 s timeout. The same floor meant the five samples all came from the first 40 s.

Fix

Each of the five samples seeks (-noaccurate_seek -ss) to the middle of one fifth of the clip and decodes a single frame from the keyframe there. Decode work no longer depends on clip length, resolution only scales the five decodes. The 15 s timeout (still overridable with HYPERFRAMES_ANALYZE_TIMEOUT_MS) now applies per sample, and on expiry the error names the sample time and the variable. Images and media with no probed duration keep the old single pass. trim=end_frame=N heads the filter chain: on ffmpeg 8, -frames:v N still lets one extra frame through the filter, so each sample counted twice (main's image path had the same leak: an animated GIF reported 6 frames).

-noaccurate_seek over an accurate seek: an accurate seek decodes from the keyframe up to the target, which on a long-GOP file (a screen recording with one keyframe) brings back a decode that scales with clip length. The sample lands on the keyframe at or before the target instead, within a GOP of it on camera footage.

Before / after

Same machine (8-core Linux, shared and under load), same generated clips, wall time of analyzeMediaGrade:

Clip (generated, 10-bit HEVC, HLG BT.2020) main this PR
1080x1920, 30 fps, 240 s 3.3-3.7 s 0.73-0.75 s (3 runs)
2160x3840, 60 fps, 180 s fails at 15.6 s: spawnSync ffmpeg ETIMEDOUT (16.1 s with no timeout) 2.08-2.13 s (3 runs)

The measured stats agree to the decimal (yAvg 126.4 vs 126.4 on the 4K clip). The table times the media-use copy; the core module behind the command measured the same way: 4K60 clip main ETIMEDOUT at 17.5 s vs 2.4 s, 240 s 1080x1920 clip 5.7 s vs 0.8 s. Generated clips decode faster than camera footage; an independent rerun on a 60 s 4K60 synthetic clip measured 8.4 s on main vs 2.1 s here.

Known limits, deliberate: the per-sample timeout means a stuck run fails after up to 5 x 15 s instead of 15 s, with the sample named. On a file with one keyframe (some screen recordings) every sample lands on that keyframe, so the grade comes from fewer distinct frames; it still finishes.

Tests

  • packages/core/src/mediaGradeAnalyzer.test.ts (new, real ffmpeg, runs in the runtime test job): a two-minute 128x128 10-bit HLG HEVC clip, black then white. On main it fails with expected 16 to be greater than 200 (all samples from the black first 40 s); on this branch the samples cover both halves. A second test sets the timeout to 1 ms and checks the error message.
  • The media-use copy measures the same clip identically (runs in CI with ffmpeg; replacing its seek with a single pass turns it red), and the parity test checks both copies seek to the same times.
  • 20/20 pass 3 runs in a row on ffmpeg 8.0.1 (the build CI pins) and once on 7.0.2; media-treatment.test.ts (34) and grade-analyzer.test.mjs (10) pass.

Pre-review pass (independent reviewer)

Blocker new test red on ffmpeg 8 (expected 10 to be 5): extra frame per sample. Fixed in 5d5ef88, verified on 8.0.1 and 7.0.2
Minor single-keyframe files sample one keyframe; per-sample timeout ceiling; image-path timeout wording (fixed); media-use copy had no ffmpeg test in CI (added)
Audited all three callers of both copies; 13 odd inputs (0.04 s clip, MKV with audio longer than video, MPEG-TS start offset, VFR, animated GIF/WebP, audio-only, truncated file, path starting with -): none worse than main, the 0.04 s clip and the long-audio MKV now work where main errored; six code mutations
Not exercised Windows locally (CI covers it), real camera footage

Grade analysis decoded the clip from the start and kept every frame until the
fps filter had emitted five samples. Past 50 s the filter rate floors at 0.1
fps, so it always decoded the first 40 s at full rate and resolution: a
3-minute 4K60 10-bit HLG clip timed out at 15 s, and every long clip was
graded from its first 40 s only.

Each sample now seeks to the middle of one fifth of the clip and decodes one
frame. The timeout applies per sample and its error names the sample time and
HYPERFRAMES_ANALYZE_TIMEOUT_MS.
@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 198 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant