Skip to content

fix(engine): exports carry full colour tags with every encoder, SDR and HDR - #5742

Merged
miguel-heygen merged 3 commits into
mainfrom
fix/sdr-tag-frames-bt709
Oct 18, 2026
Merged

miguel-heygen merged 3 commits into
mainfrom
fix/sdr-tag-frames-bt709

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator

What changes for users

MP4/MOV exports encoded with VideoToolbox (macOS --gpu, or a macOS FFmpeg without libx264) now carry their full colour description:

  • SDR: before, tagged color_space=bt709 but color_transfer and color_primaries unknown, so players guessed and could show a colour cast. Now bt709,bt709,bt709.
  • HDR: before, tagged bt2020nc but transfer and primaries unknown, so players treated the export as SDR. Now bt2020nc, smpte2084 (PQ) or arib-std-b67 (HLG), bt2020.

libx264 and libx265 exports were already fully tagged and are unchanged.

Root cause

The encoders set colour two ways: on the frames through the conversion filters, and through -color_primaries / -color_trc output options (plus x264/x265 params). The SDR conversion only set the frames' matrix (from scale=out_color_matrix=bt709), and the HDR paths set nothing on the frames. VideoToolbox ignores the output options and writes only what the frames carry. With FFmpeg 8.1.1 on macOS:

  • h264_videotoolbox given all four -color_* options writes bt709,unknown,unknown;
  • hevc_videotoolbox with bt2020/PQ options writes bt2020nc,unknown,unknown.

The fix tags the frames on every path, streamed and disk (so HLS and distributed chunks too):

  • SDR: both SDR filters (capture and raw RGB) now end in SDR_RGB_TO_TAGGED_BT709_FILTER, the BT.709 conversion plus setparams tags that ProRes already uses for the same reason.
  • HDR: every HDR branch, including the GPU branch of the disk encoder and the raw-HDR stream that had no filter before, now appends hdrFrameTagsFilter(transfer). It's built from getHdrEncoderColorParams, so the frame tags and the x265 params share one source.
  • The -color_* options stay; they still set the container tags.

Verification

  • 0.8.167 on linux/x64 with libx264: default, one-worker, low-memory, segmented, HLS and HDR-source --sdr renders are all bt709,bt709,bt709, range tv. An HDR render keeps bt2020nc/smpte2084/bt2020. So the report only reproduces with VideoToolbox.
  • macOS, FFmpeg 8.1.1, real VideoToolbox through the new filters:
    • h264_videotoolbox: bt709,bt709,bt709 (was bt709,unknown,unknown);
    • hevc_videotoolbox main10: bt2020nc,smpte2084,bt2020 and bt2020nc,arib-std-b67,bt2020 (was bt2020nc,unknown,unknown).
  • New tests run real ffmpeg and ffprobe. Each encodes through the -vf the arg builder produces, with a plain libx264 or 10-bit libx265 given no colour options, which like VideoToolbox writes only the frames' colour, then ffprobes the stream.
    • chunkEncoder.test.ts: SDR; HDR PQ and HLG; HDR PQ on the VideoToolbox branch.
    • streamingEncoder.test.ts: SDR captured and raw RGB frames; raw HDR PQ and HLG; raw HDR PQ on VideoToolbox.
    • On main all 9 fail. With the change, chunkEncoder (103) and streamingEncoder (84) pass, 3 runs each.
  • One existing expectation updated: the GPU HDR odd-dimension test now expects the HDR tags ahead of the pad.
  • tsc --noEmit in packages/engine and oxfmt --check are clean.

…ry encoder

The SDR conversion filter set the BT.709 matrix on its frames but not the
transfer or primaries. Encoders that take colour from the frames rather than
from -color_* options (VideoToolbox on macOS) wrote color_space=bt709 with
color_transfer and color_primaries unknown. The capture and raw-input SDR
filters now end in the tagged BT.709 filter ProRes already uses.
VideoToolbox HDR encodes had the same gap as SDR: given only -color_* options
it wrote bt2020nc with transfer and primaries unknown, so players read the
export as SDR. Every HDR branch of the streamed and disk encoders now tags the
frames through one hdrFrameTagsFilter built from the encoder colour params.
@miguel-heygen miguel-heygen changed the title fix(engine): SDR exports carry BT.709 transfer and primaries with every encoder fix(engine): exports carry full colour tags with every encoder, SDR and HDR Oct 17, 2026
@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 168 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

@miguel-heygen
miguel-heygen force-pushed the fix/sdr-tag-frames-bt709 branch from 33a20ef to bb9759f Compare October 17, 2026 02:31

@terencecho terencecho left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed bb9759fb95300d22aa0b4d89ea06a660d6433e06 against effective base 4e6ee6d2300290b638042ec88ef5afb7a0c27408; no concrete introduced blocker found.

  • packages/engine/src/utils/hdr.ts:109–112 derives frame tags from the existing encoder color parameters, keeping PQ/HLG transfer names aligned rather than restating them.
  • packages/engine/src/services/streamingEncoder.ts:428–447 preserves the captured-SDR/raw-SDR/raw-HDR distinction and applies padding/upload after selecting the appropriate frame filter.

Audited: all six changed files end-to-end at both BASE and HEAD, relevant real encoder/compositor/capture callers, preserved codec branches and all 15 behavioral-group pairs; two independent source-only passes completed. The final test-only successor's actual patches were reconciled too.

Verification limits: the new probes isolate -vf using lavfi and software encoders; a VideoToolbox-selected argument-builder row is not an actual VideoToolbox execution. Tests/builds/native codecs/CI were not run or queried. Pixel/range fidelity, hardware metadata propagation and final playback remain unverified; the reported native results are author-supplied evidence, not my reproduction.

— Review by tai (pr-review)

Verdict: APPROVE
Reasoning: The source changes consistently add frame tags while preserving the distinct input conversion paths and existing output metadata. I found no reachable introduced defect in the audited scope; this approval does not claim native validation.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 18, 2026
Merged via the queue into main with commit ea742c5 Oct 18, 2026
75 checks passed
@miguel-heygen
miguel-heygen deleted the fix/sdr-tag-frames-bt709 branch October 18, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants