Repository navigation
fix(producer): convert HEIC images to JPEG at compile time - #6455
Merged
Merged
Conversation
Tiled HEIC photos probe as several video streams, so the media-type preflight rejected them as image:video, and Chrome cannot decode HEIC. Sniff the ftyp brands of every <img>, join and rotate the tile grid with ffmpeg, verify the declared size, and swap in a cached JPEG.
Contributor
There was a problem hiding this comment.
Verdict: REQUEST CHANGES · 1 blocker · 2 important · 1 nit · head cb913b37
- blocker: the required
Producer: unit testsjob fails (CI run 40368001169).heifImagePrep.test.tsandhtmlCompiler.heif.test.tsspawn ffmpeg/ffprobe, but they are not listed inINTEGRATION_TEST_FILES(packages/producer/scripts/test-classification.mjs). The unit lane does not install FFmpeg, so every conversion case fails with[FFmpeg] ffprobe not found. Add both files to the list. The integration lane installs BtbN 8.0.1, which has-show_stream_groups. Editing that file also means updating its sha256 inscripts/test-reachability.json:50. - important: environment failures blame the photo (
heifImagePrep.ts:38-44). A missing ffprobe, or one older than 7.0, fails here withowner: "user"and the advice "Re-export the photo as JPEG or PNG". Older versions reject-show_stream_groups: ffprobe 5.1 fails withFailed to set value '-show_streams' for option 'show_stream_groups': Option not found. Those are exactly the x86 Lambda zip (ffprobe-static ^3.1.0) and stock Debian/Ubuntu hosts. Re-exporting cannot fix that. Report a tool/version error as a platform failure instead. - important: one failed HEIC drops the whole Studio preview transform (
studioServer.ts:579). The throw rejectstransformPreviewHtml, andstudio-server/src/routes/preview.ts:400then falls back to the untransformed HTML. The preview therefore also loses GIF prep and the deterministic font faces, where today only that<img>is broken. CatchHeifImageConversionErrorin the preview path and leave that src as it is. - nit:
heifImagePrep.ts:145takesstream_groups[0]. FFmpeg marks the primary item's grid withdisposition.default(mov_parse_tiles). If a file lists an auxiliary grid (for example an HDR gain map) first, the wrong image is converted and still passes the size check, because that check uses the same grid's declared size. Select the group whose disposition isdefault.
Contributor
Edit accuracy: accurate 271 (base branch 271), smooth 167 of thoseThe gate passes. Quarantined, measured but not gated (0) |
… image as authored Also: HEIC tests run in the integration lane; the primary grid is picked by default disposition.
Collaborator
Author
|
Fixed in 345220b:
28 tests pass 3 runs in a row; producer typecheck is clean. |
SanHsien
added a commit
to SanHsien/hyperframes
that referenced
this pull request
Oct 26, 2026
Merge 20 more upstream commits (acd0bc3..ad4d3fa, v0.8.205: heygen-com#6403, heygen-com#6363, heygen-com#6209, heygen-com#6395, heygen-com#6424, heygen-com#6426, heygen-com#6442, heygen-com#6430, heygen-com#6402, heygen-com#6435, heygen-com#6439, heygen-com#6449, heygen-com#5436, heygen-com#6455 and docs). No conflicts. Record the 11 open PRs heygen-com#6440-heygen-com#6458 and issue heygen-com#6443 in docs/DECISIONS.md (none adopted), re-check the earlier open PRs, and advance the commit watermark to ad4d3fa, the PR watermark to heygen-com#6458 and the issue watermark to heygen-com#6443. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
HEIC and HEIF images now work in every render path. The compile step converts each one to a JPEG before the page loads and rewrites the
<img src>, so the CLI, Studio preview, Lambda and cloud renders all get it from one place.No visible change
Compositions without HEIC images compile exactly as before. A HEIC image that used to fail the render now shows as a normal picture.
Why
Chrome cannot decode HEIC, and a tiled iPhone photo also reads as several video streams. The media-type check then rejects the
<img>and the render fails at the plan step. On main, with a real tiled HEIC:The media probe reports the file as
visualKind: "moving". A real CLI render on main fails with the same message (exit 1).Related work
Follows the same shape as the animated-GIF preparation (
animatedGifPrep.ts), which is the existing owner of "turn an unsupported<img>into something Chrome can show".How
heifImagePrep.tssits next to the GIF prep and is wired in the same two places:htmlCompiler.ts(renders) and Studio's preview transform.ftypbrands heic, heix, hevc, hevx, heim, heis, mif1, msf1; AVIF is excluded), not its extension. Covered: a.heicfile, a mislabeled.jpg, adata:image/heic;base64source, and a remote source already downloaded by the existing localisation pass.xstackfrom the tile layoutffprobe -show_stream_groupsreports, cropped to the declared size, and rotated by the primary item'sirotbox (heifRotation.ts). The result must equal the declared size (swapped for a quarter turn) or the conversion fails, so a lone tile can never pass.<img>never reaches the parse; one with only non-HEIC images costs a 4 KB header read per local image.HEIF_IMAGE_CONVERSION_FAILED).runFfprobeis now exported from the engine and the source resolver inanimatedGifPrep.tsis shared, so there is one resolver.Design calls: (1) a sibling of the GIF prep, not a new stage: it is the existing owner of this kind of rewrite and its two callers; (2) JPEG output only, since there is no alpha HEIC fixture to prove a PNG path; (3) tiles joined by us, because ffmpeg does not do it.
Test plan
.jpg, remote-style source, dedupe, corrupt file names the file, no-HEIC spawns nothing, compile no longer fails the media-type check); breaking rotation or the tile join fails 3 tests eachNot covered: HEIC with an alpha plane (always JPEG), a single non-tiled HEIC that carries a rotation (relies on ffmpeg's own handling), a mirror (
imir) property, CSSurl()andsrcset. The tile join needs ffprobe 7.0 or newer for-show_stream_groups; the arm64 Lambda image builds ffmpeg 7.0.2 and the GCP and CLI Docker images ship a BtbN 8.0.1 build, all with thehevcdecoder, while the x86 Lambda zip uses the npmffmpeg-staticbinary and is worth checking.