Repository navigation
perf(video): composite on an opaque canvas so frames carry no mask - #1779
Merged
Merged
Conversation
CompositeVideoClip treats a composite as transparent unless it is given bg_color or use_bgclip. Every frame then starts from a transparent RGBA canvas, the full-size first clip is alpha-composited onto it, and a second, full-size float mask composite is computed. The writer stacks that mask onto the frame and pipes RGBA to FFmpeg, where the H.264 output drops the alpha channel again. Four composites have an opaque clip covering the whole canvas, so the mask is always fully opaque and only costs time: - generate_video(): the combined footage under the subtitle clips, on every video with subtitles enabled - _fit_clip_to_canvas() in contain mode: the black letterbox canvas - slidein_transition() and slideout_transition(): the black canvas Give them bg_color=(0, 0, 0), as render_image_zoom_video() already does. The rendered frames are unchanged; the final MP4 files were byte-identical before and after on the benchmark inputs.
Owner
|
Merged, thanks! We compared real renders locally with regular subtitles, rounded translucent backgrounds, Arabic subtitles, and subtitles disabled. The output files were byte-identical, and the subtitle renders were noticeably faster. Contain mode and slide transitions checked out too. Nice improvement to the rendering path. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The final render (
generate_video) buildsCompositeVideoClip([video_clip, *text_clips])with nobg_color. MoviePy then treats the whole video as transparent, so for every frame it:compose_mask) for the same frame,The footage covers the whole canvas, so the mask is always fully opaque and only costs time. This is the stage every video with subtitles goes through, and render speed comes up in #795 and #839.
The same pattern is in the contain-mode letterbox (
_fit_clip_to_canvas) and inslidein_transition/slideout_transition: an opaque blackColorClipcovers the canvas, but the composite still carries a mask into every per-clip encode.Change
Pass
bg_color=(0, 0, 0)to those four composites, the same wayrender_image_zoom_videoalready does. With a background color MoviePy keeps the composite opaque: no mask clip, RGB frames, RGB pipe. Clip order, duration and audio are computed exactly as before, and anything not covered by a clip is still black. The subtitle-box composites are left alone, since they need transparency.Benchmark
generate_videoon a 19 s 1080x1920 video with 10 Chinese subtitle lines, the default font and style, no BGM,n_threads=2. Windows 11, i7-9700, Python 3.11, moviepy 2.2.1, FFmpeg 7.1 from imageio-ffmpeg.mainand this branch ran alternately in separate processes, 3 rounds each, at below-normal priority. The inputs are a synthetic FFmpeg test pattern (testsrc2) and a sine tone, so the encode cost is not representative of real footage, but the compositing work per frame is the same.maingenerate_video, median (min-max)All six output MP4s had the same MD5, so the rendered video is unchanged.
Tests
test_generate_video_composites_subtitles_on_an_opaque_canvas: the clip handed to the writer has no mask, keeps its duration, shows the subtitle while it is on screen and the plain footage afterwards.test/services/test_video.pyandtest/services/test_video_effects.py: 84 passed with this branch; with only the source files reverted, 2 tests and 10 subtests fail.ruff checkon the changed files passes.