Skip to content

perf(video): composite on an opaque canvas so frames carry no mask - #1779

Merged
harry0703 merged 1 commit into
harry0703:mainfrom
b84330808:perf-opaque-video-composite
Nov 7, 2026
Merged

harry0703 merged 1 commit into
harry0703:mainfrom
b84330808:perf-opaque-video-composite

Conversation

@b84330808

Copy link
Copy Markdown
Contributor

Problem

The final render (generate_video) builds CompositeVideoClip([video_clip, *text_clips]) with no bg_color. MoviePy then treats the whole video as transparent, so for every frame it:

  • starts from a transparent RGBA canvas and alpha-composites the full-size footage onto it,
  • computes a second full-size float mask composite (compose_mask) for the same frame,
  • stacks that mask onto the frame and pipes RGBA instead of RGB to FFmpeg, where the H.264 output drops the alpha again.

The footage covers the whole canvas, so the mask is always fully opaque and only costs time. This is the stage every video with subtitles goes through, and render speed comes up in #795 and #839.

The same pattern is in the contain-mode letterbox (_fit_clip_to_canvas) and in slidein_transition / slideout_transition: an opaque black ColorClip covers the canvas, but the composite still carries a mask into every per-clip encode.

Change

Pass bg_color=(0, 0, 0) to those four composites, the same way render_image_zoom_video already does. With a background color MoviePy keeps the composite opaque: no mask clip, RGB frames, RGB pipe. Clip order, duration and audio are computed exactly as before, and anything not covered by a clip is still black. The subtitle-box composites are left alone, since they need transparency.

Benchmark

generate_video on a 19 s 1080x1920 video with 10 Chinese subtitle lines, the default font and style, no BGM, n_threads=2. Windows 11, i7-9700, Python 3.11, moviepy 2.2.1, FFmpeg 7.1 from imageio-ffmpeg. main and this branch ran alternately in separate processes, 3 rounds each, at below-normal priority. The inputs are a synthetic FFmpeg test pattern (testsrc2) and a sine tone, so the encode cost is not representative of real footage, but the compositing work per frame is the same.

main this branch
generate_video, median (min-max) 59.7 s (59.4-60.1) 24.9 s (24.86-24.92)

All six output MP4s had the same MD5, so the rendered video is unchanged.

Tests

  • New test_generate_video_composites_subtitles_on_an_opaque_canvas: the clip handed to the writer has no mask, keeps its duration, shows the subtitle while it is on screen and the plain footage afterwards.
  • The contain-mode and SlideIn/SlideOut tests now also assert that the result has no mask.
  • test/services/test_video.py and test/services/test_video_effects.py: 84 passed with this branch; with only the source files reverted, 2 tests and 10 subtests fail.
  • ruff check on the changed files passes.

CompositeVideoClip treats a composite as transparent unless it is given
bg_color or use_bgclip. Every frame then starts from a transparent RGBA
canvas, the full-size first clip is alpha-composited onto it, and a
second, full-size float mask composite is computed. The writer stacks
that mask onto the frame and pipes RGBA to FFmpeg, where the H.264 output
drops the alpha channel again.

Four composites have an opaque clip covering the whole canvas, so the
mask is always fully opaque and only costs time:

- generate_video(): the combined footage under the subtitle clips, on
  every video with subtitles enabled
- _fit_clip_to_canvas() in contain mode: the black letterbox canvas
- slidein_transition() and slideout_transition(): the black canvas

Give them bg_color=(0, 0, 0), as render_image_zoom_video() already does.
The rendered frames are unchanged; the final MP4 files were
byte-identical before and after on the benchmark inputs.
@harry0703
harry0703 merged commit 66db0e5 into harry0703:main Nov 7, 2026
3 checks passed
@harry0703

Copy link
Copy Markdown
Owner

Merged, thanks! We compared real renders locally with regular subtitles, rounded translucent backgrounds, Arabic subtitles, and subtitles disabled. The output files were byte-identical, and the subtitle renders were noticeably faster. Contain mode and slide transitions checked out too. Nice improvement to the rendering path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants