Skip to content

stitchFramesToVideo: -shortest cuts the video when the audio ends before the last frame #19

Open

Description

@elwizard33

When the audio in a composition ends before the last frame (a silent outro, or narration that finishes a bit early), the rendered MP4 is cut to the length of the audio and the remaining frames are lost. In our project a 54.4 s composition came out at 52.1 s.

Small repro, with a.mp3 being any 2 second clip (I used ffmpeg -f lavfi -i "sine=frequency=440:duration=2:sample_rate=24000" -ac 1 a.mp3):

const SilentOutro = () => (
  <>
    <Fill style={{ background: '#123' }} />
    <Sequence from={0} durationInFrames={60}>
      <Audio src={staticFile('a.mp3')} />
    </Sequence>
  </>
);
// <Composition id="SilentOutro" component={SilentOutro} durationInFrames={150} fps={30} width={320} height={180} />

Rendering it with renderFrames() and stitchFramesToVideo() on master (6d6aca2, Windows 11, Node 24) gives a 2 s file instead of 5 s:

$ ffprobe -v error -show_entries stream=codec_type,duration,sample_rate -of compact=p=0 out/SilentOutro.mp4
codec_type=video|duration=2.000000
codec_type=audio|sample_rate=24000|duration=2.000000

It comes from the -shortest flag in stitch-frames-to-video.ts. It's there so audio can't make the video longer, but it also stops the output as soon as the audio ends.

You can also see the audio stays at 24 kHz, the rate of the source MP3s (TTS output is often 24 kHz), even with -b:a 192k.

What worked for me was padding the mix with apad and limiting the output with -t <frames / fps> instead of -shortest, plus -ar 48000 on the audio encoder. With that the same repro gives 5 s of video and 5 s of 48 kHz audio. I've put it in #26 in case it's useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions