Repository navigation
@remotion/serverless: Don't merge a chunk twice when a streamed chunk is retried - #12337
Conversation
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
- Dedup
fileson retried streaming chunks —stream-renderer.ts:124-126andstream-renderer.ts:147-149now guardfiles.push(filename)with!files.includes(filename), so a retried chunk that re-sends its audio/video payload off the response stream is only merged once. Filenames are derived fromoutdir+ the padded chunk index (chunkFilename), so the key is stable across attempts and unique per chunk. - Regression test —
streaming-renderer-launcher.test.ts:147reproduces the exact scenario (audio chunk lands on attempt 1, thensocket hang up), retries, and assertsfilesequals['…:audio', '…:video']. The test fails without the source change ([audio, audio, video]), so it genuinely covers the bug.
Both retry layers are covered by the single guard: the outer renderRendererFunctionWithRetry reuses files, and the inner callFunctionWithStreamingImplementation retry (lambda-client/src/call-lambda-streaming.ts:244) re-invokes through the same receivedStreamingPayload closure, which mutates the same files array. The rewrite-on-disk behavior (either writeFileSync overwrite or the getBinaryPayloadSink createWriteStream truncating) means the dropped duplicate stays correct.
The s3Renderer path is unaffected — it only pushes at status.state === 'completed' immediately before returning success, which ends the retry loop, so a chunk can never be pushed twice there.
The author's noted follow-up (a stream drop after chunk-complete retrying a finished chunk and double-counting its timing via addChunkCompleted) is pre-existing and out of scope for this change.
deepseek-v4.1-flash ｜ 𝕏
|
Thanks! |
When a renderer chunk is retried over response streaming, the retry sends its audio and video chunks again, and each name gets pushed into the same
fileslist. If the first attempt got its audio chunk through before failing (say a socket hang up while the video chunk is still streaming),filesends up with that audio file twice.concatVideos()passes the list straight to the ffmpeg concat step, so the merged audio contains that chunk twice and everything after it is out of sync. Same for the video file if the stream drops after both were sent.Both retry layers hit this:
renderRendererFunctionWithRetry()reusesfiles, andcallFunctionWithStreamingImplementation()retries with the same message handler.Chunk file names are fixed per chunk, so the retry already overwrites the same file on disk. The fix just skips a name that's already in the list.
Added a test next to the artifact replay one: the first attempt sends the audio chunk and fails with a socket hang up, the second sends everything, and
fileshas each chunk once.Related but not in this PR: if the stream drops after
chunk-complete, the chunk gets retried even though it finished, and its timing is counted twice in the cost estimate. Happy to send that separately.