Repository navigation
fix: bound the streaming mel buffer to a sliding window (fixes #63, the streaming RSS leak) - #69
Merged
mudler merged 1 commit intoSep 26, 2026
Conversation
, the streaming RSS leak) Root cause: parakeet_capi_stream_feed's streaming session keeps every mel frame from stream_begin onward in mel_buf and rebuilds that whole, ever-growing array on every feed call, even though the decoder (feed_available's window()) never reads a frame older than mel_buffer_idx - pre_encode_cache_size() frames back — so those old frames are provably dead the moment the decoder consumes past them. Because each rebuild requests a uniquely, ever-larger allocation for the life of the stream, the allocator can never reuse an earlier call's freed (now permanently too-small) block, so process RSS grows with the cumulative history of buffer sizes rather than the small, bounded set of frames actually still needed — which is why neither stream_free nor a fresh stream_begin reclaims it. The fix: track the absolute frame index of mel_buf's first column (mel_buf_origin) and, on every append, drop the prefix strictly before mel_buffer_idx - pre_encode_cache_size() — the exact bound window() already relies on — so mel_buf stays bounded to O(pre_encode_cache_size + chunk_size) instead of O(stream length). No API change, no behavioural change: window()'s absolute-index reads are translated into mel_buf's origin-relative local columns, and the surviving frames are carried forward byte-for-byte. Measurement (CPU + Accelerate build, realtime_eou_120m-v1 q8, 120 s of silence fed in 20 ms chunks, RSS sampled at 30/60/90/120 s of audio fed): growth over the last 60 s drops from 178 MB to 0.5 MB; the same run's wall time drops from ~14 minutes to ~13 seconds, since the unbounded rebuild was O(T) per call, O(T²) total. A LibriSpeech utterance's transcript is byte-identical before and after.
mudler
approved these changes
Sep 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #63.
Root cause (two sentences):
parakeet_capi_stream_feed's streaming session keeps every mel frame fromstream_beginonward inmel_bufand rebuilds that whole, ever-growing array on every feed call, even though the decoder (feed_available'swindow()) never reads a frame older thanmel_buffer_idx - pre_encode_cache_size()frames back — so those old frames are provably dead the moment the decoder consumes past them. Because each rebuild requests a uniquely, ever-larger allocation for the life of the stream, the allocator can never reuse an earlier call's freed (now permanently too-small) block, so process RSS grows with the cumulative history of buffer sizes rather than the small, bounded set of frames actually still needed — explaining why neitherstream_freenor a freshstream_beginreclaims it.The fix: track the absolute frame index of
mel_buf's first column (mel_buf_origin) and, on every append, drop the prefix strictly beforemel_buffer_idx - pre_encode_cache_size()— the exact boundwindow()already relies on — somel_bufstays bounded toO(pre_encode_cache_size + chunk_size)instead ofO(stream length). No API change, no behavioural change:window()'s absolute-index reads are translated intomel_buf's new origin-relative local columns, and the surviving frames are carried forward byte-for-byte. This is the same bounded-sliding-window discipline the per-layer conv/attention caches inStreamingEncoderalready follow; the mel buffer was the one accumulator that did not.Measurement (CPU + Accelerate build,
realtime_eou_120m-v1q8, 120 s of silence fed in 20 ms chunks, RSS sampled at 30/60/90/120 s of audio fed): growth over the last 60 s drops from 178 MB to 0.5 MB; the same run's wall time drops from ~14 minutes to ~13 seconds, since the unbounded rebuild was O(T) per call, O(T²) total. A LibriSpeech utterance's transcript is byte-identical before and after (sha256d7a620231a849b5a531ab103a8aaac9ade7d088dc05af7a77fc0633d0d797b27).Built and checked on macOS arm64 (CPU,
-DPARAKEET_SHARED=ON -DPARAKEET_GGML_METAL=OFF), zero warnings.