[lib] Add ZSTD_d_stableOutBuffer + fix single-pass mode for empty frames#2094
[lib] Add ZSTD_d_stableOutBuffer + fix single-pass mode for empty frames#2094terrelln merged 6 commits intofacebook:devfrom
Conversation
|
Looks good to me. I'm just curious how many users would be interested by this new capability ? |
Yes, I expect this to be a reasonably common scenario, as far as streaming goes. We could use this in Managed Compression and Most of the time the IOBuf's aren't chained, but when they are we either have to copy the input into a single buffer (Managed Compression does this), or use streaming (folly does this). Both would be served by this API. Managed Compression would save an allocation, and might save copies if there are blocks that are entirely contained in one buffer. folly would save the allocation of the output buffer, and the extra memcpy. |
|
Fixed a memory leak in zstreamtest |
3 participants
This flag allows users to skip the output buffer allocation + memcpy if they guarantee that the
ZSTD_outBuffer*passed toZSTD_decompressStream()is large enough to fit the entire frame, and is never modified by the user.Also fixes a bug where empty frames weren't eligible for single-pass mode. It caused a small allocation (4 bytes), and is a little bit slower.
Adds tests for single-pass and stable output buffer modes.
Fuzz tests
ZSTD_d_stableOutputto ensure it never crashes.Fixes #2093