-
Notifications
You must be signed in to change notification settings - Fork 814
Switch from seek-bzip to unbzip2-stream #1887
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
src/packet/compressed_data.js
Outdated
| data = await streamReadToEnd(data); | ||
| } | ||
| data = toStream(data); | ||
| return data.pipeThrough(new Unbzip2Stream()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is more of a comment on the stream tools API, but I think the interaction between the helpers is confusing; I think we should have ArrayStream support pipeThrough; or alternatively add a (required) arg to toStream like convertArrayStream so that it takes care of it internally.
In any case the toStream function docs need updating, since it's advertise to always return a ReadableStream
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't want to use ArrayStreams here because in that case all of the data gets decompressed regardless of what happens after it, since ArrayStreams don't have backpressure support. Since we want to limit the output size of the stream afterwards (in a future PR), we want to use a real stream here.
You're right that the docs need updating, though.
2093378 to
6da27bd
Compare
009f50d to
dfb68d9
Compare
dfb68d9 to
6d99111
Compare
To allow streaming decompression of bzip2 data.
6d99111 to
6f570a0
Compare
To allow streaming decompression of bzip2 data.