feat(puffin): Add Puffin crate and CompressionCodec#745
Conversation
fqaiser94
force-pushed
the
puffin_crate_and_compression
branch
from
8f1ae48 to
3eb4fc5
Compare
fqaiser94
force-pushed
the
puffin_crate_and_compression
branch
from
3eb4fc5 to
336df2b
Compare
There was a problem hiding this comment.
This makes sense to me :) Thanks for working on this @fqaiser94 🙌
Xuanwo
left a comment
Xuanwo
left a comment
There was a problem hiding this comment.
Thank you @fqaiser94 for working on this, and also thank you @Fokko for the review. Looks like a good start point for me. Let's move!
liurenjie1024
left a comment
liurenjie1024
left a comment
There was a problem hiding this comment.
Thanks @fqaiser94 for this pr, just left a comment, others LGTM!
There was a problem hiding this comment.
Do we really need to put this outside of iceberg crate? I'm thinking how will we deal with PuffinReader/PuffindWriter, which will depend on FileIO?
There was a problem hiding this comment.
I'm thinking how will we deal with PuffinReader/PuffindWriter, which will depend on FileIO?
I don't understand why this would cause issues since FileIO is a publicly exposed interface? 🤔
Do we really need to put this outside of iceberg crate?
That said, I think you're right.
I don't see any compelling reasons to separate this functionality into a separate crate outside of the iceberg crate.
I didn't consider this option; I just forgot 😅
Here's a quick follow-up PR to move this functionality inside of the existing iceberg crate: #789
4 participants
Part of #744
Summary
CompressionCodec(see iceberg/puffin-spec/compression-codecs for more information)Context
Out of Scope