Skip to content

Update tar_ext::BuilderExt#5112

Merged
xzfc merged 4 commits intodevfrom
direct-tar-builder-update
Oct 4, 2024
Merged

Update tar_ext::BuilderExt#5112
xzfc merged 4 commits intodevfrom
direct-tar-builder-update

Conversation

@xzfc
Copy link
Copy Markdown
Member

@xzfc xzfc commented Sep 20, 2024

This PR updates tar_ext::BuilderExt with the following changes:

  1. Generalize: instead of taking std::fs::File, take impl Write + Seek. Later this requirement will be reduced further to impl Write to support both File and an async stream (SyncIoBridge).
  2. Add a workaround for drop: tar::Builder<W>::drop might panic if the underlying writer panics when using in async context. See a comment for BuilderBox and newly added unit tests for more details on the issue.

Comment thread lib/common/common/src/tar_ext.rs Outdated

impl Write for WriteBox {
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
if !self.enabled.load(Ordering::Relaxed) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we need release/acquire here to prevent reordering incorrectly?

Comment thread lib/common/common/src/tar_ext.rs Outdated

impl Write for WriteBox {
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
if !self.enabled.load(Ordering::Relaxed) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand it correctly: the tar impl will attempt to write the final 1K zero bytes. But this write will be rejected here if we unset the enabled flag, preventing the panic?

If we explicitly finalize ourselves the tar impl doesn't write anything and we don't hit this branch at all (during drop).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's correct.

- Generalize: instead of taking `std::fs::File`, take
  `impl Write + Seek`.
- Add a workaround for drop: `tar::Builder<W>::drop` might panic if the
  underlying writer panics when using in async context.
@xzfc xzfc force-pushed the direct-tar-builder-update branch from 10e4768 to e80a392 Compare September 26, 2024 09:59
@xzfc xzfc requested a review from timvisee October 1, 2024 21:40
@xzfc xzfc requested a review from generall October 3, 2024 18:46
@xzfc xzfc merged commit 7932286 into dev Oct 4, 2024
@xzfc xzfc deleted the direct-tar-builder-update branch October 4, 2024 14:10
@generall generall added this to the Stream Snapshots milestone Oct 8, 2024
generall pushed a commit that referenced this pull request Oct 8, 2024
* tar_ext::BuilderExt: generalize and add drop workaround

- Generalize: instead of taking `std::fs::File`, take
  `impl Write + Seek`.
- Add a workaround for drop: `tar::Builder<W>::drop` might panic if the
  underlying writer panics when using in async context.

* Drop unnecessary write_fn usage

* tar_ext::BuilderExt: support both seekable and streaming variants

* tar_ext::BuilderExt: support both owned/borrowed variants
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants