Skip to content

Use pools.Copy for archive file copy operations#48605

Merged
thaJeztah merged 1 commit intomoby:masterfrom
akx:48601-pull-opt-2
Oct 11, 2024
Merged

Use pools.Copy for archive file copy operations#48605
thaJeztah merged 1 commit intomoby:masterfrom
akx:48601-pull-opt-2

Conversation

@akx
Copy link
Copy Markdown
Contributor

@akx akx commented Oct 8, 2024

What I did

While investigating #48601, I looked at what io.Copy does, and found that it allocates a fresh buffer for each operation by default. This sounds wasteful e.g. when copying small files (Python files, for one, can often be < 32k in size) from tar streams to disk or vice versa.

How I did it

I found #14268 had introduced pools.Copy, which uses a pool of 32k buffers (the default for io.Copy when you don't pass in a buffer), so this switches the use of io.Copy to that implementation.

How to verify it

Using the same Hyperfining as in the sibling PR #48602...

Very small improvement, but not a lot of code for it either.

On 367c910 (master):

Benchmark 1: docker rmi -f quay.io/singularity/singularity:v4.1.0-arm64 && docker system prune -f && docker pull quay.io/singularity/singularity:v4.1.0-arm64
  Time (mean ± σ):     39.036 s ±  0.775 s    [User: 0.051 s, System: 0.047 s]
  Range (min … max):   37.866 s … 40.025 s    10 runs

On 367125e (this HEAD):

Benchmark 1: docker rmi -f quay.io/singularity/singularity:v4.1.0-arm64 && docker system prune -f && docker pull quay.io/singularity/singularity:v4.1.0-arm64
  Time (mean ± σ):     38.651 s ±  0.851 s    [User: 0.053 s, System: 0.044 s]
  Range (min … max):   37.266 s … 40.369 s    10 runs

Description for the changelog

Use buffer pool when copying from/to tar

A picture of a cute animal (not mandatory but encouraged)

Some sort of cat, drawn by me just now.
catsnek

@thaJeztah thaJeztah added area/distribution Image Distribution status/2-code-review area/performance kind/refactor PR's that refactor, or clean-up code labels Oct 11, 2024
@thaJeztah thaJeztah added this to the 28.0.0 milestone Oct 11, 2024
Copy link
Copy Markdown
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thaJeztah thaJeztah merged commit 8cc4029 into moby:master Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/distribution Image Distribution area/performance kind/refactor PR's that refactor, or clean-up code status/2-code-review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants