Speed up cache size command by MatthewMckee4 · Pull Request #17015 · astral-sh/uv

MatthewMckee4 · 2025-12-06T14:20:33Z

Summary

uv cache size can be quite slow. Here i use https://github.com/sharkdp/diskus to walk the cache directory with in multiple threads.

Add cli option to set the number of threads and default to std::thread::available_parallelism() or 1.

Test Plan

Added cli statement with info log test.

I believe this is a fair test, where i set cache dir to a large directory.

matthew@matthew-main ~/develop/personal/uv                                                                                                                                                                                                                 [14:17:50]                                                                                                                                                                                                                                       [±cache-size-speed-up ✓▴]
> $ uv cache size --preview-features cache-size -H --cache-dir ~/develop/                                                                                                                                                                   [±cache-size-speed-up ✓▴]
75.7GiB

matthew@matthew-main ~/develop/personal/uv                                                                                                                                                                                                                 [14:18:24]
> $ hyperfine 'uv cache size --preview-features cache-size -H --cache-dir ~/develop/' 'target/debug/uv cache size --preview-features cache-size -H --cache-dir ~/develop/'                                                                  [±cache-size-speed-up ✓▴]
Benchmark 1: uv cache size --preview-features cache-size -H --cache-dir ~/develop/
  Time (mean ± σ):      1.059 s ±  0.014 s    [User: 0.171 s, System: 0.884 s]
  Range (min … max):    1.048 s …  1.097 s    10 runs

Benchmark 2: target/debug/uv cache size --preview-features cache-size -H --cache-dir ~/develop/
  Time (mean ± σ):     413.8 ms ±  17.1 ms    [User: 5789.2 ms, System: 1682.0 ms]
  Range (min … max):   386.3 ms … 441.6 ms    10 runs

Summary
  target/debug/uv cache size --preview-features cache-size -H --cache-dir ~/develop/ ran
    2.56 ± 0.11 times faster than uv cache size --preview-features cache-size -H --cache-dir ~/develop/

zanieb · 2025-12-06T14:22:05Z

Cargo.toml

 zeroize = { version = "1.8.1" }
 zip = { version = "2.2.3", default-features = false, features = ["deflate", "zstd", "bzip2", "lzma", "xz"] }
 zstd = { version = "0.13.3" }
+diskus = { git = "https://github.com/sharkdp/diskus", version = "0.8.0" }


We can't add Git dependencies, it'll break our crates.io publish

Ah, okay, happy for me to copy most of David's code over then?

I'm not sure yet.

@sharkdp are you interested in publishing the crate? Do you think we should just vendor the parts we need?

I think we should just vendor any speed-ups because this is pulling in a bunch of new dependencies (e.g., a second version of Clap).

I updated diskus today (it has always been published on crates.io, so no need for Git dependencies). You should now be able to pull it in as a relatively lightweight dependency using default-features = false. I also made an update specifically for counting apparent file size (which is what uv did here before: sum up metadata.len()), to make that more consistent with what du -sb does (exclude the size of directory entries themselves). I also cleaned up the diskus API a bit. If you want to depend on it here, the code should be something like:

let result = DiskUsage::new(&[cache.root()]).apparent_size().count(); let total_bytes = result.ignore_errors().size_in_bytes();

(or leave out the call to .apparent_size() if you'd rather want to count disk usage).

Note that diskus uses a heuristic for the default number of workers which should ideally not be overwritten for the best possible performance (at least according to benchmarks I did years ago).

I am also completely fine with uv vendoring diskus. It's not a lot of code. In that case, you should potentially make those two updates mentioned above, though.

crates/uv/src/commands/cache_size.rs

crates/uv-cli/src/lib.rs

crates/uv/Cargo.toml

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.9.17` -> `0.9.18` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.9.18`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0918) [Compare Source](astral-sh/uv@0.9.17...0.9.18) Released on 2025-12-16. ##### Enhancements - Add value hints to command line arguments to improve shell completion accuracy ([#17080](astral-sh/uv#17080)) - Improve error handling in `uv publish` ([#17096](astral-sh/uv#17096)) - Improve rendering of multiline error messages ([#17132](astral-sh/uv#17132)) - Support redirects in `uv publish` ([#17130](astral-sh/uv#17130)) - Include Docker images with the alpine version, e.g., `python3.x-alpine3.23` ([#17100](astral-sh/uv#17100)) ##### Configuration - Accept `--torch-backend` in `[tool.uv]` ([#17116](astral-sh/uv#17116)) ##### Performance - Speed up `uv cache size` ([#17015](astral-sh/uv#17015)) - Initialize S3 signer once ([#17092](astral-sh/uv#17092)) ##### Bug fixes - Avoid panics due to reads on failed requests ([#17098](astral-sh/uv#17098)) - Enforce latest-version in `@latest` requests ([#17114](astral-sh/uv#17114)) - Explicitly set `EntryType` for file entries in tar ([#17043](astral-sh/uv#17043)) - Ignore `pyproject.toml` index username in lockfile comparison ([#16995](astral-sh/uv#16995)) - Relax error when using `uv add` with `UV_GIT_LFS` set ([#17127](astral-sh/uv#17127)) - Support file locks on ExFAT on macOS ([#17115](astral-sh/uv#17115)) - Change schema for `exclude-newer` into optional string ([#17121](astral-sh/uv#17121)) ##### Documentation - Drop arm musl caveat from Docker documentation ([#17111](astral-sh/uv#17111)) - Fix version reference in resolver example ([#17085](astral-sh/uv#17085)) - Better documentation for `exclude-newer*` ([#17079](astral-sh/uv#17079)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

MatthewMckee4 added 2 commits December 6, 2025 14:11

Speed up cache size command

9a0f4b6

Update cli reference

d64f136

zanieb reviewed Dec 6, 2025

View reviewed changes

MatthewMckee4 marked this pull request as draft December 6, 2025 14:22

MatthewMckee4 added 3 commits December 6, 2025 14:49

Vendor code from diskus

d90e641

Sort uv crate Cargo.toml

13307d2

Update test comment

968045c

MatthewMckee4 marked this pull request as ready for review December 6, 2025 16:30

MatthewMckee4 added 2 commits December 7, 2025 14:10

Use diskus again

2ced216

Remove crossbeamchannel dep

5908572

samypr100 reviewed Dec 8, 2025

View reviewed changes

crates/uv/src/commands/cache_size.rs Outdated Show resolved Hide resolved

samypr100 reviewed Dec 8, 2025

View reviewed changes

crates/uv-cli/src/lib.rs Outdated Show resolved Hide resolved

samypr100 reviewed Dec 8, 2025

View reviewed changes

crates/uv/Cargo.toml Outdated Show resolved Hide resolved

konstin added enhancement New feature or improvement to existing functionality performance Potential performance improvement labels Dec 8, 2025

MatthewMckee4 added 5 commits December 8, 2025 16:06

Sort Cargo.toml dependencies

b51955c

Merge main

8e63679

regenerate reference/cli.md

6fe811d

Remove reference/cli.md

a5f16fc

Remove threads cli option

47a150d

charliermarsh approved these changes Dec 11, 2025

View reviewed changes

konstin approved these changes Dec 11, 2025

View reviewed changes

charliermarsh enabled auto-merge (squash) December 11, 2025 17:07

charliermarsh disabled auto-merge December 11, 2025 17:10

charliermarsh merged commit 6de869c into astral-sh:main Dec 11, 2025
253 of 272 checks passed

BrewTestBot mentioned this pull request Dec 16, 2025

uv 0.9.18 Homebrew/homebrew-core#258895

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Speed up cache size command#17015

Speed up cache size command#17015
charliermarsh merged 12 commits intoastral-sh:mainfrom
MatthewMckee4:cache-size-speed-up

MatthewMckee4 commented Dec 6, 2025 •

edited

Loading

Uh oh!

zanieb Dec 6, 2025

Uh oh!

MatthewMckee4 Dec 6, 2025 •

edited

Loading

Uh oh!

zanieb Dec 6, 2025

Uh oh!

charliermarsh Dec 6, 2025

Uh oh!

sharkdp Dec 7, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Comments

Conversation

MatthewMckee4 commented Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Uh oh!

zanieb Dec 6, 2025

Choose a reason for hiding this comment

Uh oh!

MatthewMckee4 Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zanieb Dec 6, 2025

Choose a reason for hiding this comment

Uh oh!

charliermarsh Dec 6, 2025

Choose a reason for hiding this comment

Uh oh!

sharkdp Dec 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

MatthewMckee4 commented Dec 6, 2025 •

edited

Loading

MatthewMckee4 Dec 6, 2025 •

edited

Loading

sharkdp Dec 7, 2025 •

edited

Loading