Migrate to `zlib-rs` (again) by charliermarsh · Pull Request #11894 · astral-sh/uv

charliermarsh · 2025-03-02T17:01:08Z

Summary

I believe zlib-rs is now a better choice on ARM and x86, so I'm just going to assume it's a better choice everywhere. It's much easier to build (removes our CMake dependency), and in my benchmarking, it's substantially faster on ARM and faster or ~exactly even on my x86 Windows machine.

We migrated to zlib-rs once before (#9184); however, I later reverted it as I learned that they were only doing compile-time feature detection, and so zlib-rs was meaningfully slower on x86. They now perform runtime feature detection: https://trifectatech.org/blog/zlib-rs-is-faster-than-c/.

To benchmark, I wrote a script to create a local Simple API-compliant registry (see the commit history) for a single package. Then I ran the install-cold benchmark against that registry to install NumPy.

On ARM:

❯ uv run resolver --uv-pip-path ../../zlib-ng --uv-pip-path ../../zlib-rs \
        --benchmark install-cold \
        req.txt --warmup 10 --min-runs 30
Benchmark 1: ../../zlib-ng (install-cold)
  Time (mean ± σ):     165.7 ms ±  34.7 ms    [User: 64.4 ms, System: 93.2 ms]
  Range (min … max):   141.8 ms … 293.2 ms    30 runs

Benchmark 2: ../../zlib-rs (install-cold)
  Time (mean ± σ):     150.9 ms ±  16.2 ms    [User: 57.4 ms, System: 86.4 ms]
  Range (min … max):   135.3 ms … 202.4 ms    30 runs

Summary
  ../../zlib-rs (install-cold) ran
    1.10 ± 0.26 times faster than ../../zlib-ng (install-cold)

I benchmarked this about 100 times on my Windows machine and found it difficult to conclude anything beyond "They're nearly the same". Here's an example:

PS C:\Users\crmar\workspace\puffin> hyperfine --prepare "uv venv" "zlib-rs.exe pip sync ./scripts/benchmark/req.txt" "zlib-ng.exe pip sync ./scripts/benchmark/req.txt" "zlib-rs.exe pip sync ./scripts/benchmark/req.txt" "zlib-ng.exe pip sync ./scripts/benchmark/req.txt" --runs 10 --warmup 5
Benchmark 1: zlib-rs.exe pip sync ./scripts/benchmark/req.txt
  Time (mean ± σ):     240.6 ms ±  10.8 ms    [User: 6.1 ms, System: 92.2 ms]
  Range (min … max):   229.4 ms … 267.9 ms    10 runs

Benchmark 2: zlib-ng.exe pip sync ./scripts/benchmark/req.txt
  Time (mean ± σ):     241.3 ms ±   6.2 ms    [User: 7.7 ms, System: 90.6 ms]
  Range (min … max):   233.9 ms … 252.1 ms    10 runs

Benchmark 3: zlib-rs.exe pip sync ./scripts/benchmark/req.txt
  Time (mean ± σ):     242.8 ms ±   7.7 ms    [User: 6.2 ms, System: 23.4 ms]
  Range (min … max):   236.1 ms … 262.8 ms    10 runs

Benchmark 4: zlib-ng.exe pip sync ./scripts/benchmark/req.txt
  Time (mean ± σ):     245.9 ms ±   5.7 ms    [User: 1.5 ms, System: 59.4 ms]
  Range (min … max):   240.9 ms … 257.3 ms    10 runs

Summary
  zlib-rs.exe pip sync ./scripts/benchmark/req.txt ran
    1.00 ± 0.05 times faster than zlib-ng.exe pip sync ./scripts/benchmark/req.txt
    1.01 ± 0.06 times faster than zlib-rs.exe pip sync ./scripts/benchmark/req.txt
    1.02 ± 0.05 times faster than zlib-ng.exe pip sync ./scripts/benchmark/req.txt

Closes #11885.

zanieb · 2025-03-02T19:14:31Z

Can we drop cmake from the contributing guide then?

charliermarsh · 2025-03-02T19:20:28Z

Sure... Do you know if a C compiler is still "required"? The instructions seem sort of ancient.

konstin · 2025-03-03T10:30:04Z

On a ubuntu container, with this PR, you need only gcc and make to build uv. It successfully removes the cmake dependency, so you only need the sudo apt install build-essential you need for building ~everything.

BurntSushi

Love it!

charliermarsh · 2025-03-03T15:53:38Z

Thanks @konstin. I'll restore the instructions around build-essential.

konstin · 2025-03-03T17:11:30Z

For me it's a question of who we're targeting: Anyone with dev experience will have build-essential installed as you hit when compiling almost anything, if we're targeting beginners (i think we have some contributing to uv), it helps being explicit on "curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh && sudo apt install build-essential and you're ready to develop".

charliermarsh · 2025-03-03T17:22:10Z

I left it in for now, just seems unrelated to this change to remove it.

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.6.3` -> `0.6.4` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.6.4`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#064) [Compare Source](astral-sh/uv@0.6.3...0.6.4) ##### Enhancements - Upgrade pypy3.10 to v7.3.19 ([#11814](astral-sh/uv#11814)) - Allow configuring log verbosity from the CLI (i.e., `-vvv`) ([#11758](astral-sh/uv#11758)) - Warn when duplicate index names found in single file ([#11824](astral-sh/uv#11824)) ##### Bug fixes - Always store registry index on resolution packages ([#11815](astral-sh/uv#11815)) - Avoid error on relative paths in `uv tool uninstall` ([#11889](astral-sh/uv#11889)) - Avoid silently dropping errors in directory enumeration ([#11890](astral-sh/uv#11890)) - Disable interactive git terminal prompts during fetches ([#11744](astral-sh/uv#11744)) - Discover Windows registry (PEP 514) Python versions across 32/64-bit ([#11801](astral-sh/uv#11801)) - Don't panic on Ctrl-C in confirm prompt ([#11706](astral-sh/uv#11706)) - Fix non-directory in workspace on Windows ([#11833](astral-sh/uv#11833)) - Make interpreter caching robust to OS upgrades ([#11875](astral-sh/uv#11875)) - Respect `include-system-site-packages` in layered environments ([#11873](astral-sh/uv#11873)) - Suggest `uv tool update-shell` in PowerShell ([#11846](astral-sh/uv#11846)) - Update code page to `65001` before setting environment variables in virtual environments ([#11831](astral-sh/uv#11831)) - Use hash instead of full wheel name in wheels bucket ([#11738](astral-sh/uv#11738)) - Fix version string truncation while generating cache_key ([#11830](astral-sh/uv#11830)) - Explicitly handle ctrl-c in confirmation prompt instead of using a signal handler ([#11897](astral-sh/uv#11897)) ##### Performance - Avoid cloning to string when creating cache path ([#11772](astral-sh/uv#11772)) - Avoid redundant clones in version containment check ([#11767](astral-sh/uv#11767)) - Avoid string allocation when enumerating tool names ([#11910](astral-sh/uv#11910)) - Avoid using owned `String` for package name constructors ([#11768](astral-sh/uv#11768)) - Avoid using owned `String` in deserializers ([#11764](astral-sh/uv#11764)) - Migrate to `zlib-rs` (again) ([#11894](astral-sh/uv#11894)) - Remove unnecessary clones when adding package names ([#11771](astral-sh/uv#11771)) - Skip unquote allocation for non-quoted strings ([#11813](astral-sh/uv#11813)) - Use `SmallString` for filenames and URLs ([#11765](astral-sh/uv#11765)) - Use a Boxed slice for version specifiers ([#11766](astral-sh/uv#11766)) - Use matches over contains for extra value parsing ([#11770](astral-sh/uv#11770)) ##### Documentation - Avoid fallback to PyPI in mixed CPU/CUDA example ([#11115](astral-sh/uv#11115)) - Docs: Clarify that setting cache-keys overrides defaults ([#11895](astral-sh/uv#11895)) - Document our MSRV policy ([#11898](astral-sh/uv#11898)) - Fix reference to macOS cache path ([#11845](astral-sh/uv#11845)) - Fix typo in `no_default_groups` documentation and changelog ([#11928](astral-sh/uv#11928)) - Update the "Locking and syncing" page ([#11647](astral-sh/uv#11647)) - Update alternative indexes documentation to use new interface ([#10826](astral-sh/uv#10826)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

uv no longer requires cmake; see astral-sh/uv#11894

charliermarsh added the performance Potential performance improvement label Mar 2, 2025

charliermarsh marked this pull request as ready for review March 2, 2025 17:04

charliermarsh requested review from BurntSushi and konstin March 3, 2025 02:19

charliermarsh added the no-build Disable building binaries in CI label Mar 3, 2025

charliermarsh force-pushed the charlie/bench-zlib branch from 6e60f38 to 4f6b40d Compare March 3, 2025 02:19

konstin approved these changes Mar 3, 2025

View reviewed changes

BurntSushi approved these changes Mar 3, 2025

View reviewed changes

charliermarsh added 2 commits March 3, 2025 10:59

Benhcmark zlib

df3280c

Change backend

4c4f32a

Reverts

f842395

charliermarsh force-pushed the charlie/bench-zlib branch from 4f6b40d to f842395 Compare March 3, 2025 17:21

charliermarsh enabled auto-merge (squash) March 3, 2025 17:22

charliermarsh merged commit c3d809d into main Mar 3, 2025
84 checks passed

charliermarsh deleted the charlie/bench-zlib branch March 3, 2025 17:29

BrewTestBot mentioned this pull request Mar 4, 2025

uv 0.6.4 Homebrew/homebrew-core#209642

Merged

marxin mentioned this pull request Mar 15, 2025

zlib: switch to faster native Rust implementation davidlattimore/wild#563

Merged

zanieb mentioned this pull request May 9, 2025

Consider distributing zlib-ng astral-sh/python-build-standalone#464

Open

zanieb added a commit to zanieb/uv-snap that referenced this pull request Sep 8, 2025

Remove cmake from build-packages

02d054d

uv no longer requires cmake; see astral-sh/uv#11894

zanieb mentioned this pull request Sep 8, 2025

Remove cmake from build-packages lengau/uv-snap#13

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Migrate to `zlib-rs` (again)#11894

Migrate to `zlib-rs` (again)#11894
charliermarsh merged 3 commits intomainfrom
charlie/bench-zlib

charliermarsh commented Mar 2, 2025 •

edited

Loading

Uh oh!

zanieb commented Mar 2, 2025

Uh oh!

charliermarsh commented Mar 2, 2025

Uh oh!

konstin commented Mar 3, 2025

Uh oh!

BurntSushi left a comment

Uh oh!

charliermarsh commented Mar 3, 2025

Uh oh!

konstin commented Mar 3, 2025

Uh oh!

charliermarsh commented Mar 3, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Comments

Conversation

charliermarsh commented Mar 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

zanieb commented Mar 2, 2025

Uh oh!

charliermarsh commented Mar 2, 2025

Uh oh!

konstin commented Mar 3, 2025

Uh oh!

BurntSushi left a comment

Choose a reason for hiding this comment

Uh oh!

charliermarsh commented Mar 3, 2025

Uh oh!

konstin commented Mar 3, 2025

Uh oh!

charliermarsh commented Mar 3, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

charliermarsh commented Mar 2, 2025 •

edited

Loading