Skip to content

Conversation

@simonw
Copy link
Owner

@simonw simonw commented Dec 26, 2025

HTTP range requests for metadata. Wheel files are zip archives, and zip archives put their file listing at the end. uv tries PEP 658 metadata first, falls back to HTTP range requests for the zip central directory, then full wheel download, then building from source. Each step is slower and riskier. The design makes the fast path cover 99% of cases. None of this requires Rust.

That came from a description of how https://github.com/astral-sh/uv works. Investigate this. First clone that repo to /tmp - then I want you to find the relevant code in the Rust source. Build a separate Python CLI script that uses httpx and the same range trick to return just the metadata (printed neatly) given a URL to a wheel file. Test it on https://files.pythonhosted.org/packages/d1/9f/b22de67c44c8fdac517889feb6f45a90cbc2783f71c70ea9e2614e815126/llm-0.28-py3-none-any.whl

In your report explain exactly how the uv mechanism works in detail and explain your Python recreation too, both with illustrative snippets of the core code

Investigate how uv extracts wheel metadata efficiently using HTTP range requests instead of downloading entire wheels. Key findings:

  • ZIP files store central directory at the end, enabling partial fetches
  • uv fetches last 16KB to get file listing, then just the METADATA file
  • Python recreation (wheel_metadata.py) achieves 70%+ bandwidth savings

Bonus: Also covers uv's compact version packing (u64) for fast comparison. Includes version_packing.py demonstrating the technique.

https://gistpreview.github.io/?0f04e4d1a240bfc3065df5082b629884/index.html

Investigate how uv extracts wheel metadata efficiently using HTTP range
requests instead of downloading entire wheels. Key findings:

- ZIP files store central directory at the end, enabling partial fetches
- uv fetches last 16KB to get file listing, then just the METADATA file
- Python recreation (wheel_metadata.py) achieves 70%+ bandwidth savings

Bonus: Also covers uv's compact version packing (u64) for fast comparison.
Includes version_packing.py demonstrating the technique.
@simonw
Copy link
Owner Author

simonw commented Dec 26, 2025

I later prompted this in the same session:

Compact version representation. uv packs versions into u64 integers where possible, making comparison and hashing fast. Over 90% of versions fit in one u64. This is micro-optimization that compounds across millions of comparisons.

Extra task: find the Rust code that does that, explain it, do a Python port of that as well

@simonw simonw merged commit 289a89c into main Dec 26, 2025
@simonw
Copy link
Owner Author

simonw commented Dec 26, 2025

Weird, that second follow-up prompt wasn't captured in the session.json at all. Filed an issue with Claude Code here:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants