feat: add dynamically generated sysconfig replacement mappings by samypr100 · Pull Request #13441 · astral-sh/uv

samypr100 · 2025-05-14T02:29:09Z

Summary

Implementation referenced in #12239 (comment)

This makes the sysconfig replacements mappings dynamically generated from https://github.com/astral-sh/python-build-standalone/blob/main/cpython-unix/targets.yml

Test Plan

cargo dev tests, and tested scenario from #12901 (comment)

samypr100 · 2025-05-14T02:49:09Z

@zanieb Are you ok with this general approach? There is an big assumption here that targets.yml will be the canonical way to obtain these mappings going forward.

It has the (good? or bad?) side-effect that as soon as targets.yml adds a new non-mapped entry, it will fail the generate tests. We can lock it down to only reference build-standalone releases though, but that may require manual updating/upkeep.

crates/uv-dev/src/generate_sysconfig_mappings.rs

zanieb · 2025-05-20T01:48:31Z

Are you ok with this general approach?

I think so? The targets file may change, but I do think static declaration of the targets is helpful.

I can take a closer look, but I'm at the PyCon Sprints this week and am pretty distracted.

@Gankra might have some thoughts.

Gankra

This all seems vaguely coherent and yeah using a source of truth seems good. Could you give some insight on why the shell script if the rust code already needs to know how to do http requests to github?

Alternatively, could the work the script does be done by fetch-download-metadata.py, so local invokers can just run One Thing?

crates/uv-dev/src/generate_sysconfig_mappings.rs

samypr100 · 2025-05-29T02:18:39Z

This all seems vaguely coherent and yeah using a source of truth seems good. Could you give some insight on why the shell script if the rust code already needs to know how to do http requests to github?

Good question. The shell script exists to update the python-build-standalone release tag used by the .rs file for to generate the sysconfig metadata. This was a change done after #13441 (comment).

Alternatively, could the work the script does be done by fetch-download-metadata.py, so local invokers can just run One Thing?

My thought was to integrate it with the existing cargo dev workflow and tests. I think an alternate script would suffer from the same situation? We'd have to be specific about the release tag we would be pulling in the changes from and it would have to be routinely updated as per #13441 (comment) it was not desirable to point it main. Some other options I can think about are

Make a change to the existing code to point to the latest release tag instead every time (rather than main) which will make it "break" less often when running the cargo dev tests (or equivalent alternate python script run).
We use an environment variable on the existing .rs file instead to control the tag it pulls from and leave it unset locally (making the tests pass when its unset), but CI sets it.

crates/uv-python/src/sysconfig/generated_mappings.rs

crates/uv-dev/src/generate_sysconfig_mappings.rs

codspeed-hq · 2025-05-31T03:18:54Z

CodSpeed Walltime Performance Report

Merging #13441 will improve performances by 15.79%

_{Comparing samypr100:dynamic-sysconfig (c92019a) with main (13a86a2)}

Summary

⚡ 1 improvements
✅ 11 untouched benchmarks

Benchmarks breakdown

	Benchmark	`BASE`	`HEAD`	Change
⚡	`wheelname_parsing_failure[flyte-long-extension]`	110 ns	95 ns	+15.79%

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.7.7` -> `0.7.13` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.7.13`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0713) [Compare Source](astral-sh/uv@0.7.12...0.7.13) ##### Python - Add Python 3.14.0b2 - Add Python 3.13.5 - Fix stability of `uuid.getnode` on 3.13 See the [`python-build-standalone` release notes](https://github.com/astral-sh/python-build-standalone/releases/tag/20250612) for more details. ##### Enhancements - Download versions in `uv python pin` if not found ([#13946](astral-sh/uv#13946)) - Use TTY detection to determine if SIGINT forwarding is enabled ([#13925](astral-sh/uv#13925)) - Avoid fetching an exact, cached Git commit, even if it isn't locked ([#13748](astral-sh/uv#13748)) - Add `zstd` and `deflate` to `Accept-Encoding` ([#13982](astral-sh/uv#13982)) - Build binaries for riscv64 ([#12688](astral-sh/uv#12688)) ##### Bug fixes - Check if relative URL is valid directory before treating as index ([#13917](astral-sh/uv#13917)) - Ignore Python discovery errors during `uv python pin` ([#13944](astral-sh/uv#13944)) - Do not allow `uv add --group ... --script` ([#13997](astral-sh/uv#13997)) ##### Preview changes - Build backend: Support namespace packages ([#13833](astral-sh/uv#13833)) ##### Documentation - Add 3.14 to the supported platform reference ([#13990](astral-sh/uv#13990)) - Add an `llms.txt` to uv ([#13929](astral-sh/uv#13929)) - Add supported macOS version to the platform reference ([#13993](astral-sh/uv#13993)) - Update platform support reference to include Python implementation list ([#13991](astral-sh/uv#13991)) - Update pytorch.md ([#13899](astral-sh/uv#13899)) - Update the CLI help and reference to include references to the Python bin directory ([#13978](astral-sh/uv#13978)) ### [`v0.7.12`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0712) [Compare Source](astral-sh/uv@0.7.11...0.7.12) ##### Enhancements - Add `uv python pin --rm` to remove `.python-version` pins ([#13860](astral-sh/uv#13860)) - Don't hint at versions removed by `excluded-newer` ([#13884](astral-sh/uv#13884)) - Add hint to use `tool.uv.environments` on resolution error ([#13455](astral-sh/uv#13455)) - Add hint to use `tool.uv.required-environments` on resolution error ([#13575](astral-sh/uv#13575)) - Improve `python pin` error messages ([#13862](astral-sh/uv#13862)) ##### Bug fixes - Lock environments during `uv sync`, `uv add` and `uv remove` to prevent race conditions ([#13869](astral-sh/uv#13869)) - Add `--no-editable` to `uv export` for `pylock.toml` ([#13852](astral-sh/uv#13852)) ##### Documentation - List `.gitignore` in project init files ([#13855](astral-sh/uv#13855)) - Move the pip interface documentation into the concepts section ([#13841](astral-sh/uv#13841)) - Remove the configuration section in favor of concepts / reference ([#13842](astral-sh/uv#13842)) - Update Git and GitHub Actions docs to mention `gh auth login` ([#13850](astral-sh/uv#13850)) ##### Preview - Fix directory glob traversal fallback preventing exclusion of all files ([#13882](astral-sh/uv#13882)) ### [`v0.7.11`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0711) [Compare Source](astral-sh/uv@0.7.10...0.7.11) ##### Python - Add Python 3.14.0b1 - Add Python 3.13.4 - Add Python 3.12.11 - Add Python 3.11.13 - Add Python 3.10.18 - Add Python 3.9.23 ##### Enhancements - Add Pyodide support ([#12731](astral-sh/uv#12731)) - Better error message for version specifier with missing operator ([#13803](astral-sh/uv#13803)) ##### Bug fixes - Downgrade `reqwest` and `hyper-util` to resolve connection reset errors over IPv6 ([#13835](astral-sh/uv#13835)) - Prefer `uv`'s binary's version when checking if it's up to date ([#13840](astral-sh/uv#13840)) ##### Documentation - Use "terminal driver" instead of "shell" in `SIGINT` docs ([#13787](astral-sh/uv#13787)) ### [`v0.7.10`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0710) [Compare Source](astral-sh/uv@0.7.9...0.7.10) ##### Enhancements - Add `--show-extras` to `uv tool list` ([#13783](astral-sh/uv#13783)) - Add dynamically generated sysconfig replacement mappings ([#13441](astral-sh/uv#13441)) - Add data locations to install wheel logs ([#13797](astral-sh/uv#13797)) ##### Bug fixes - Avoid redaction of placeholder `git` username when using SSH authentication ([#13799](astral-sh/uv#13799)) - Propagate credentials to files on devpi indexes ending in `/+simple` ([#13743](astral-sh/uv#13743)) - Restore retention of credentials for direct URLs in `uv export` ([#13809](astral-sh/uv#13809)) ### [`v0.7.9`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#079) [Compare Source](astral-sh/uv@0.7.8...0.7.9) ##### Python The changes reverted in [0.7.8](#078) have been restored. See the [`python-build-standalone` release notes](https://github.com/astral-sh/python-build-standalone/releases/tag/20250529) for more details. ##### Enhancements - Improve obfuscation of credentials in URLs ([#13560](astral-sh/uv#13560)) - Allow running non-default Python implementations via `uvx` ([#13583](astral-sh/uv#13583)) - Add `uvw` as alias for `uv` without console window on Windows ([#11786](astral-sh/uv#11786)) - Allow discovery of x86-64 managed Python builds on macOS ([#13722](astral-sh/uv#13722)) - Differentiate between implicit vs explicit architecture requests ([#13723](astral-sh/uv#13723)) - Implement ordering for Python architectures to prefer native installations ([#13709](astral-sh/uv#13709)) - Only show the first match per platform (and architecture) by default in `uv python list` ([#13721](astral-sh/uv#13721)) - Write the path of the parent environment to an `extends-environment` key in the `pyvenv.cfg` file of an ephemeral environment ([#13598](astral-sh/uv#13598)) - Improve the error message when libc cannot be found, e.g., when using the distroless containers ([#13549](astral-sh/uv#13549)) ##### Performance - Avoid rendering info log level ([#13642](astral-sh/uv#13642)) - Improve performance of `uv-python` crate's manylinux submodule ([#11131](astral-sh/uv#11131)) - Optimize `Version` display ([#13643](astral-sh/uv#13643)) - Reduce number of reference-checks for `uv cache clean` ([#13669](astral-sh/uv#13669)) ##### Bug fixes - Avoid reinstalling dependency group members with `--all-packages` ([#13678](astral-sh/uv#13678)) - Don't fail direct URL hash checking with dependency metadata ([#13736](astral-sh/uv#13736)) - Exit early on `self update` if global `--offline` is set ([#13663](astral-sh/uv#13663)) - Fix cases where the uv lock is incorrectly marked as out of date ([#13635](astral-sh/uv#13635)) - Include pre-release versions in `uv python install --reinstall` ([#13645](astral-sh/uv#13645)) - Set `LC_ALL=C` for git when checking git worktree ([#13637](astral-sh/uv#13637)) - Avoid rejecting Windows paths for remote Python download JSON targets ([#13625](astral-sh/uv#13625)) ##### Preview - Add `uv add --bounds` to configure version constraints ([#12946](astral-sh/uv#12946)) ##### Documentation - Add documentation about Python versions to Tools concept page ([#7673](astral-sh/uv#7673)) - Add example of enabling Dependabot ([#13692](astral-sh/uv#13692)) - Fix `exclude-newer` date format for persistent configuration files ([#13706](astral-sh/uv#13706)) - Quote versions variables in GitLab documentation ([#13679](astral-sh/uv#13679)) - Update Dependabot support status ([#13690](astral-sh/uv#13690)) - Explicitly specify to add a new repo entry to the repos list item in the `.pre-commit-config.yaml` ([#10243](astral-sh/uv#10243)) - Add integration with marimo guide ([#13691](astral-sh/uv#13691)) - Add pronunciation to README ([#5336](astral-sh/uv#5336)) ### [`v0.7.8`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#078) [Compare Source](astral-sh/uv@0.7.7...0.7.8) ##### Python We are reverting most of our Python changes from `uv 0.7.6` and `uv 0.7.7` due to a miscompilation that makes the Python interpreter behave incorrectly, resulting in spurious type-errors involving str. This issue seems to be isolated to x86\_64 Linux, and affected at least Python 3.12, 3.13, and 3.14. The following changes that were introduced in those versions of uv are temporarily being reverted while we test and deploy a proper fix for the miscompilation: - Add Python 3.14 on musl - free-threaded Python on musl - Add Python 3.14.0a7 - Statically link `libpython` into the interpreter on Linux for a significant performance boost See [the issue for details](astral-sh/uv#13610). ##### Documentation - Remove misleading line in pin documentation ([#13611](astral-sh/uv#13611)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

samypr100 temporarily deployed to release May 14, 2025 02:29 — with GitHub Actions Inactive

samypr100 force-pushed the dynamic-sysconfig branch from cd7b80a to facc385 Compare May 14, 2025 02:45

samypr100 temporarily deployed to release May 14, 2025 02:45 — with GitHub Actions Inactive

samypr100 temporarily deployed to release May 14, 2025 03:02 — with GitHub Actions Inactive

samypr100 force-pushed the dynamic-sysconfig branch from 305497d to 58b75ab Compare May 14, 2025 03:02

samypr100 temporarily deployed to release May 14, 2025 03:03 — with GitHub Actions Inactive

samypr100 force-pushed the dynamic-sysconfig branch from 58b75ab to 2208ebb Compare May 20, 2025 01:30

samypr100 temporarily deployed to release May 20, 2025 01:30 — with GitHub Actions Inactive

samypr100 marked this pull request as ready for review May 20, 2025 01:31

zanieb reviewed May 20, 2025

View reviewed changes

crates/uv-dev/src/generate_sysconfig_mappings.rs Outdated Show resolved Hide resolved

zanieb added the no-build Disable building binaries in CI label May 20, 2025

samypr100 force-pushed the dynamic-sysconfig branch 3 times, most recently from fa252d8 to 70cb649 Compare May 26, 2025 22:22

Gankra approved these changes May 27, 2025

View reviewed changes

crates/uv-dev/src/generate_sysconfig_mappings.rs Show resolved Hide resolved

crates/uv-dev/src/generate_sysconfig_mappings.rs Show resolved Hide resolved

zanieb assigned Gankra May 28, 2025

zanieb reviewed May 29, 2025

View reviewed changes

crates/uv-python/src/sysconfig/generated_mappings.rs Outdated Show resolved Hide resolved

zanieb reviewed May 29, 2025

View reviewed changes

crates/uv-dev/src/generate_sysconfig_mappings.rs Outdated Show resolved Hide resolved

samypr100 force-pushed the dynamic-sysconfig branch from fb1a4f1 to 3667d11 Compare May 29, 2025 13:02

zanieb approved these changes May 29, 2025

View reviewed changes

geofft self-requested a review May 30, 2025 15:44

feat: add dynamically generated sysconfig replacement mappings

5fdc21f

bump to pbs release targets to 20250529

c92019a

samypr100 force-pushed the dynamic-sysconfig branch from 3667d11 to c92019a Compare May 31, 2025 03:02

zanieb merged commit d65c146 into astral-sh:main Jun 2, 2025
100 checks passed

samypr100 deleted the dynamic-sysconfig branch June 2, 2025 18:15

BrewTestBot mentioned this pull request Jun 3, 2025

uv 0.7.10 Homebrew/homebrew-core#225630

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

feat: add dynamically generated sysconfig replacement mappings#13441

feat: add dynamically generated sysconfig replacement mappings#13441
zanieb merged 2 commits intoastral-sh:mainfrom
samypr100:dynamic-sysconfig

samypr100 commented May 14, 2025 •

edited

Loading

Uh oh!

samypr100 commented May 14, 2025 •

edited

Loading

Uh oh!

Uh oh!

zanieb commented May 20, 2025

Uh oh!

Gankra left a comment

Uh oh!

Uh oh!

Uh oh!

samypr100 commented May 29, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

codspeed-hq bot commented May 31, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments

Conversation

samypr100 commented May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Uh oh!

samypr100 commented May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

zanieb commented May 20, 2025

Uh oh!

Gankra left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

samypr100 commented May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codspeed-hq bot commented May 31, 2025

CodSpeed Walltime Performance Report

Merging #13441 will improve performances by 15.79%

Summary

Benchmarks breakdown

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

samypr100 commented May 14, 2025 •

edited

Loading

samypr100 commented May 14, 2025 •

edited

Loading

samypr100 commented May 29, 2025 •

edited

Loading