Write the path of the parent environment to an `extends-environment` key in the `pyvenv.cfg` file of an ephemeral environment by AlexWaygood · Pull Request #13598 · astral-sh/uv

AlexWaygood · 2025-05-22T17:18:02Z

Summary

Ephemeral environments created by uv run --with extend a parent (virtual or system) environment by adding a .pth file to the ephemeral environment's site-packages directory. The pth file contains Python code to dynamically add the parent environment's site-packages directory to Python's import search paths in addition to the ephemeral environment's site-packages directory. This works well at runtime, but is not currently understood by static analysis tools like ty.

As discussed in astral-sh/ty#320, ty could implement support for these virtual environments by attempting to parse the Python code uv is writing into these .pth files. This has many disadvantages however:

It seems somewhat error-prone. The specific Python code uv is writing into the .pth files could change at any time -- and if another tool implements a similar ephemeral-environment feature, it could use different Python idioms.
It would not be particularly efficient for ty to have to attempt to parse Python snippets in every .pth file in the site-packages directory (it could hurt our ability to effectively implement a persistent cache even if it doesn't cost that much in wall time, since we need to know the import search paths very early on).
Philosophically, it feels out of spirit with the kind of analysis a static type checker should be expected to do.

Instead, this PR implements a feature where uv additionally writes the sys.prefix of the parent environment to a new extends-environment key of the ephemeral environment's pyvenv.cfg file. This should make it much easier for ty -- and potentially other static-analysis tools -- to statically and reliably understand the relationship between the two environments. If ty sees that a pyvenv.cfg file has a uv key in it and also has a extends-environment key in it, it will be able to easily add the site-packages directory of the parent environment to its resolved search paths as well as the site-packages directory of the ephemeral child environment.

A question was raised in astral-sh/ty#320 (comment) about whether it was standards-compliant to add new keys to pyvenv.cfg files created by uv. I think it is. pyvenv.cfg files are in general very under-specified: only a small handful of keys are currently consistent between virtual-environment creation tools. Different tools already often add bespoke keys to their pyvenv.cfg files -- see astral-sh/ty#320 (comment) for some comparisons between uv, the virtualenv Python library, and the standard-library venv module.

Test Plan

I added a test that demonstrates that the extends-environment key is written with the correct value to the pyvenv.cfg file of an ephemeral environment.

…key in the `pyvenv.cfg` file of an ephemeral environment

crates/uv/tests/it/run.rs

Co-authored-by: konsti <[email protected]>

zanieb · 2025-05-22T18:20:54Z

As a note, this will suffer from the same issue as reported in #12889 — but it won't be "worse".

AlexWaygood · 2025-05-22T18:25:23Z

Argh, well, it seems I need some help with getting the venv directory filtering to work on Windows. (I haven't actually changed anything to do with the filtering there -- not sure why it's not working on this snapshot?)

Other than that, this is ready for review!

zanieb · 2025-05-22T18:30:55Z

(I haven't actually changed anything to do with the filtering there -- not sure why it's not working on this snapshot?)

I don't think we've ever filtered escape_for_python paths (which I guess differ on Windows).

AlexWaygood · 2025-05-22T20:22:16Z

Well, the test is finally passing on all platforms... If there's a better way to write a test for this, LMK!

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.7.7` -> `0.7.13` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.7.13`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0713) [Compare Source](astral-sh/uv@0.7.12...0.7.13) ##### Python - Add Python 3.14.0b2 - Add Python 3.13.5 - Fix stability of `uuid.getnode` on 3.13 See the [`python-build-standalone` release notes](https://github.com/astral-sh/python-build-standalone/releases/tag/20250612) for more details. ##### Enhancements - Download versions in `uv python pin` if not found ([#13946](astral-sh/uv#13946)) - Use TTY detection to determine if SIGINT forwarding is enabled ([#13925](astral-sh/uv#13925)) - Avoid fetching an exact, cached Git commit, even if it isn't locked ([#13748](astral-sh/uv#13748)) - Add `zstd` and `deflate` to `Accept-Encoding` ([#13982](astral-sh/uv#13982)) - Build binaries for riscv64 ([#12688](astral-sh/uv#12688)) ##### Bug fixes - Check if relative URL is valid directory before treating as index ([#13917](astral-sh/uv#13917)) - Ignore Python discovery errors during `uv python pin` ([#13944](astral-sh/uv#13944)) - Do not allow `uv add --group ... --script` ([#13997](astral-sh/uv#13997)) ##### Preview changes - Build backend: Support namespace packages ([#13833](astral-sh/uv#13833)) ##### Documentation - Add 3.14 to the supported platform reference ([#13990](astral-sh/uv#13990)) - Add an `llms.txt` to uv ([#13929](astral-sh/uv#13929)) - Add supported macOS version to the platform reference ([#13993](astral-sh/uv#13993)) - Update platform support reference to include Python implementation list ([#13991](astral-sh/uv#13991)) - Update pytorch.md ([#13899](astral-sh/uv#13899)) - Update the CLI help and reference to include references to the Python bin directory ([#13978](astral-sh/uv#13978)) ### [`v0.7.12`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0712) [Compare Source](astral-sh/uv@0.7.11...0.7.12) ##### Enhancements - Add `uv python pin --rm` to remove `.python-version` pins ([#13860](astral-sh/uv#13860)) - Don't hint at versions removed by `excluded-newer` ([#13884](astral-sh/uv#13884)) - Add hint to use `tool.uv.environments` on resolution error ([#13455](astral-sh/uv#13455)) - Add hint to use `tool.uv.required-environments` on resolution error ([#13575](astral-sh/uv#13575)) - Improve `python pin` error messages ([#13862](astral-sh/uv#13862)) ##### Bug fixes - Lock environments during `uv sync`, `uv add` and `uv remove` to prevent race conditions ([#13869](astral-sh/uv#13869)) - Add `--no-editable` to `uv export` for `pylock.toml` ([#13852](astral-sh/uv#13852)) ##### Documentation - List `.gitignore` in project init files ([#13855](astral-sh/uv#13855)) - Move the pip interface documentation into the concepts section ([#13841](astral-sh/uv#13841)) - Remove the configuration section in favor of concepts / reference ([#13842](astral-sh/uv#13842)) - Update Git and GitHub Actions docs to mention `gh auth login` ([#13850](astral-sh/uv#13850)) ##### Preview - Fix directory glob traversal fallback preventing exclusion of all files ([#13882](astral-sh/uv#13882)) ### [`v0.7.11`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0711) [Compare Source](astral-sh/uv@0.7.10...0.7.11) ##### Python - Add Python 3.14.0b1 - Add Python 3.13.4 - Add Python 3.12.11 - Add Python 3.11.13 - Add Python 3.10.18 - Add Python 3.9.23 ##### Enhancements - Add Pyodide support ([#12731](astral-sh/uv#12731)) - Better error message for version specifier with missing operator ([#13803](astral-sh/uv#13803)) ##### Bug fixes - Downgrade `reqwest` and `hyper-util` to resolve connection reset errors over IPv6 ([#13835](astral-sh/uv#13835)) - Prefer `uv`'s binary's version when checking if it's up to date ([#13840](astral-sh/uv#13840)) ##### Documentation - Use "terminal driver" instead of "shell" in `SIGINT` docs ([#13787](astral-sh/uv#13787)) ### [`v0.7.10`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0710) [Compare Source](astral-sh/uv@0.7.9...0.7.10) ##### Enhancements - Add `--show-extras` to `uv tool list` ([#13783](astral-sh/uv#13783)) - Add dynamically generated sysconfig replacement mappings ([#13441](astral-sh/uv#13441)) - Add data locations to install wheel logs ([#13797](astral-sh/uv#13797)) ##### Bug fixes - Avoid redaction of placeholder `git` username when using SSH authentication ([#13799](astral-sh/uv#13799)) - Propagate credentials to files on devpi indexes ending in `/+simple` ([#13743](astral-sh/uv#13743)) - Restore retention of credentials for direct URLs in `uv export` ([#13809](astral-sh/uv#13809)) ### [`v0.7.9`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#079) [Compare Source](astral-sh/uv@0.7.8...0.7.9) ##### Python The changes reverted in [0.7.8](#078) have been restored. See the [`python-build-standalone` release notes](https://github.com/astral-sh/python-build-standalone/releases/tag/20250529) for more details. ##### Enhancements - Improve obfuscation of credentials in URLs ([#13560](astral-sh/uv#13560)) - Allow running non-default Python implementations via `uvx` ([#13583](astral-sh/uv#13583)) - Add `uvw` as alias for `uv` without console window on Windows ([#11786](astral-sh/uv#11786)) - Allow discovery of x86-64 managed Python builds on macOS ([#13722](astral-sh/uv#13722)) - Differentiate between implicit vs explicit architecture requests ([#13723](astral-sh/uv#13723)) - Implement ordering for Python architectures to prefer native installations ([#13709](astral-sh/uv#13709)) - Only show the first match per platform (and architecture) by default in `uv python list` ([#13721](astral-sh/uv#13721)) - Write the path of the parent environment to an `extends-environment` key in the `pyvenv.cfg` file of an ephemeral environment ([#13598](astral-sh/uv#13598)) - Improve the error message when libc cannot be found, e.g., when using the distroless containers ([#13549](astral-sh/uv#13549)) ##### Performance - Avoid rendering info log level ([#13642](astral-sh/uv#13642)) - Improve performance of `uv-python` crate's manylinux submodule ([#11131](astral-sh/uv#11131)) - Optimize `Version` display ([#13643](astral-sh/uv#13643)) - Reduce number of reference-checks for `uv cache clean` ([#13669](astral-sh/uv#13669)) ##### Bug fixes - Avoid reinstalling dependency group members with `--all-packages` ([#13678](astral-sh/uv#13678)) - Don't fail direct URL hash checking with dependency metadata ([#13736](astral-sh/uv#13736)) - Exit early on `self update` if global `--offline` is set ([#13663](astral-sh/uv#13663)) - Fix cases where the uv lock is incorrectly marked as out of date ([#13635](astral-sh/uv#13635)) - Include pre-release versions in `uv python install --reinstall` ([#13645](astral-sh/uv#13645)) - Set `LC_ALL=C` for git when checking git worktree ([#13637](astral-sh/uv#13637)) - Avoid rejecting Windows paths for remote Python download JSON targets ([#13625](astral-sh/uv#13625)) ##### Preview - Add `uv add --bounds` to configure version constraints ([#12946](astral-sh/uv#12946)) ##### Documentation - Add documentation about Python versions to Tools concept page ([#7673](astral-sh/uv#7673)) - Add example of enabling Dependabot ([#13692](astral-sh/uv#13692)) - Fix `exclude-newer` date format for persistent configuration files ([#13706](astral-sh/uv#13706)) - Quote versions variables in GitLab documentation ([#13679](astral-sh/uv#13679)) - Update Dependabot support status ([#13690](astral-sh/uv#13690)) - Explicitly specify to add a new repo entry to the repos list item in the `.pre-commit-config.yaml` ([#10243](astral-sh/uv#10243)) - Add integration with marimo guide ([#13691](astral-sh/uv#13691)) - Add pronunciation to README ([#5336](astral-sh/uv#5336)) ### [`v0.7.8`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#078) [Compare Source](astral-sh/uv@0.7.7...0.7.8) ##### Python We are reverting most of our Python changes from `uv 0.7.6` and `uv 0.7.7` due to a miscompilation that makes the Python interpreter behave incorrectly, resulting in spurious type-errors involving str. This issue seems to be isolated to x86\_64 Linux, and affected at least Python 3.12, 3.13, and 3.14. The following changes that were introduced in those versions of uv are temporarily being reverted while we test and deploy a proper fix for the miscompilation: - Add Python 3.14 on musl - free-threaded Python on musl - Add Python 3.14.0a7 - Statically link `libpython` into the interpreter on Linux for a significant performance boost See [the issue for details](astral-sh/uv#13610). ##### Documentation - Remove misleading line in pin documentation ([#13611](astral-sh/uv#13611)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

Write the path of the parent environment to an extends-environment …

508bb5b

…key in the `pyvenv.cfg` file of an ephemeral environment

AlexWaygood added the virtualenv Related to virtual environments label May 22, 2025

konstin reviewed May 22, 2025

View reviewed changes

crates/uv/tests/it/run.rs Outdated Show resolved Hide resolved

Update crates/uv/tests/it/run.rs

c04856e

Co-authored-by: konsti <[email protected]>

AlexWaygood force-pushed the alex/ephemeral-venv-parent branch from deb4f20 to 6097ad7 Compare May 22, 2025 18:13

AlexWaygood marked this pull request as ready for review May 22, 2025 18:25

AlexWaygood mentioned this pull request May 22, 2025

Project dependencies not detected when using uv run --with ty astral-sh/ty#320

Closed

AlexWaygood force-pushed the alex/ephemeral-venv-parent branch 4 times, most recently from b8aba36 to dd14cdc Compare May 22, 2025 20:08

make regexes more robust (hopefully fix Windows?)

d6bc6e0

AlexWaygood force-pushed the alex/ephemeral-venv-parent branch from dd14cdc to d6bc6e0 Compare May 22, 2025 20:11

charliermarsh approved these changes May 27, 2025

View reviewed changes

AlexWaygood merged commit 7bba3d0 into main May 27, 2025
86 checks passed

AlexWaygood deleted the alex/ephemeral-venv-parent branch May 27, 2025 12:46

BrewTestBot mentioned this pull request May 30, 2025

uv 0.7.9 Homebrew/homebrew-core#225272

Merged

AlexWaygood mentioned this pull request Aug 14, 2025

Respect parent Python interpreter that invoked ty as a candidate Python env astral-sh/ty#989

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Write the path of the parent environment to an `extends-environment` key in the `pyvenv.cfg` file of an ephemeral environment#13598

Write the path of the parent environment to an `extends-environment` key in the `pyvenv.cfg` file of an ephemeral environment#13598
AlexWaygood merged 3 commits intomainfrom
alex/ephemeral-venv-parent

AlexWaygood commented May 22, 2025

Uh oh!

Uh oh!

zanieb commented May 22, 2025

Uh oh!

AlexWaygood commented May 22, 2025

Uh oh!

zanieb commented May 22, 2025

Uh oh!

AlexWaygood commented May 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Comments

Conversation

AlexWaygood commented May 22, 2025

Summary

Test Plan

Uh oh!

Uh oh!

zanieb commented May 22, 2025

Uh oh!

AlexWaygood commented May 22, 2025

Uh oh!

zanieb commented May 22, 2025

Uh oh!

AlexWaygood commented May 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants