Prevent concurrent updates of the environment in `uv run` by oconnor663 · Pull Request #14153 · astral-sh/uv

oconnor663 · 2025-06-20T14:07:02Z

This is very similar to the locking added for uv sync, uv add, and uv remove in #13869. Improving our (f)locking in general is tracked in
#13883.

This is a minimal change that un-breaks parallel invocations of uv run. It's too easy to confuse locking as in "lockfile generation" and locking as in "file locking to prevent race conditions", so we probably need a broader overhaul of how these things are named. It's also too easy to miss a callsite here, or to accidentally retain one of these locks across run_to_completion, so ideally we could move this file locking down to a lower-level function that's shared by all these callers, possibly pip::operations::install. I want to get this minimal change through testing and review before I tackle either of those.

Here's a repro of the current race condition in uv run, which this PR fixes. Add a bunch of dependencies to provoke it, as suggested in #12751:

$ cd `mktemp -d`
$ uv init --package --name scratch
Initialized project `scratch`
$ uv add boto3 fastapi numba pandas polars protobuf pyarrow pydantic requests urllib3 scikit-learn jupyter
...
$ rm -r .venv && uv run -q scratch & uv run -q scratch
[1] 1451463
error: Failed to install: jupyterlab_widgets-3.0.15-py3-none-any.whl (jupyterlab-widgets==3.0.15)
  Caused by: failed to copy file from /home/jacko/.cache/uv/archive-v0/zXFcriPVQ1wOXqf_UUK1u/jupyterlab_widgets-3.0.15.data/data/share/jupyter/labextensions/@jupyter-widgets/jupyterlab-manager/schemas/@jupyter-widgets/jupyterlab-manager/plugin.json to /tmp/tmp.NFEGYgsQnF/.venv/lib/python3.13/site-packages/jupyterlab_widgets-3.0.15.data/data/share/jupyter/labextensions/@jupyter-widgets/jupyterlab-manager/schemas/@jupyter-widgets/jupyterlab-manager/plugin.json: No such file or directory (os error 2)

With this change:

$ rm -r .venv && ~/uv/target/fast-build/uv run -q scratch & ~/uv/target/fast-build/uv run -q scratch
[1] 1452335
Hello from scratch!
[1]  + done       ~/uv/target/fast-build/uv run -q scratch
Hello from scratch!

# Run 100 invocations in parallel.
$ rm -r .venv && for i in `seq 100` ; do ~/uv/target/fast-build/uv run -q scratch & done
[lots of job output, no failures]

This is very similar to the locking added for `uv sync`, `uv add`, and `uv remove` in #13869. Improving our (f)locking in general is tracked in #13883. This is a minimal change that un-breaks parallel invocations of `uv run`. It's too easy to confuse locking as in "lockfile generation" and locking as in "file locking to prevent race conditions", so we probably need a broader overhaul of how these things are named. It's also too easy to miss a callsite here, or to accidentally retain one of these locks across `run_to_completion`, so ideally we could move this file locking down to a lower-level function that's shared by all these callers, possibly `pip::operations::install`. I want to get this minimal change through testing and review before I tackle either of those.

oconnor663 · 2025-06-20T15:22:08Z

I wonder if this is a flake? Reruning it. (Edit: Yep: #14160.)

    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
    Snapshot: run_groups_requires_python-4
    Source: crates/uv/tests/it/run.rs:4690
    ────────────────────────────────────────────────────────────────────────────────
    Expression: snapshot
    ────────────────────────────────────────────────────────────────────────────────
    -old snapshot
    +new results
    ────────────┬───────────────────────────────────────────────────────────────────
        1     1 │ exit_code: 0
        2     2 │ ----- stdout -----
        3     3 │ 
        4     4 │ ----- stderr -----
              5 │+Using CPython 3.12.[X] interpreter at: [PYTHON-3.12]
              6 │+Removed virtual environment at: .venv
              7 │+Creating virtual environment at: .venv
        5     8 │ Resolved 6 packages in [TIME]
        6       │-Audited 2 packages in [TIME]
              9 │+Installed 2 packages in [TIME]
             10 │+ + sniffio==1.3.1
             11 │+ + typing-extensions==4.10.0
    ────────────┴───────────────────────────────────────────────────────────────────
    Stopped on the first failure. Run `cargo insta test` to run all snapshots.
    test run::run_groups_requires_python ... FAILED

konstin · 2025-06-20T16:04:01Z

crates/uv/src/commands/project/run.rs

                .await?
                .into_environment()?;

+                let _lock = environment.lock().await?;


nit: we can more one statement later, just before update_environment

oconnor663 · 2025-06-20T20:23:11Z

Manual testing for the two script cases. First unlocked:

$ cd `mktemp -d`
$ uv init --script myscript.py
Initialized script at `myscript.py`
$ uv add --script myscript.py boto3 fastapi numba pandas polars protobuf pyarrow pydantic requests urllib3 scikit-learn jupyter
Updated `myscript.py`
$ for i in `seq 10` ; do
    uv run myscript.py &
done
[...lots of job output...]
error: Failed to install: widgetsnbextension-4.0.14-py3-none-any.whl (widgetsnbextension==4.0.14)
  Caused by: failed to rename file from /home/jacko/.cache/uv/environments-v2/myscript-e99e98cc3e25a2bf/lib/python3.13/site-packages/.tmpRU8hBM/widgetsnbextension.json to /home/jacko/.cache/uv/environments-v2/mys
cript-e99e98cc3e25a2bf/lib/python3.13/site-packages/widgetsnbextension-4.0.14.data/data/etc/jupyter/nbconfig/notebook.d/widgetsnbextension.json: No such file or directory (os error 2)
[...lots more errors...]

Fixed with this PR:

$ uv init --script myscript.py
Initialized script at `myscript.py`
$ uv add --script myscript.py boto3 fastapi numba pandas polars protobuf pyarrow pydantic requests urllib3 scikit-learn jupyter
Updated `myscript.py`
$ for i in `seq 10` ; do
    ~/uv/target/fast-build/uv run myscript.py &
done
[...job output, no errors...]

And now the locked case:

$ uv init --script myscript.py
Initialized script at `myscript.py`
$ uv add --script myscript.py boto3 fastapi numba pandas polars protobuf pyarrow pydantic requests urllib3 scikit-learn jupyter
Updated `myscript.py`
$ uv lock --script myscript.py
Resolved 124 packages in 74ms
$ for i in `seq 10` ; do
    uv run myscript.py &
done
[...lots of job output...]
error: Failed to install: jupyterlab_widgets-3.0.15-py3-none-any.whl (jupyterlab-widgets==3.0.15)
  Caused by: failed to remove directory `/home/jacko/.cache/uv/environments-v2/myscript-ae497719a2d6056a/lib/python3.13/site-packages/jupyterlab_widgets-3.0.15.data`: Directory not empty (os error 39)
[...lots more errors...]

Fixed with this PR:

$ uv init --script myscript.py
Initialized script at `myscript.py`
$ uv add --script myscript.py boto3 fastapi numba pandas polars protobuf pyarrow pydantic requests urllib3 scikit-learn jupyter
Updated `myscript.py`
$ uv lock --script myscript.py
Resolved 124 packages in 12ms
$ for i in `seq 10` ; do
    ~/uv/target/fast-build/uv run myscript.py &
done
[...job output, no errors...]

zanieb · 2025-06-20T20:38:42Z

As a minor note, I'd title this to be user-facing so it makes sense in the changelog

What's the actual user-facing change? "Lock the Python environment during updates in uv run"? or "Prevent concurrent changes to the environment during uv run"?

oconnor663 · 2025-06-20T20:43:28Z

Got it. Yes, I'd go with either of your descriptions above, or maybe "lock the virtual environment when uv run does an implicit sync, to prevent concurrent modifications".

This is very similar to the locking added for `uv sync`, `uv add`, and `uv remove` in #13869. Improving our (f)locking in general is tracked in #13883.

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.7.13` -> `0.7.14` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.7.14`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0714) [Compare Source](astral-sh/uv@0.7.13...0.7.14) ##### Enhancements - Add XPU to `--torch-backend` ([#14172](astral-sh/uv#14172)) - Add ROCm backends to `--torch-backend` ([#14120](astral-sh/uv#14120)) - Remove preview label from `--torch-backend` ([#14119](astral-sh/uv#14119)) - Add `[tool.uv.dependency-groups].mygroup.requires-python` ([#13735](astral-sh/uv#13735)) - Add auto-detection for AMD GPUs ([#14176](astral-sh/uv#14176)) - Show retries for HTTP status code errors ([#13897](astral-sh/uv#13897)) - Support transparent Python patch version upgrades ([#13954](astral-sh/uv#13954)) - Warn on empty index directory ([#13940](astral-sh/uv#13940)) - Publish to DockerHub ([#14088](astral-sh/uv#14088)) ##### Performance - Make cold resolves about 10% faster ([#14035](astral-sh/uv#14035)) ##### Bug fixes - Don't use walrus operator in interpreter query script ([#14108](astral-sh/uv#14108)) - Fix handling of changes to `requires-python` ([#14076](astral-sh/uv#14076)) - Fix implied `platform_machine` marker for `win_amd64` platform tag ([#14041](astral-sh/uv#14041)) - Only update existing symlink directories on preview uninstall ([#14179](astral-sh/uv#14179)) - Serialize Python requests for tools as canonicalized strings ([#14109](astral-sh/uv#14109)) - Support netrc and same-origin credential propagation on index redirects ([#14126](astral-sh/uv#14126)) - Support reading `dependency-groups` from pyproject.tomls with no `[project]` ([#13742](astral-sh/uv#13742)) - Handle an existing shebang in `uv init --script` ([#14141](astral-sh/uv#14141)) - Prevent concurrent updates of the environment in `uv run` ([#14153](astral-sh/uv#14153)) - Filter managed Python distributions by platform before querying when included in request ([#13936](astral-sh/uv#13936)) ##### Documentation - Replace cuda124 with cuda128 ([#14168](astral-sh/uv#14168)) - Document the way member sources shadow workspace sources ([#14136](astral-sh/uv#14136)) - Sync documented PyTorch integration index for CUDA and ROCm versions from PyTorch website ([#14100](astral-sh/uv#14100)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

oconnor663 requested review from konstin and zanieb June 20, 2025 14:07

oconnor663 temporarily deployed to uv-test-registries June 20, 2025 14:08 — with GitHub Actions Inactive

zanieb self-assigned this Jun 20, 2025

konstin approved these changes Jun 20, 2025

View reviewed changes

zanieb mentioned this pull request Jun 20, 2025

run_groups_requires_python flakes with Removed virtual environment at ... #14160

Closed

move one lock acquisition down a bit

661cdf1

oconnor663 enabled auto-merge (squash) June 20, 2025 20:26

oconnor663 temporarily deployed to uv-test-registries June 20, 2025 20:28 — with GitHub Actions Inactive

oconnor663 merged commit 0133bcc into main Jun 20, 2025
87 checks passed

oconnor663 deleted the jack/run_locking branch June 20, 2025 20:34

oconnor663 mentioned this pull request Jun 20, 2025

Stronger locking for parallel operations #13883

Open

11 tasks

zanieb changed the title ~~(f)lock during uv run~~ Prevent concurrent updates of the environment in uv run Jun 21, 2025

zanieb pushed a commit that referenced this pull request Jun 21, 2025

(f)lock during uv run (#14153)

8c574c2

This is very similar to the locking added for `uv sync`, `uv add`, and `uv remove` in #13869. Improving our (f)locking in general is tracked in #13883.

BrewTestBot mentioned this pull request Jun 23, 2025

uv 0.7.14 Homebrew/homebrew-core#227918

Merged

zanieb mentioned this pull request Sep 17, 2025

Weird behaviour when running concurrent uv run --script with different python versions #15918

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Prevent concurrent updates of the environment in `uv run`#14153

Prevent concurrent updates of the environment in `uv run`#14153
oconnor663 merged 2 commits intomainfrom
jack/run_locking

oconnor663 commented Jun 20, 2025

Uh oh!

oconnor663 commented Jun 20, 2025 •

edited

Loading

Uh oh!

konstin Jun 20, 2025

Uh oh!

oconnor663 commented Jun 20, 2025

Uh oh!

Uh oh!

zanieb commented Jun 20, 2025

Uh oh!

oconnor663 commented Jun 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments

Conversation

oconnor663 commented Jun 20, 2025

Uh oh!

oconnor663 commented Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

konstin Jun 20, 2025

Choose a reason for hiding this comment

Uh oh!

oconnor663 commented Jun 20, 2025

Uh oh!

Uh oh!

zanieb commented Jun 20, 2025

Uh oh!

oconnor663 commented Jun 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

oconnor663 commented Jun 20, 2025 •

edited

Loading