Document improvements to build-isolation setups by charliermarsh · Pull Request #15326 · astral-sh/uv

charliermarsh · 2025-08-16T15:51:58Z

Summary

There's a lot here (maybe it should go somewhere else?), but this is a complex topic and I wanted to include a lot of copy-pasteable examples for common cases.

Closes #10694.
Closes #13959.
Closes #15248.
Closes #15316.

vwxyzjn

Love the examples! The flash attention one doesn't work, but I tested deep_ep and deep_gemm which all works!

vwxyzjn · 2025-08-16T17:50:06Z

docs/concepts/projects/config.md

+```toml title="pyproject.toml"
+[project]
+name = "project"
+version = "0.1.0"
+description = "..."
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = ["flash-attn", "torch"]

-```console
-$ uv sync --extra compile
- + cchardet==2.1.7
- - cython==3.0.11
- - setuptools==73.0.1
+[tool.uv.extra-build-dependencies]
+flash-attn = [{ dependencies = ["torch"], match-runtime = true }]
+
+[tool.uv.extra-build-variables]
+flash-attn = { FLASH_ATTENTION_SKIP_CUDA_BUILD = "TRUE" }


This doesn't work:

ubuntu@costa-dev-worker-0:~/code/thirdparty/test_uv$ cat pyproject.toml [project] name = "project" version = "0.1.0" description = "..." readme = "README.md" requires-python = ">=3.12" dependencies = ["flash-attn", "torch"] [tool.uv.extra-build-dependencies] flash-attn = [{ dependencies = ["torch"], match-runtime = true }] [tool.uv.extra-build-variables] flash-attn = { FLASH_ATTENTION_SKIP_CUDA_BUILD = "TRUE" } ubuntu@costa-dev-worker-0:~/code/thirdparty/test_uv$ uv sync warning: Failed to parse `pyproject.toml` during settings discovery: TOML parse error at line 10, column 14 | 10 | flash-attn = [{ dependencies = ["torch"], match-runtime = true }] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ data did not match any variant of untagged enum ExtraBuildDependencyWire error: Failed to parse: `pyproject.toml` Caused by: TOML parse error at line 10, column 14 | 10 | flash-attn = [{ dependencies = ["torch"], match-runtime = true }] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ data did not match any variant of untagged enum ExtraBuildDependencyWire

Oh thank you for trying this, there’s a typo 😩

vwxyzjn · 2025-08-16T22:40:52Z

I have put a minimal repro here: https://github.com/vwxyzjn/minimal-uv-deep-ep-gemm-installation/tree/main

zanieb

I have some minor edits and, as you said, we probably want to find a better home for all this content, but let's just get this in and iterate from there since it's so important.

tobiasdiez · 2025-08-19T02:03:46Z

These are very nice improvements. Thanks a lot for everyone that worked on this!

Another use case of "no-build-isolation" of the main project is that some build backends (like python-meson) allow to recompile files on the fly in editable installs. For this to work, naturally the build dependencies of the main project need to be available also at runtime. (More details at #10694 (comment))
It's not clear to me if those recent changes cover this use case as well. One still needs to create an "extra" for the build dependencies, duplicating what is already in build-system.requires, and then use uv sync --extra build, right? Or is there a way to sync the build requirements of the main project without duplicating them to an extra?

charliermarsh · 2025-08-19T09:25:48Z

Thanks @tobiasdiez! I can take a look and add some documentation for that. Can you give me a minimal pyproject.toml + a set of commands to reproduce, for testing?

zsol · 2025-08-20T10:12:49Z

docs/concepts/projects/config.md


-You could run the following sequence of commands to sync `flash-attn`:
+The use of `extra-build-dependencies` and `extra-build-variables` are tracked in the uv cache, such
+that changes to these settings will not trigger a reinstall and rebuild of the affected packages.


This is a typo

Suggested change

that changes to these settings will not trigger a reinstall and rebuild of the affected packages.

that changes to these settings will trigger a reinstall and rebuild of the affected packages.

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.8.11` -> `0.8.12` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.8.12`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0812) [Compare Source](astral-sh/uv@0.8.11...0.8.12) ##### Python - Add 3.13.7 - Improve performance of zstd in Python 3.14 See the [python-build-standalone release notes](https://github.com/astral-sh/python-build-standalone/releases/tag/20250818) for details. ##### Enhancements - Add an `aarch64-pc-windows-msvc` target for `python-platform` ([#15347](astral-sh/uv#15347)) - Add fallback parent process detection to `uv tool update-shell` ([#15356](astral-sh/uv#15356)) - Install non-build-isolation packages in a second phase ([#15306](astral-sh/uv#15306)) - Add hint when virtual environments are included in source distributions ([#15202](astral-sh/uv#15202)) - Add Docker images derived from `buildpack-deps:trixie`, `debian:trixie-slim`, `alpine:3.22` ([#15351](astral-sh/uv#15351)) ##### Bug fixes - Reject already-installed wheels built with outdated settings ([#15289](astral-sh/uv#15289)) - Skip interpreters that are not found on query ([#15315](astral-sh/uv#15315)) - Handle dotted package names in script path resolution ([#15300](astral-sh/uv#15300)) - Reject `match-runtime = true` for dynamic packages ([#15292](astral-sh/uv#15292)) ##### Documentation - Document improvements to build-isolation setups ([#15326](astral-sh/uv#15326)) - Fix reference documentation recommendation to use `uv cache clean` instead of `clear` ([#15313](astral-sh/uv#15313)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

tobiasdiez · 2025-09-13T00:16:44Z

docs/concepts/projects/config.md

-installed in the project environment _prior_ to installing the package itself. This can be achieved
-by separating out the build dependencies and the packages that require them into distinct extras.
-For example:
+To ensure that a build dependency matches the version of the package that is or will be installed in


Does this also ensure that the package annotated with match-runtime = true will only be built once? So in the example below (assuming that torch wouldn't ship wheels), would torch be built once as the build dependency of deepspeed and then another time to be installed in the main env? Or does uv cache the built of torch and reuses it?

I hope it is cached - in this case, match-runtime might be also a very useful option to set for packages that take a long time to build. I can open a PR to add this if wished.

tobiasdiez · 2025-09-13T01:14:43Z

Thanks @tobiasdiez! I can take a look and add some documentation for that. Can you give me a minimal pyproject.toml + a set of commands to reproduce, for testing?

Sorry for taking so long to come back to you. Here is an example pyproject.toml that shows the problem:

[project]
name = "test"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.12"
dependencies = []
[build-system]
build-backend = 'mesonpy'
requires = ["cython>=3.1.3", "meson-python>=0.18.0"]

This is just what you get with uv init plus adding the meson-python as a build backend. To run uv sync you also need a simple meson.build, say

project(
  'test',
  ['c', 'cpp', 'cython'],
  meson_version: '>=1.2',
)

py_module = import('python')
py = py_module.find_installation(pure: false)

py.install_sources(
  'main.py',
)

In a real application you would now actually use cython to compile an extension etc, but for the sake of having a simple example let's ignore that.

After uv sync the package is installed in editable mode, and in particular you have a file .venv\Lib\site-packages\_test_editable_loader.py which intercepts all imports and rebuilds the project if necessary. This means you need to have the build dependencies also installed at runtime. This is however not the case currently if you run uv sync.

Current workaround is:

uv pip install meson-python cython
uv sync --no-build-isolation

So it's similar to the problems discussed in this PR, but the difference is that the no-build-isolation concerns the main project and not one of its dependencies.

Let me know if this is sufficient to understand the problem, or if you would prefer to have a full example that actually compiles something etc.

tobiasdiez · 2025-12-22T12:01:37Z

@charliermarsh Do you have any update on how to use build isolation for the main project? We have migrated a few projects to meson recently, and every single one of them faces the problem that one has to manually install meson-python+cython using uv pip.

charliermarsh added the documentation Improvements or additions to documentation label Aug 16, 2025

charliermarsh force-pushed the charlie/build-docs branch from 72a475a to 15ccd3f Compare August 16, 2025 16:02

charliermarsh requested a review from zanieb August 16, 2025 16:07

charliermarsh marked this pull request as ready for review August 16, 2025 16:07

charliermarsh force-pushed the charlie/build-docs branch from 15ccd3f to 5df7509 Compare August 16, 2025 16:29

charliermarsh mentioned this pull request Aug 16, 2025

Two-stage uv sync is annoying to use in pytorch projects #10694

Closed

vwxyzjn reviewed Aug 16, 2025

View reviewed changes

Document improvements to build-isolation setups

602d02f

charliermarsh force-pushed the charlie/build-docs branch from 5df7509 to 602d02f Compare August 16, 2025 21:53

zanieb approved these changes Aug 18, 2025

View reviewed changes

zanieb merged commit fa3c20a into main Aug 18, 2025
82 checks passed

zanieb deleted the charlie/build-docs branch August 18, 2025 21:15

BrewTestBot mentioned this pull request Aug 19, 2025

uv 0.8.12 Homebrew/homebrew-core#234044

Merged

zsol reviewed Aug 20, 2025

View reviewed changes

tobiasdiez reviewed Sep 13, 2025

View reviewed changes

	that changes to these settings will not trigger a reinstall and rebuild of the affected packages.
	that changes to these settings will trigger a reinstall and rebuild of the affected packages.

Comments

Conversation

charliermarsh commented Aug 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

vwxyzjn left a comment

Choose a reason for hiding this comment

Uh oh!

vwxyzjn Aug 16, 2025

Choose a reason for hiding this comment

Uh oh!

charliermarsh Aug 16, 2025

Choose a reason for hiding this comment

Uh oh!

vwxyzjn commented Aug 16, 2025

Uh oh!

zanieb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tobiasdiez commented Aug 19, 2025

Uh oh!

charliermarsh commented Aug 19, 2025

Uh oh!

zsol Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

charliermarsh Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

zsol Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

tobiasdiez Sep 13, 2025

Choose a reason for hiding this comment

Uh oh!

tobiasdiez commented Sep 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tobiasdiez commented Dec 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

charliermarsh commented Aug 16, 2025 •

edited

Loading

tobiasdiez commented Sep 13, 2025 •

edited

Loading