Respect `--no-build-isolation-package` in `uv sync` by charliermarsh · Pull Request #6605 · astral-sh/uv

charliermarsh · 2024-08-25T13:59:29Z

Summary

This was an oversight. The existing test was (correctly) failing, but for the wrong reason (failing to build the package during resolution).

karimknaebel · 2024-08-25T18:12:57Z

Still doesn't work for me. --no-build-isolation works, --no-build-isolation-package doesn't. Not only with sync, but also add, pip, and config files.

➜ uv --version
uv 0.3.3 (1c580723c 2024-08-25)

➜ cat pyproject.toml
[project]
name = "6605"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
    "source-distribution @ https://files.pythonhosted.org/packages/10/1f/57aa4cce1b1abf6b433106676e15f9fa2c92ed2bd4cf77c3b50a9e9ac773/source_distribution-0.0.1.tar.gz",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

➜ uv sync
Resolved 2 packages in 2ms
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: 6605 @ file:///tmp/6605
  Caused by: Build backend failed to build wheel through `build_editable()` with exit status: 1
--- stdout:

--- stderr:
Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "/home/abou_zeid/.cache/uv/builds-v0/.tmp10Vbaz/lib/python3.12/site-packages/hatchling/build.py", line 83, in build_editable
    return os.path.basename(next(builder.build(directory=wheel_directory, versions=['editable'])))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/abou_zeid/.cache/uv/builds-v0/.tmp10Vbaz/lib/python3.12/site-packages/hatchling/builders/plugin/interface.py", line 90, in build
    self.metadata.validate_fields()
  File "/home/abou_zeid/.cache/uv/builds-v0/.tmp10Vbaz/lib/python3.12/site-packages/hatchling/metadata/core.py", line 266, in validate_fields
    self.core.validate_fields()
  File "/home/abou_zeid/.cache/uv/builds-v0/.tmp10Vbaz/lib/python3.12/site-packages/hatchling/metadata/core.py", line 1376, in validate_fields
    getattr(self, attribute)
  File "/home/abou_zeid/.cache/uv/builds-v0/.tmp10Vbaz/lib/python3.12/site-packages/hatchling/metadata/core.py", line 1230, in dependencies
    self._dependencies = list(self.dependencies_complex)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/abou_zeid/.cache/uv/builds-v0/.tmp10Vbaz/lib/python3.12/site-packages/hatchling/metadata/core.py", line 1215, in dependencies_complex
    raise ValueError(message)
ValueError: Dependency #1 of field `project.dependencies` cannot be a direct reference unless field `tool.hatch.metadata.allow-direct-references` is set to `true`
---

➜ uv sync --no-build-isolation-package source-distribution
Resolved 2 packages in 2ms
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: 6605 @ file:///tmp/6605
  Caused by: Build backend failed to build wheel through `build_editable()` with exit status: 1
--- stdout:

--- stderr:
Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "/home/abou_zeid/.cache/uv/builds-v0/.tmpUBzU1v/lib/python3.12/site-packages/hatchling/build.py", line 83, in build_editable
    return os.path.basename(next(builder.build(directory=wheel_directory, versions=['editable'])))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/abou_zeid/.cache/uv/builds-v0/.tmpUBzU1v/lib/python3.12/site-packages/hatchling/builders/plugin/interface.py", line 90, in build
    self.metadata.validate_fields()
  File "/home/abou_zeid/.cache/uv/builds-v0/.tmpUBzU1v/lib/python3.12/site-packages/hatchling/metadata/core.py", line 266, in validate_fields
    self.core.validate_fields()
  File "/home/abou_zeid/.cache/uv/builds-v0/.tmpUBzU1v/lib/python3.12/site-packages/hatchling/metadata/core.py", line 1376, in validate_fields
    getattr(self, attribute)
  File "/home/abou_zeid/.cache/uv/builds-v0/.tmpUBzU1v/lib/python3.12/site-packages/hatchling/metadata/core.py", line 1230, in dependencies
    self._dependencies = list(self.dependencies_complex)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/abou_zeid/.cache/uv/builds-v0/.tmpUBzU1v/lib/python3.12/site-packages/hatchling/metadata/core.py", line 1215, in dependencies_complex
    raise ValueError(message)
ValueError: Dependency #1 of field `project.dependencies` cannot be a direct reference unless field `tool.hatch.metadata.allow-direct-references` is set to `true`
---

➜ uv sync --no-build-isolation
Resolved 2 packages in 3ms
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: 6605 @ file:///tmp/6605
  Caused by: Build backend failed to build wheel through `build_editable()` with exit status: 1
--- stdout:

--- stderr:
Traceback (most recent call last):
  File "<string>", line 8, in <module>
ModuleNotFoundError: No module named 'hatchling'
---

charliermarsh · 2024-08-25T20:04:10Z

This is not necessarily incorrect... If you set --no-build-isolation, then the failure is probably a failure to build 6605 itself due to missing hatchling. But if you set --no-build-isolation-package source-distribution, then the failure is in building 6605 but due to an error in how you've packaged it, so it never even gets to the part where it needs to build source-distribution.

charliermarsh · 2024-08-25T20:04:35Z

Here's a working example:

[project]
name = "foo"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
    "source-distribution @ https://files.pythonhosted.org/packages/10/1f/57aa4cce1b1abf6b433106676e15f9fa2c92ed2bd4cf77c3b50a9e9ac773/source_distribution-0.0.1.tar.gz"
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.metadata]
allow-direct-references = true

❯ cargo run sync --no-build-isolation-package source-distribution
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.15s
     Running `/Users/crmarsh/workspace/uv/target/debug/uv sync --no-build-isolation-package source-distribution`
Resolved 2 packages in 8ms
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: source-distribution @ https://files.pythonhosted.org/packages/10/1f/57aa4cce1b1abf6b433106676e15f9fa2c92ed2bd4cf77c3b50a9e9ac773/source_distribution-0.0.1.tar.gz
  Caused by: Build backend failed to build wheel through `build_wheel()` with exit status: 1
--- stdout:

--- stderr:
Traceback (most recent call last):
  File "<string>", line 8, in <module>
ModuleNotFoundError: No module named 'hatchling'
---

Though I'd recommend using tool.uv.sources instead so you don't have to fight hatchling:

[project]
name = "foo"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.12"
dependencies = ["source-distribution"]

[tool.uv.sources]
source-distribution = { url = "https://files.pythonhosted.org/packages/10/1f/57aa4cce1b1abf6b433106676e15f9fa2c92ed2bd4cf77c3b50a9e9ac773/source_distribution-0.0.1.tar.gz" }

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

karimknaebel · 2024-08-26T08:00:09Z

When I add

[tool.hatch.metadata]
allow-direct-references = true

from your example, it always builds successfully no matter what, even with build isolation enabled.

I still could not find a single case where --no-build-isolation-package did have any effect at all.

tdejager · 2024-08-26T08:03:55Z

@charliermarsh I made some similar changes in: #6517 where I also found that the options were not passed around correctly. I think the changes are the same and I'll merge this PR in there. I think the other changes would still be relevant, I hope.

charliermarsh · 2024-08-26T12:23:24Z

@kabouzeid -- Are you running from source? None of those changes are released yet. Given:

[project]
name = "foo"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
    "source-distribution @ https://files.pythonhosted.org/packages/10/1f/57aa4cce1b1abf6b433106676e15f9fa2c92ed2bd4cf77c3b50a9e9ac773/source_distribution-0.0.1.tar.gz"
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.metadata]
allow-direct-references = true

Here's my running the exact sequence of commands -- not sure how else to help!

❯ cargo run sync --no-build-isolation-package source-distribution
Using Python 3.12.1
Creating virtualenv at: .venv
Resolved 2 packages in 13ms
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: source-distribution @ https://files.pythonhosted.org/packages/10/1f/57aa4cce1b1abf6b433106676e15f9fa2c92ed2bd4cf77c3b50a9e9ac773/source_distribution-0.0.1.tar.gz
  Caused by: Build backend failed to build wheel through `build_wheel()` with exit status: 1
--- stdout:

--- stderr:
Traceback (most recent call last):
  File "<string>", line 8, in <module>
ModuleNotFoundError: No module named 'hatchling'
---

❯ cargo run sync
Resolved 2 packages in 7ms
   Built foo @ file:///Users/crmarsh/workspace/puffin/foo
   Built source-distribution @ https://files.pythonhosted.org/packages/10/1f/57aa4cce1b1abf6b433106676e15f9fa2c92ed2bd4cf77c3b50a9e9ac773/source_distribution-0.0.1.tar.gz
Prepared 2 packages in 400ms
Installed 2 packages in 1ms
 + foo==0.1.0 (from file:///Users/crmarsh/workspace/puffin/foo)
 + source-distribution==0.0.1 (from https://files.pythonhosted.org/packages/10/1f/57aa4cce1b1abf6b433106676e15f9fa2c92ed2bd4cf77c3b50a9e9ac773/source_distribution-0.0.1.tar.gz)

karimknaebel · 2024-08-26T13:36:22Z

Yes from source. Commit is shown in uv --version of my console log above. I will check again later today.

charliermarsh · 2024-08-26T13:37:09Z

Do you have the build cached?

karimknaebel · 2024-08-26T17:01:59Z

No, I also tried with export UV_NO_CACHE=1 during the whole time to be sure. I'm not sure we were talking about the same thing, because I don't understand the example.

However, I just tried, and it seems that #6517 fixes my issue.

charliermarsh · 2024-08-26T17:18:35Z

Ok. I'm very confused, because it's the exact same example that you provided in #6605 (comment). And that change should have absolutely no effect on the source-distribution package. Were you testing with a different package?

karimknaebel · 2024-08-26T17:45:15Z

I pulled that example out of the uv tests because I assumed something must be wrong with the tests as well, since they don't catch the bug. I tested with a few different packages, and also with source-distribution. I don't really know what source-distribution and allow-direct-references = true do. In the example I posted --no-build-isolation-package had no effect. Maybe you tried with a newer commit, because I got the same result as you with the newest commit uv 0.3.3 (622053237 2024-08-26).

For the other packages (eg flash-attn and minkowskiengine), it works now with --no-build-isolation-package , while it had no effect before. This PR didn't fix the issue for me, but it did work with uv 0.3.3 (622053237 2024-08-26).

charliermarsh · 2024-08-26T17:48:42Z

Okay, thanks. It sounds like you were testing with a different package then. (source-distribution is just a PyPI package that we publish to test source distributions, like requests or any other PyPI package.)

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.3.2` -> `0.3.5` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.3.5`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#035) [Compare Source](astral-sh/uv@0.3.4...0.3.5) ##### Enhancements - Add support for `--allow-insecure-host` (aliased to `--trusted-host`) ([#6591](astral-sh/uv#6591)) - Read requirements from `requires.txt` when available ([#6655](astral-sh/uv#6655)) - Respect `tool.uv.environments` in `pip compile --universal` ([#6663](astral-sh/uv#6663)) - Use relative paths by default in `uv add` ([#6686](astral-sh/uv#6686)) - Improve messages for empty solves and installs ([#6588](astral-sh/uv#6588)) ##### Bug fixes - Avoid reusing state across tool upgrades ([#6660](astral-sh/uv#6660)) - Detect musl and error for musl Python builds ([#6643](astral-sh/uv#6643)) - Ignore `send` errors in installer ([#6667](astral-sh/uv#6667)) ##### Documentation - Add development section to Docker guide and reference new example project ([#6666](astral-sh/uv#6666)) - Add docs for `constraint-dependencies` and `override-dependencies` ([#6596](astral-sh/uv#6596)) - Clarify package priority order in pip compatibility guide ([#6619](astral-sh/uv#6619)) - Fix docs for disabling build isolation with `uv sync` ([#6674](astral-sh/uv#6674)) - Improve consistency of directory lookup instructions in Docker ([#6665](astral-sh/uv#6665)) - Improve lockfile concept documentation, add coverage for upgrades ([#6698](astral-sh/uv#6698)) - Shift the order of some of the Docker guide content ([#6664](astral-sh/uv#6664)) - Use `python` to highlight requirements and use more content tabs ([#6549](astral-sh/uv#6549)) ### [`v0.3.4`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#034) [Compare Source](astral-sh/uv@0.3.3...0.3.4) ##### CLI - Show `--editable` on the `uv add` CLI ([#6608](astral-sh/uv#6608)) - Add `--refresh` to `tool run` warning for `--with` dependencies ([#6609](astral-sh/uv#6609)) ##### Bug fixes - Allow per dependency build isolation for `setup.py`-based projects ([#6517](astral-sh/uv#6517)) - Avoid un-strict syncing by-default for build isolation ([#6606](astral-sh/uv#6606)) - Respect `--no-build-isolation-package` in `uv sync` ([#6605](astral-sh/uv#6605)) - Respect extras and markers on virtual dev dependencies ([#6620](astral-sh/uv#6620)) - Support PEP 723 scripts in GUI files ([#6611](astral-sh/uv#6611)) - Update lockfile after setting minimum bounds in `uv add` ([#6618](astral-sh/uv#6618)) - Use relative paths for `--find-links` and local registries ([#6566](astral-sh/uv#6566)) - Use separate types to represent raw vs. resolver markers ([#6646](astral-sh/uv#6646)) - Parse wheels `WHEEL` and `METADATA` files as email messages ([#6616](astral-sh/uv#6616)) - Support unquoted hrefs in `--find-links` and other HTML sources ([#6622](astral-sh/uv#6622)) - Don't canonicalize paths to user requirements ([#6560](astral-sh/uv#6560)) ##### Documentation - Add FastAPI guide to overview ([#6603](astral-sh/uv#6603)) - Add docs for disabling build isolation with `uv sync` ([#6607](astral-sh/uv#6607)) - Add example of reading script from stdin using echo ([#6567](astral-sh/uv#6567)) - Add tip to use intermediate layers in Docker builds ([#6650](astral-sh/uv#6650)) - Clarify need to include `pyproject.toml` with `--no-install-project` ([#6581](astral-sh/uv#6581)) - Move `WORKDIR` directive in Docker examples ([#6652](astral-sh/uv#6652)) - Remove duplicate `WORKDIR` directive in Docker example ([#6651](astral-sh/uv#6651)) ### [`v0.3.3`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#033) [Compare Source](astral-sh/uv@0.3.2...0.3.3) ##### Enhancements - Add `uv sync --no-install-project` to skip installation of the project ([#6538](astral-sh/uv#6538)) - Add `uv sync --no-install-workspace` to skip installation of all workspace members ([#6539](astral-sh/uv#6539)) - Add `uv sync --no-install-package` to skip installation of specific packages ([#6540](astral-sh/uv#6540)) - Show previous version in self update message ([#6473](astral-sh/uv#6473)) ##### CLI - Add `--no-project` alias for `uv python pin --no-workspace` ([#6514](astral-sh/uv#6514)) - Ignore `.python-version` files in `uv venv` with `--no-config` ([#6513](astral-sh/uv#6513)) - Include virtual environment interpreters in `uv python find` ([#6521](astral-sh/uv#6521)) - Respect `-` as stdin channel for `uv run` ([#6481](astral-sh/uv#6481)) - Revert changes to pyproject.toml when sync fails duing `uv add` ([#6526](astral-sh/uv#6526)) ##### Configuration - Add `UV_COMPILE_BYTECODE` environment variable ([#6530](astral-sh/uv#6530)) ##### Bug fixes - Set `VIRTUAL_ENV` for `uv run` invocations ([#6543](astral-sh/uv#6543)) - Ignore errors in workspace discovery with `--no-project` ([#6554](astral-sh/uv#6554)) ##### Documentation - Add documentation for `uv python find` ([#6527](astral-sh/uv#6527)) - Add uv tool install example in Docker ([#6547](astral-sh/uv#6547)) - Document why we do lower bounds ([#6516](astral-sh/uv#6516)) - Fix to miss string termination in PowerShell commands for shell autocompletion documentation ([#6491](astral-sh/uv#6491)) - Fix incorrect workspace members keyword ([#6502](astral-sh/uv#6502)) - Use proper environment variables for Windows ([#6433](astral-sh/uv#6433)) - Improve caveat in `uvx` note ([#6546](astral-sh/uv#6546)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

charliermarsh added the bug Something isn't working label Aug 25, 2024

charliermarsh force-pushed the charlie/pkg branch from 411db37 to ceb65ff Compare August 25, 2024 14:06

Respect --no-build-isolation-package in uv sync

af5e430

charliermarsh force-pushed the charlie/pkg branch from ceb65ff to af5e430 Compare August 25, 2024 14:08

charliermarsh enabled auto-merge (squash) August 25, 2024 14:08

charliermarsh merged commit d0198ab into main Aug 25, 2024

charliermarsh deleted the charlie/pkg branch August 25, 2024 14:15

charliermarsh mentioned this pull request Aug 25, 2024

Add docs for disabling build isolation with uv sync #6607

Merged

tdejager mentioned this pull request Aug 26, 2024

Allow per dependency build isolation for setup.py projects as well #6517

Merged

BrewTestBot mentioned this pull request Aug 26, 2024

uv 0.3.4 Homebrew/homebrew-core#182564

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Respect `--no-build-isolation-package` in `uv sync`#6605

Respect `--no-build-isolation-package` in `uv sync`#6605
charliermarsh merged 1 commit intomainfrom
charlie/pkg

charliermarsh commented Aug 25, 2024 •

edited

Loading

Uh oh!

karimknaebel commented Aug 25, 2024

Uh oh!

charliermarsh commented Aug 25, 2024 •

edited

Loading

Uh oh!

charliermarsh commented Aug 25, 2024 •

edited

Loading

Uh oh!

karimknaebel commented Aug 26, 2024

Uh oh!

tdejager commented Aug 26, 2024

Uh oh!

charliermarsh commented Aug 26, 2024

Uh oh!

karimknaebel commented Aug 26, 2024 •

edited

Loading

Uh oh!

charliermarsh commented Aug 26, 2024

Uh oh!

karimknaebel commented Aug 26, 2024

Uh oh!

charliermarsh commented Aug 26, 2024

Uh oh!

karimknaebel commented Aug 26, 2024

Uh oh!

charliermarsh commented Aug 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments

Conversation

charliermarsh commented Aug 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

karimknaebel commented Aug 25, 2024

Uh oh!

charliermarsh commented Aug 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

charliermarsh commented Aug 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

karimknaebel commented Aug 26, 2024

Uh oh!

tdejager commented Aug 26, 2024

Uh oh!

charliermarsh commented Aug 26, 2024

Uh oh!

karimknaebel commented Aug 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

charliermarsh commented Aug 26, 2024

Uh oh!

karimknaebel commented Aug 26, 2024

Uh oh!

charliermarsh commented Aug 26, 2024

Uh oh!

karimknaebel commented Aug 26, 2024

Uh oh!

charliermarsh commented Aug 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

charliermarsh commented Aug 25, 2024 •

edited

Loading

charliermarsh commented Aug 25, 2024 •

edited

Loading

charliermarsh commented Aug 25, 2024 •

edited

Loading

karimknaebel commented Aug 26, 2024 •

edited

Loading