Support conflict markers in `uv export` by charliermarsh · Pull Request #11643 · astral-sh/uv

charliermarsh · 2025-02-19T22:29:28Z

Summary

Today, if you have a lockfile that includes conflict markers, we write those markers out to requirements.txt in uv export. This is problematic, since no tool will ever evaluate those markers correctly downstream.

This PR adds handling for the conflict markers, though it's quite involved. Specifically, we have a new reachability algorithm that tracks, for each node, the reachable marker for that node and the marker conditions under which each conflict item is true (at that node).

I'm slightly worried that this algorithm could be wrong for graphs with cycles, but we only use this logic for lockfiles with conflicts anyway, so I think it's a strict improvement over the status quo.

Closes #11559.

Closes #11548.

BurntSushi

This LGTM. The comments in the code (especially the examples) were actually incredibly helpful for being able to follow what was going on.

One question I had here that I'm a little fuzzy on is: why is it okay to completely remove conflict markers from the output of uv export, but we can't do the same for uv.lock?

BurntSushi · 2025-02-20T15:19:42Z

crates/uv-resolver/src/lock/requirements_txt.rs

+        let mut conflicts = if target.lock().conflicts.is_empty() {
+            None
+        } else {
+            Some(FxHashMap::default())


It looks like MarkerTree::TRUE is the only thing that's ever stored as a value in this particular map? Or am I wrong about that? If so, I think you could use a FxHashSet here instead.

We end up cloning it when doing the conflict_marker_reachability thing, and then the markers can later vary, so this felt a bit easier.

charliermarsh · 2025-02-20T20:19:20Z

One question I had here that I'm a little fuzzy on is: why is it okay to completely remove conflict markers from the output of uv export, but we can't do the same for uv.lock?

I think the difference is: for uv export, we know the set of user-enabled extras and groups (whereas we don't for the lockfile). Like, uv export is invoked via uv export --extra cpu. The output isn't intended to work for arbitrary extras and groups; the set of enabled extras and groups is encoded at the time the user runs the command, unlike with the lockfile.

BurntSushi · 2025-02-21T12:10:37Z

The output isn't intended to work for arbitrary extras and groups

Ah! That's what I was missing. Thanks for explaining!

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.6.0` -> `0.6.3` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.6.3`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#063) [Compare Source](astral-sh/uv@0.6.2...0.6.3) ##### Enhancements - Allow quotes around command-line options in `requirement.txt files` ([#11644](astral-sh/uv#11644)) - Initialize PEP 723 script in `uv lock --script` ([#11717](astral-sh/uv#11717)) ##### Configuration - Accept multiple `.env` files in `UV_ENV_FILE` ([#11665](astral-sh/uv#11665)) ##### Performance - Reduce overhead in converting resolutions ([#11660](astral-sh/uv#11660)) - Use `SmallString` on `Hashes` ([#11756](astral-sh/uv#11756)) - Use a `Box` for `Yanked` on `File` ([#11755](astral-sh/uv#11755)) - Use a `SmallString` for the `Yanked` enum ([#11715](astral-sh/uv#11715)) - Use boxed slices for hash vector ([#11714](astral-sh/uv#11714)) - Use install concurrency for bytecode compilation too ([#11615](astral-sh/uv#11615)) ##### Bug fixes - Avoid installing duplicate dependencies across conflicting groups ([#11653](astral-sh/uv#11653)) - Check subdirectory existence after cache heal ([#11719](astral-sh/uv#11719)) - Include uppercase platforms for Windows wheels ([#11681](astral-sh/uv#11681)) - Respect existing PEP 723 script settings in `uv add` ([#11716](astral-sh/uv#11716)) - Reuse refined interpreter to create tool environment ([#11680](astral-sh/uv#11680)) - Skip removed directories during bytecode compilation ([#11633](astral-sh/uv#11633)) - Support conflict markers in `uv export` ([#11643](astral-sh/uv#11643)) - Treat lockfile as outdated if (empty) extras are added ([#11702](astral-sh/uv#11702)) - Display path separators as backslashes on Windows ([#11667](astral-sh/uv#11667)) - Display the built file name instead of the canonicalized name in `uv build` ([#11593](astral-sh/uv#11593)) - Fix message when there are no buildable packages ([#11722](astral-sh/uv#11722)) - Re-allow HTTP schemes for Git dependencies ([#11687](astral-sh/uv#11687)) ##### Documentation - Add anchor links to arguments and options in the CLI reference ([#11754](astral-sh/uv#11754)) - Add link to environment marker specification ([#11748](astral-sh/uv#11748)) - Fix missing a closing bracket in the `cache-keys` setting ([#11669](astral-sh/uv#11669)) - Remove the last edited date from documentation pages ([#11753](astral-sh/uv#11753)) - Fix readme typo ([#11742](astral-sh/uv#11742)) ### [`v0.6.2`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#062) [Compare Source](astral-sh/uv@0.6.1...0.6.2) ##### Enhancements - Add support for constraining build dependencies with `tool.uv.build-constraint-dependencies` ([#11585](astral-sh/uv#11585)) - Sort dependency group keys when adding new group ([#11591](astral-sh/uv#11591)) ##### Performance - Use an `Arc` for index URLs ([#11586](astral-sh/uv#11586)) ##### Bug fixes - Allow use of x86-64 Python on ARM Windows ([#11625](astral-sh/uv#11625)) - Fix an issue where conflict markers could instigate a very large lock file ([#11293](astral-sh/uv#11293)) - Fix duplicate packages with multiple conflicting extras declared ([#11513](astral-sh/uv#11513)) - Respect color settings for log messages ([#11604](astral-sh/uv#11604)) - Eagerly reject unsupported Git schemes ([#11514](astral-sh/uv#11514)) ##### Documentation - Add documentation for specifying Python versions in tool commands ([#11598](astral-sh/uv#11598)) ### [`v0.6.1`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#061) [Compare Source](astral-sh/uv@0.6.0...0.6.1) ##### Enhancements - Allow users to mark platforms as "required" for wheel coverage ([#10067](astral-sh/uv#10067)) - Warn for builds in non-build and workspace root pyproject.toml ([#11394](astral-sh/uv#11394)) ##### Bug fixes - Add `--all` to `uvx --reinstall` message ([#11535](astral-sh/uv#11535)) - Fallback to `GET` on HTTP 400 when attempting to use range requests for wheel download ([#11539](astral-sh/uv#11539)) - Prefer local variants in preference selection ([#11546](astral-sh/uv#11546)) - Respect verbatim executable name in `uvx` ([#11524](astral-sh/uv#11524)) ##### Documentation - Add documentation for required environments ([#11542](astral-sh/uv#11542)) - Note that `main.py` used to be `hello.py` ([#11519](astral-sh/uv#11519)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

## Summary Today, if you have a lockfile that includes conflict markers, we write those markers out to `requirements.txt` in `uv export`. This is problematic, since no tool will ever evaluate those markers correctly downstream. This PR adds handling for the conflict markers, though it's quite involved. Specifically, we have a new reachability algorithm that tracks, for each node, the reachable marker for that node _and_ the marker conditions under which each conflict item is `true` (at that node). I'm slightly worried that this algorithm could be wrong for graphs with cycles, but we only use this logic for lockfiles with conflicts anyway, so I think it's a strict improvement over the status quo. Closes astral-sh#11559. Closes astral-sh#11548.

charliermarsh added 4 commits February 19, 2025 13:40

Try to fix conflict markers with export

b7b3747

Drop markers

03166e9

Clean up

d42305c

Reverts

db561ce

charliermarsh added the bug Something isn't working label Feb 19, 2025

charliermarsh requested a review from BurntSushi February 19, 2025 22:29

BurntSushi approved these changes Feb 20, 2025

View reviewed changes

charliermarsh merged commit 5f6529a into main Feb 20, 2025
74 checks passed

charliermarsh deleted the charlie/ex branch February 20, 2025 20:19

BrewTestBot mentioned this pull request Feb 24, 2025

uv 0.6.3 Homebrew/homebrew-core#208817

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Support conflict markers in `uv export`#11643

Support conflict markers in `uv export`#11643
charliermarsh merged 4 commits intomainfrom
charlie/ex

charliermarsh commented Feb 19, 2025 •

edited

Loading

Uh oh!

BurntSushi left a comment

Uh oh!

BurntSushi Feb 20, 2025

Uh oh!

charliermarsh Feb 20, 2025

Uh oh!

charliermarsh commented Feb 20, 2025

Uh oh!

Uh oh!

BurntSushi commented Feb 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Conversation

charliermarsh commented Feb 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

BurntSushi left a comment

Choose a reason for hiding this comment

Uh oh!

BurntSushi Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

charliermarsh Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

charliermarsh commented Feb 20, 2025

Uh oh!

Uh oh!

BurntSushi commented Feb 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

charliermarsh commented Feb 19, 2025 •

edited

Loading