Normalize `platform_system` to `sys_platform` by charliermarsh · Pull Request #9949 · astral-sh/uv

charliermarsh · 2024-12-16T23:46:10Z

Summary

A revival of an old idea (#9344) that I have slightly more confidence in now. I abandoned this idea because (1) it couldn't capture that, e.g., platform_system == 'Windows' and sys_platform == 'foo' (or some other unknown value) are disjoint, and (2) I thought that Android returned "android" for one of sys_platform or platform_system, which would've made this logic incorrect.

However, it looks like Android... doesn't do that? And the values here are almost always in a small, known set. So in the end, the tradeoffs here actually seem pretty good.

Vis-a-vis our current solution, this can (e.g.) simplify out expressions like sys_platform == 'win32' or platform_system == 'Windows'.

zanieb · 2024-12-17T00:02:17Z

crates/uv/tests/it/pip_compile.rs

    click==8.1.7
        # via flask
-    colorama==0.4.6 ; platform_system == 'Windows' and sys_platform != 'darwin'
+    colorama==0.4.6 ; sys_platform == 'win32'


zanieb

I'm into this. Some churn for outputs though... I wonder if we should hold off on this until another minor version? (though these are less meaningful than some of the other resolution changes we've been making...)

charliermarsh · 2024-12-17T00:29:48Z

I'm somewhat in favor of just shipping it, since it won't invalidate existing lockfiles.

samypr100 · 2024-12-17T02:27:46Z

I thought that Android returned "android"

Note, per PEP-738, it will return "android" starting in 3.13. Similar as "ios" in 3.13 for iPhones per PEP-730

charliermarsh · 2024-12-17T02:34:07Z

Thank you! Luckily it looks like it will also return "Android" for platform.system(), phew.

konstin · 2024-12-17T11:33:43Z

crates/uv-pep508/src/marker/algebra.rs

-                Edges::from_string(operator, value),
-            ),
+            } => {
+                // Normalize `platform_system` markers to `sys_platform` nodes.


I would feel more confident if we could codify this upstream (packaging.python.org probably), so uv doesn't end up making assumptions that python redistributors don't make.

Can you open a convo about it?

Also, do we have any context on why there is platform_system and sys_platform?

And maybe it's obvious, but I think a comment saying why we normalize to sys_platform (instead of the other way around) would be good too.

https://discuss.python.org/t/clarify-usage-of-platform-system/70900 from Konsti's prompt in the PyPA Discord

BurntSushi

Nice!

BurntSushi · 2024-12-17T15:38:59Z

crates/uv-pep508/src/marker/algebra.rs

-                Edges::from_string(operator, value),
-            ),
+            } => {
+                // Normalize `platform_system` markers to `sys_platform` nodes.


Also, do we have any context on why there is platform_system and sys_platform?

BurntSushi · 2024-12-17T15:39:39Z

crates/uv-pep508/src/marker/algebra.rs

-                Edges::from_string(operator, value),
-            ),
+            } => {
+                // Normalize `platform_system` markers to `sys_platform` nodes.


And maybe it's obvious, but I think a comment saying why we normalize to sys_platform (instead of the other way around) would be good too.

BurntSushi

(Meant to approve.)

charliermarsh · 2024-12-17T17:17:08Z

From the PyPA Discord, it sounds like this isn't really safe to assume in all cases:

I gave the example of iPad, but looking at the docs, but some Unix OSes will have the major version number appended in sys.platform... in fact, that should be the case for FreeBSD, NetBSD, OpenBSD, and SunOS in your example

It might be fine for macOS, Windows, and Linux though...? The downside is that the fewer values we rewrite, the more we risk messing up when we see a marker like platform_system == "Darwin" and platform_system == "FreeBSD", since we'd then rewrite that to sys_platform == "darwin" and platform_system == "FreeBSD", and no longer realize they're disjoint.

charliermarsh · 2024-12-18T03:15:07Z

Maybe we run with this for macOS, Linux, Windows, and any other platforms that don't encode a version number.

Then, we can use our existing "incompatible markers" infrastructure to mark those other platforms as incompatible with these sys_platform values? Like, we can mark platform_system == "FreeBSD" as incompatible with all of these sys_platform values.

charliermarsh · 2024-12-18T15:03:26Z

Ok, I did a second pass on this with more care around known semantics. I think it's good to go. Our strategy has two parts: (1) we normalize platform_system to sys_platform when we can; and (2) when we can't, we encode as many known incompatibilities as possible. (1) is preferable to (2), but (1) isn't always possible.

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.5.8` -> `0.5.11` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.5.11`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0511) [Compare Source](astral-sh/uv@0.5.10...0.5.11) ##### Enhancements - Normalize `platform_system` to `sys_platform` ([#9949](astral-sh/uv#9949)) - Improve retry mechanisms on Windows for `copy_atomic` and `write_atomic` ([#10026](astral-sh/uv#10026)) - Add nuance to prefetch logging ([#9984](astral-sh/uv#9984)) - Update to [`python-build-standalone 20241219`](https://github.com/astral-sh/python-build-standalone/releases/tag/20241219) ##### Preview features - Build backend: Preserve executable bits for scripts in distributions ([#10027](astral-sh/uv#10027)) - Build backend: Handle case where `metadata_directory` already contains `dist-info` directory ([#10005](astral-sh/uv#10005)) ##### Performance - Batch resolver pre-fetches per fork ([#10029](astral-sh/uv#10029)) ##### Bug fixes - Allow `--script` to be provided with `uv run -` ([#10035](astral-sh/uv#10035)) - Allow `uv run` arguments when reading from `stdin` ([#10034](astral-sh/uv#10034)) - Prefer higher Python lower-bounds when forking ([#10007](astral-sh/uv#10007)) - Remove references to deprecated `first-match` ([#10036](astral-sh/uv#10036)) ##### Documentation - Add `uv python install --preview` to the documentation ([#10010](astral-sh/uv#10010)) - Fix `uv python install --default` note about multiple requests ([#10011](astral-sh/uv#10011)) - Fix typo in Caching docs ([#10032](astral-sh/uv#10032)) - Remove remaining references to deprecated `first-match` ([#10038](astral-sh/uv#10038)) - Supplement missing separators for `UV_INSTALL_DIR` directions on Windows ([#9507](astral-sh/uv#9507)) ### [`v0.5.10`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0510) [Compare Source](astral-sh/uv@0.5.9...0.5.10) ##### Enhancements - Improve backtracking behavior when packages conflict repeatedly ([#9843](astral-sh/uv#9843)) - Patch Python `sysconfig` values such as `AR` at `ar` install time ([#9905](astral-sh/uv#9905)) - Patch Python `sysconfig` values such as `clang` to `cc` at install time ([#9916](astral-sh/uv#9916)) - Skip `--native-tls` in `pip compile` header ([#9913](astral-sh/uv#9913)) - Add resolver error hint for no-binary and no-build failures ([#9948](astral-sh/uv#9948)) - Improve build error messages ([#9660](astral-sh/uv#9660)) - Reduce redundant Python version incompatibilities in resolver error message ([#9957](astral-sh/uv#9957)) - Reduce redundant enumeration of all package versions in some resolver errors ([#9885](astral-sh/uv#9885)) - Improve display of ranges when pre-releases are not allowed ([#9944](astral-sh/uv#9944)) - Improve error messages for `uv remove` ([#9959](astral-sh/uv#9959)) - Improve phrasing for single term incompatibilities ([#9953](astral-sh/uv#9953)) - Improve styling of `uv remove` dependency hints ([#9960](astral-sh/uv#9960)) - Omit trailing zeros on Python requirements inferred from versions ([#9952](astral-sh/uv#9952)) - Show a concise error message for missing `version` field ([#9912](astral-sh/uv#9912)) - Use the build options value to improve hints for no wheel / source distribution errors ([#9950](astral-sh/uv#9950)) ##### Bug fixes - Allow multiple disjoint URLs in overrides ([#9893](astral-sh/uv#9893)) - Include explicit indexes in publish index choice ([#9932](astral-sh/uv#9932)) - Fix Python interpreter detection for 32-bit operating systems on 64-bit hosts ([#9970](astral-sh/uv#9970)) ##### Documentation - Fix typo "operation system" ([#9971](astral-sh/uv#9971)) - Clarify uninstallation docs ([#9938](astral-sh/uv#9938)) - Add a note to say that dependencies between workspace members are editable ([#9363](astral-sh/uv#9363)) - Correctly document default value of `fork-strategy` setting ([#9931](astral-sh/uv#9931)) - Use double quotes for Windows support in examples ([#9946](astral-sh/uv#9946)) - Remove `pypy` from top-level pin example ([#9896](astral-sh/uv#9896)) - Update references to `python-build-standalone` to reflect the transferred project ([#9977](astral-sh/uv#9977)) - Use a different Ruff version in documentation ([#9943](astral-sh/uv#9943)) - Change example so it works as-is on `powershell` and `cmd.exe` ([#9903](astral-sh/uv#9903)) - Clarify best practice for Python matrix strategy in GitHub Actions ([#9454](astral-sh/uv#9454)) - Add documentation for `uv-lock` and `uv-export` pre-commit hooks ([#9872](astral-sh/uv#9872)) ##### Preview features - Build backend: Fix pre-PEP 639 license files ([#9965](astral-sh/uv#9965)) ### [`v0.5.9`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#059) [Compare Source](astral-sh/uv@0.5.8...0.5.9) ##### Enhancements - Fork version selection based on `requires-python` requirements ([#9827](astral-sh/uv#9827)) - Patch `sysconfig` data at install time ([#9857](astral-sh/uv#9857)) - Remove `-isysroot` when patching sysconfig ([#9860](astral-sh/uv#9860)) ##### Configuration - Introduce a `--fork-strategy` preference mode ([#9868](astral-sh/uv#9868)) - Add support for `UV_OFFLINE` ([#9795](astral-sh/uv#9795)) ##### Bug fixes - Avoid `panic!()` when current directory does not exist ([#9876](astral-sh/uv#9876)) - Avoid reusing interpreter metadata when running under Rosetta ([#9846](astral-sh/uv#9846)) - Avoid trailing slash when deserializing from lockfile ([#9848](astral-sh/uv#9848)) - Fix bug in terms when collapsing unavailable versions in resolver errors ([#9877](astral-sh/uv#9877)) - Fix suggestion to use `uv help python` on invalid install requests ([#9820](astral-sh/uv#9820)) - Skip root when assessing prefix viability ([#9823](astral-sh/uv#9823)) - Avoid spurious 'Upgraded tool environment' in `uv tool upgrade` ([#9870](astral-sh/uv#9870)) ##### Rust API - Upgrade minimum Rust version to 1.83 ([#9815](astral-sh/uv#9815)) ##### Documentation - Document the `--fork-strategy` setting ([#9887](astral-sh/uv#9887)) ##### Preview features - Build backend: Allow underscores in entrypoints ([#9825](astral-sh/uv#9825)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

zanieb reviewed Dec 17, 2024

View reviewed changes

zanieb approved these changes Dec 17, 2024

View reviewed changes

charliermarsh force-pushed the charlie/normalize branch from f6a01a4 to d3a6fec Compare December 17, 2024 00:17

charliermarsh force-pushed the charlie/normalize branch from d3a6fec to d730923 Compare December 17, 2024 02:40

charliermarsh requested review from BurntSushi and konstin December 17, 2024 02:40

charliermarsh force-pushed the charlie/normalize branch 2 times, most recently from 04d2dd5 to 64efddb Compare December 17, 2024 02:55

konstin approved these changes Dec 17, 2024

View reviewed changes

BurntSushi reviewed Dec 17, 2024

View reviewed changes

BurntSushi approved these changes Dec 17, 2024

View reviewed changes

charliermarsh force-pushed the charlie/normalize branch from 64efddb to b3aa56e Compare December 18, 2024 15:02

charliermarsh added 2 commits December 18, 2024 10:15

Normalize platform_system to sys_platform

86b3246

Amend

ebb68fc

charliermarsh force-pushed the charlie/normalize branch from b3aa56e to ebb68fc Compare December 18, 2024 15:17

charliermarsh merged commit dd760ee into main Dec 18, 2024

charliermarsh deleted the charlie/normalize branch December 18, 2024 15:29

charliermarsh mentioned this pull request Dec 18, 2024

Add symmetrical expressions to mutual incompatibilities #9945

Closed

BrewTestBot mentioned this pull request Dec 20, 2024

uv 0.5.11 Homebrew/homebrew-core#201865

Merged

Comments

Conversation

charliermarsh commented Dec 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

zanieb Dec 17, 2024

Choose a reason for hiding this comment

Uh oh!

zanieb left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

charliermarsh commented Dec 17, 2024

Uh oh!

samypr100 commented Dec 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

charliermarsh commented Dec 17, 2024

Uh oh!

konstin Dec 17, 2024

Choose a reason for hiding this comment

Uh oh!

charliermarsh Dec 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BurntSushi Dec 17, 2024

Choose a reason for hiding this comment

Uh oh!

BurntSushi Dec 17, 2024

Choose a reason for hiding this comment

Uh oh!

zanieb Dec 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BurntSushi left a comment

Choose a reason for hiding this comment

Uh oh!

BurntSushi Dec 17, 2024

Choose a reason for hiding this comment

Uh oh!

BurntSushi Dec 17, 2024

Choose a reason for hiding this comment

Uh oh!

BurntSushi left a comment

Choose a reason for hiding this comment

Uh oh!

charliermarsh commented Dec 17, 2024

Uh oh!

charliermarsh commented Dec 18, 2024

Uh oh!

charliermarsh commented Dec 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

charliermarsh commented Dec 16, 2024 •

edited

Loading

zanieb left a comment •

edited

Loading

samypr100 commented Dec 17, 2024 •

edited

Loading

charliermarsh Dec 17, 2024 •

edited

Loading

zanieb Dec 17, 2024 •

edited

Loading