Hi @Affanmir, thanks for this contribution.

I have some questions about the background, motivation, and creation of this change. It is unusual to receive a PR that directly competes with an existing, pending PR by another contributor that is already going through code review. We like to keep things collegial and friendly here, and so in this case it might have been appropriate to first discuss your change with the author of that PR, and with the maintainers of the project.

It would be helpful if you could jump on Slack, introduce yourself there, and give us a bit of background about your use of Pants, your motivation for this change, and how you created it using Claude.

Thanks!

Hi!

As far as I can see, this PR solves all the issues related to UV for Pants. Can you tell me if there's anything I can do to help make sure these changes make it to the main branch?

Hi!

As far as I can see, this PR solves all the issues related to UV for Pants. Can you tell me if there's anything I can do to help make sure these changes make it to the main branch?

We are still reviewing this change, and considering whether this is the way to go WRT uv integration. We intend to get either something along the lines of this PR, or something that uses a different approach, merged soon.

Hi!
As far as I can see, this PR solves all the issues related to UV for Pants. Can you tell me if there's anything I can do to help make sure these changes make it to the main branch?

We are still reviewing this change, and considering whether this is the way to go WRT uv integration. We intend to get either something along the lines of this PR, or something that uses a different approach, merged soon.

We are going to discuss this at the monthly Pants developer meeting on Monday, which is a huddle in the #development channel on Slack, this Monday (4/13) at 1 PM PST, and all are welcome to join and participate!

will take a look this week in conjunction with our internal needs.

Hi @Affanmir , thanks for the contribution! The implementation looks reasonable to me.

Although for our specific needs, --universal is too permissive, from performance perspective, it can be wasteful, and from functional POV, not all wheels can be resolved in the universal sense.

So we would ask for uv to handle a matrix of, for example,

python_versions = [3.10, 3.11]
platforms = ["x86_64-manylinux_2_34", "aarch64-apple-darwin"]

I don't think uv handles that natively, so this will become a 2x2=4 uv calls, and then we need to synthesize the result.

@benjyw how do you feel about it? I am fine landing this one first, and then we can add our ask later. It should be complimentary/additive to this implementation.

Hi @Affanmir , thanks for the contribution! The implementation looks reasonable to me.

Although for our specific needs, --universal is too permissive, from performance perspective, it can be wasteful, and from functional POV, not all wheels can be resolved in the universal sense.

So we would ask for uv to handle a matrix of, for example,

python_versions = [3.10, 3.11]
platforms = ["x86_64-manylinux_2_34", "aarch64-apple-darwin"]

I don't think uv handles that natively, so this will become a 2x2=4 uv calls, and then we need to synthesize the result.

@benjyw how do you feel about it? I am fine landing this one first, and then we can add our ask later. It should be complimentary/additive to this implementation.

I'm not sure how we would synthesize the result (which is a fully-pinned requirements.txt). And for uv pip compile the --python-version is a lower bound, so you would not need python versions in the matrix anyway.

If we were using uv lock then the lockfile is always universal, in that it captures the available metadata for all markers. True, it doesn't guarantee that a package will later install on a certain platform (there may be no wheel for a given platform, and/or the sdist may not build on that platform). But this is not an issue at lockfile creation time. uv pip compile is not as well documented, but it seems reasonable to hypothesize that it's similar, since I have to assume that it's using uv lock under the hood. We'd need to check all this of course. But if so, what is the issue with --universal ? Do you have evidence of a performance issue?

I'm not sure how we would synthesize the result (which is a fully-pinned requirements.txt). And for uv pip compile the --python-version is a lower bound, so you would not need python versions in the matrix anyway.

If we were using uv lock then the lockfile is always universal, in that it captures the available metadata for all markers. True, it doesn't guarantee that a package will later install on a certain platform (there may be no wheel for a given platform, and/or the sdist may not build on that platform). But this is not an issue at lockfile creation time. uv pip compile is not as well documented, but it seems reasonable to hypothesize that it's similar, since I have to assume that it's using uv lock under the hood. We'd need to check all this of course. But if so, what is the issue with --universal ? Do you have evidence of a performance issue?

Ah I see. Thanks!

uv's output is just a requirements.txt, so perf impact should be small (if not constraining python versions and platforms)

The potential bigger perf impact is around pex turning the requirements.txt into the pex lock, iiuc it is going to download all applicable linux and mac wheels, which is more than ["x86_64-manylinux_2_34", "aarch64-apple-darwin"]

Ref:

❯ will pex genererate the pex lock for all platforms and python versions? how does relate to --universal?                                                                                     

  Searched for 1 pattern (ctrl+o to expand)

● Short answer: the universality scope is parallel between the two tools, but uv pins versions while PEX enumerates artifacts.

  uv's job (resolution):
  - --universal (lockfile.py:163-164) → uv picks one pinned version per package that works across the whole interpreter-constraint range and all platforms. That's what lets a single requirements.txt feed PEX.
  - Non-universal (strict/sources) → requires exactly one major.minor (lockfile.py:310-320); uv pins for just that Python.

  PEX's job (artifact enumeration): Even with --no-transitive --requirement=<uv output>, PEX still hits indexes to record hashes/URLs for each (platform × python) combination into the PEX lock. The scope is
  controlled by --style + --target-system + interpreter constraints (lockfile.py:246-265, 386-421):
  - lock_style="universal" → --target-system=linux --target-system=mac plus the full IC range. PEX records wheels/sdists for linux+mac across every Python in IC.
  - lock_style="strict"/"sources" without complete_platforms → no --target-system, PEX locks only for the current platform.
  - complete_platforms set → PEX uses those platform JSONs instead, and IC is suppressed (lockfile.py:411-415). This is rejected when uv is enabled (lockfile.py:221-225).

  So "universal" means the same thing in both stages — uv resolves once across the range, PEX then fans out artifact entries for linux+mac × IC.

❯ what is IC?  IC range                                                                                                                                                                                           

● IC = Interpreter Constraints — the Python-version range a resolve is allowed to run under (e.g. CPython>=3.9,<3.13).

  In this codebase: InterpreterConstraints (lockfile.py:21, 85), sourced from [python].interpreter_constraints or [python].resolves_to_interpreter_constraints[<resolve>].

  "IC range" = the set of Python major.minor versions that satisfy those constraints (e.g. 3.9, 3.10, 3.11, 3.12). For a universal lock, uv resolves one version per package that works across that whole range,
  and PEX records artifacts for every Python in the range × each target platform.

❯ is IC range configurable in this case?                                                                                                                                                                          

  Searched for 1 pattern (ctrl+o to expand)

● Yes — two configurable inputs shape the IC range:

  1. [python].interpreter_constraints (or [python].resolves_to_interpreter_constraints[<resolve>]) — the constraint itself (e.g. ">=3.9,<3.13"), consumed at lockfile.py:581-585.
  2. [python].interpreter_versions_universe (setup.py:161) — the set of major.minor versions Pants considers candidates when expanding those constraints. Default is all versions Pants supports; narrow it to
  shrink the range.

  How each lock_style uses them under uv:
  - universal (lockfile.py:303-307): only the minimum version in the range is passed to uv as --python-version. PEX then fans artifacts across the full IC × universe.
  - strict / sources (lockfile.py:310-320): the IC must partition to exactly one major.minor within the universe, or uv errors out. So effectively the IC range must be a single Python version.

❯ how about platform constraint when pex turns requirements.txt into a pex lock?                                                                                                                                  

  Read 1 file (ctrl+o to expand)

● When uv feeds PEX, the platform knobs are reduced to two hardcoded paths (lockfile.py:220-265):

  - complete_platforms — rejected outright. uv errors before PEX is invoked (lockfile.py:221-225). This is the usual "lock for these exact platforms" knob in Pants, and it's unavailable under uv.
  - lock_style="universal" → PEX gets --target-system=linux --target-system=mac, hardcoded. Windows is deliberately excluded. Not configurable.
  - lock_style="strict" or "sources" → no --target-system passed; PEX locks for the current machine's platform only (whatever OS/arch runs generate-lockfiles). Also not configurable.

  So under uv today, the only platform surface is: (a) universal = linux+mac fixed, or (b) strict/sources = "whatever box ran this." If you need per-resolve platform control, you'd have to switch that resolve
  back to the PEX resolver and use resolves_to_complete_platforms.

❯ are linux and x86_64-manylinux_2_34 separete, or the former includes the latter?                                                                                                                                

● They're different granularities, and linux includes x86_64-manylinux_2_34.

  - --target-system=linux is a broad OS-family filter PEX uses during universal resolution. It tells PEX "don't prune wheels that could run on some Linux" — it spans all arches (x86_64, aarch64, …) and all
  manylinux/musllinux variants.
  - x86_64-manylinux_2_34 (as seen in a complete-platform JSON or wheel tag) is a specific ABI point: one arch + one glibc floor + one Python ABI. It's one of many points linux covers.

  So in a universal lock built with --target-system=linux, PEX records artifacts for whichever Linux wheel tags exist on the index (typically several manylinux variants plus sdist fallback), not just 2_34. If
  you need to pin to exactly x86_64-manylinux_2_34, that's complete_platforms territory — which, again, this uv path rejects.

❯ so if i only want x86_64-manylinux_2_34, using 'linux' can be wasteful, right? since pex is going to download all the applicable linux wheels                                                                   

● Yes — in two ways, though the cost shape matters:

  1. Lock size / creation time: with --target-system=linux PEX records artifacts for every Linux wheel tag available on the index (x86_64 manylinux_2_17/2_28/2_34, aarch64, musllinux, sdists, etc.). Each
  becomes a locked_requirements entry with its own URL and hash. More artifacts to fetch metadata for at lock time and a larger lockfile on disk.
  2. Resolution surface: uv's universal resolver must also pick a version that satisfies all those tags (including, say, an aarch64-only constraint). Occasionally this forces an older pin than x86_64-manylinux
  alone would need.

  Lock-consumption time isn't wasteful — when you later build a PEX for a specific target, PEX only downloads the matching artifact.

  The lean answer is complete_platforms with an x86_64-manylinux_2_34 JSON — which is exactly what this PR disallows under uv (lockfile.py:221-225). So today, under the uv resolver, you eat the linux+mac
  fan-out or you switch that resolve back to the PEX resolver.

So point is that I think we can land this, but may need to follow up on adding a bit more platform constraints to help with perf.

So point is that I think we can land this, but may need to follow up on adding a bit more platform constraints to help with perf.

That may be, but also the LLM BS is BS. Back in October the 2.64.0 release had Pex defaulting to 0 downloads for modern Pips and indexes like PyPI that serve PEP-658 distribution metadata .

It would be really awesome if Pants folks stopped arm-chairing and actually got their hands dirty and did real work to suss all this out.

@wisechengyi FWIW, in your dialogue with the LLM, the LLM gets several other facts wrong. This is a super poor basis for decision making. Even without LLMs developers tend to be pro hand-wavers. You, traditionally, have not been one of these; so I find this worrying!

Was following this PR since it's related to #23197 and ran some benchmarks. Turns out the bottleneck is pex's artifact enumeration, not the resolve step.

Setup: 49 top-level requirements (requests, django, fastapi, boto3, pandas, etc.) → 137 resolved packages, --style=universal --target-system linux --target-system mac, CPython==3.11.*, uv 0.10.1, pex 2.93.0, hyperfine 1.20.0, macOS ARM64.

--no-transitive barely helps — ~98% of pex's time is artifact enumeration (wheel metadata/hash collection), not resolution:

pex3 lock create (warm):                 28.7s
pex3 lock create --no-transitive (warm): 28.2s  → only 0.5s saved

Cold cache (3 runs, cache cleared before each):

Warm cache (3 runs, warmup 1):

Both uv approaches resolve the same 137 packages at identical versions. Output is requirements.txt with hashes — Pants already supports consuming this format via --requirement + --no-transitive in _setup_pex_requirements.

Reproduction:

hyperfine --runs 3 --warmup 1 \
  -n "pex3 lock create" \
    "pex3 lock create --style=universal --target-system linux --target-system mac \
     --interpreter-constraint 'CPython==3.11.*' -r requirements.txt -o out.json" \
  -n "this PR: uv+pex" \
    "uv pip compile requirements.txt --python-version 3.11 --no-header --universal -o pinned.txt \
     && pex3 lock create --style=universal --target-system linux --target-system mac \
     --interpreter-constraint 'CPython==3.11.*' --no-transitive -r pinned.txt -o out.json" \
  -n "bypass pex: uv pip compile" \
    "uv pip compile requirements.txt --python-version 3.11 \
     --generate-hashes --no-header --universal -o out.txt"

The real speedup comes from skipping pex3 lock create entirely. Pants' consumption path already handles requirements.txt lockfiles with no changes needed.

Yes. Thanks for the perf breakdown. Unfortunately, that was a bit of a waste of your time. This has been known for a good while now, but Pants folks have been super poor about getting a hold of the tangle of issues, PRs and slack threads. I incessantly hound them, which they don't respond well (or at all) to. I'm just pointing out yet again that your time is being wasted by Pants maintainers not gathering threads, making a plan and announcing it.

@seungwoo-ji-03, for example, are you aware your merged PR may have been wasted effort itself?: https://pantsbuild.slack.com/archives/C0D7TNJHL/p1777401294195339?thread_ts=1777401294.195339&cid=C0D7TNJHL

@jsirois Thanks for pointing this out — I wasn't aware of benjyw's pending work.
It'd be helpful if related PRs were flagged or closed with a note when a replacement is in progress, so contributors can avoid duplicating effort.

Yes, sorry, @seungwoo-ji-03, I should have been much more public about this work (which I started after #23197 was merged, after discussion at the recent developer monthly call). Just got a lot going on, and trying to get this work done in the time cracks between everything else.

That said, I have a mostly-working implementation of uv lockfile support (inspired by this PR and by your #23197) that doesn't require the intermediate conversions and pex invocations, and creates pexes as needed from a uv-synced venv. I'm ironing out an issue with console scripts, and then should have it ready for review, hopefully today.

Note that this is superseded by the more comprehensive solution in #23302. Thanks for all the effort on this front, and for the prodding to finally get this done properly.

Closing since #23302 is now merged. We'd love your feedback on that one (I do have your first round of fixes, and will look at how to apply them). Thanks!