Skip to content

python: allow it as a build-tool again#49201

Merged
tgamblin merged 3 commits intospack:developfrom
alalazo:bugfix/python-as-build-tool
Feb 26, 2025
Merged

python: allow it as a build-tool again#49201
tgamblin merged 3 commits intospack:developfrom
alalazo:bugfix/python-as-build-tool

Conversation

@alalazo
Copy link
Copy Markdown
Member

@alalazo alalazo commented Feb 26, 2025

fixes #44289

Python was removed from being a build tool in #46980, due to issues when reusing specs. This PR adds a new rule to match the interpreter among different Python packages, in clingo.

It also adds a bunch of new "build-tools", so that specs like:

py-matplotlib backend=tkagg

can be concretized in one go.

Modifications:

A performance benchmark is in #49201 (comment)

fixes spack#44289

Python was removed from being a build tool,
due to issues when reusing specs. This PR
adds a new rule to match the interpreter among
different Python packages, in clingo.

It also adds a bunch of new "build-tools", so that
specs like:

py-matplotlib backend=tkagg

can be concretized in one go.

Signed-off-by: Massimiliano Culpo <[email protected]>
@spackbot-app

This comment was marked as off-topic.

@alalazo alalazo added the bugfix Something wasn't working, here's a fix label Feb 26, 2025
@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Feb 26, 2025

@Chrismarsh I didn't try to install yet:

Screenshot from 2025-02-26 12-21-09

One hour later:

Screenshot from 2025-02-26 14-12-02

Signed-off-by: Massimiliano Culpo <[email protected]>
Signed-off-by: Massimiliano Culpo <[email protected]>
@spackbot-app spackbot-app bot added new-version stand-alone-tests Stand-alone (or smoke) tests for installed packages tests General test capability(ies) labels Feb 26, 2025
@alalazo alalazo marked this pull request as ready for review February 26, 2025 15:45
@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Feb 26, 2025

Still need to measure the impact on performance, for adding a few more dupes by default

@Chrismarsh
Copy link
Copy Markdown
Contributor

This allows me to concretize from fresh. Trying the install now.

@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Feb 26, 2025

The usual benchmark, run on:

  • Spack: 1.0.0.dev0 (dbd5311)
  • Python: 3.13.0
  • Platform: linux-ubuntu20.04-icelake

seems fine:

radiuss.txt
radiuss.develop.csv
radiuss.pr.csv

radiuss

@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Feb 26, 2025

@spackbot run pipeline

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Feb 26, 2025

I've started that pipeline for you!

@tgamblin tgamblin merged commit 3caa313 into spack:develop Feb 26, 2025
37 checks passed
@alalazo alalazo deleted the bugfix/python-as-build-tool branch February 27, 2025 06:28
white238 pushed a commit that referenced this pull request Mar 3, 2025
Python was removed from being a build tool in #46980, due to issues
when reusing specs. This PR adds a new rule to match the interpreter
among different Python packages, in clingo.

It also adds a bunch of new "build-tools", so that specs like:
```
py-matplotlib backend=tkagg
```
can be concretized in one go.

Modifications:
- [x] Make `py-matplotlib backend=tkagg` concretizable
- [x] Add unit-tests to ensure situations like in #46980 do not happen

---------

Signed-off-by: Massimiliano Culpo <[email protected]>
@Meax5qiu
Copy link
Copy Markdown

Still need to measure the impact on performance, for adding a few more dupes by default

Here is another data point: Concretization of cairo took 50 minutes longer after this commit. See #49341

@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Mar 11, 2025

@Meax5qiu Still investigating, but currently I think the big issue with cairo is 7e65c57 which:

  • adds a lot of conflicting defaults
  • makes all the variant conditionals

This makes the concretizer jump across very different solutions, since all of them have penalties applied, and backtrack a lot. This PR makes things much "worse" for cairo because it extends the search space, that due to the apparently "wrong" modeling needs to be explored much more than for other specs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Something wasn't working, here's a fix core PR affects Spack core functionality defaults dependencies new-version python stand-alone-tests Stand-alone (or smoke) tests for installed packages tests General test capability(ies) update-package

Projects

None yet

Development

Successfully merging this pull request may close these issues.

python+tkinter cannot be concretized with --fresh python+tkinter in a unified environment: "Cyclic dependency detected"

4 participants