Skip to content

solver: requirements on virtual can influence provider weights#51282

Merged
haampie merged 5 commits intospack:developfrom
alalazo:solver/account-for-strong-preferences
Sep 10, 2025
Merged

solver: requirements on virtual can influence provider weights#51282
haampie merged 5 commits intospack:developfrom
alalazo:solver/account-for-strong-preferences

Conversation

@alalazo
Copy link
Copy Markdown
Member

@alalazo alalazo commented Sep 8, 2025

fixes #51267

Before this change, provider weights were determined just by package preferences. This could cause issues in cases where the preferred, or required, provider is not the "default" provider.

This PR changes the ASP setup to take into account the most common cases of requirements and strong preferences, so that the provider weights are set accordingly.

Before this change, provider weights were determined just by
package preferences.

This could cause issues in cases where the preferred, or
required, provider is not the default package preference.

Signed-off-by: Massimiliano Culpo <[email protected]>
@alalazo alalazo added the bugfix Something wasn't working, here's a fix label Sep 8, 2025
@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Sep 8, 2025

@climbfuji

Signed-off-by: Massimiliano Culpo <[email protected]>
@haampie
Copy link
Copy Markdown
Member

haampie commented Sep 9, 2025

Needs a bit more work to support conflicts, cause in the current implementation you'd end up preferring the provider you conflict with, meaning the solver always takes a penalty in the requirements weights, allowing it to toggle variants that drop dependencies on languages (which I guess means minimizing the number of dependencies in general).

It should drop the conflicting providers from the list of preferred ones to start counting at 0 for these penalties.

@alalazo alalazo requested a review from haampie September 9, 2025 11:30
@climbfuji
Copy link
Copy Markdown
Contributor

@alalazo Things are looking better with this PR. As a reminder, these are the four versions I tried. Before this PR, v001 and v003 were bad, v002 and v004 were good.

packages:
    # v001
    c:
      prefer:
      - oneapi
    cxx:
      prefer:
      - oneapi
    fortran:
      prefer:
      - oneapi
    # v003, v004
    #c:
    #  prefer:
    #  - intel-oneapi-compilers
    #cxx:
    #  prefer:
    #  - intel-oneapi-compilers
    #fortran:
    #  prefer:
    #  - intel-oneapi-compilers
    all:
      # v002
      #prefer:
      #- '%oneapi'
      providers:
        # v004
        #c: [intel-oneapi-compilers, gcc]
        #cxx: [intel-oneapi-compilers, gcc]
        #fortran: [intel-oneapi-compilers, gcc]
...

Results: v001 is still bad (ok), v002/v003/v004 are good. There are slight differences between the concretization of v002 and v003/v004, but v003 and v004 are now identical

[unified-env-oneapi] [dom@8W089Y3 spack-dev-20250904]$ cat log.concretize.unified-env-oneapi.v001.gcc | wc -l
177
[unified-env-oneapi] [dom@8W089Y3 spack-dev-20250904]$ cat log.concretize.unified-env-oneapi.v002.gcc | wc -l
12
[unified-env-oneapi] [dom@8W089Y3 spack-dev-20250904]$ cat log.concretize.unified-env-oneapi.v003.gcc | wc -l
13
[unified-env-oneapi] [dom@8W089Y3 spack-dev-20250904]$ cat log.concretize.unified-env-oneapi.v004.gcc | wc -l
13
[unified-env-oneapi] [dom@8W089Y3 spack-dev-20250904]$ cat log.concretize.unified-env-oneapi.v002.gcc | wc -l
12
[unified-env-oneapi] [dom@8W089Y3 spack-dev-20250904]$ cat log.concretize.unified-env-oneapi.v002.gcc
 -   35iqzzx      ^[email protected]~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=almalinux9 target=skylake %c,[email protected]
[e]  igrmpzp          ^[email protected]~binutils+bootstrap~graphite~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++' platform=linux os=almalinux9 target=skylake
[+]  agedjlg          ^[email protected] build_system=generic platform=linux os=almalinux9 target=skylake
[+]  b2hf3vf          ^[email protected]+optimize+pic+shared build_system=makefile platform=linux os=almalinux9 target=skylake %c,[email protected]
[+]  3nisfcs      ^[email protected]~guile build_system=generic platform=linux os=almalinux9 target=skylake %[email protected]
 -   yxa2iug              ^[email protected]~dev~docs+src build_system=generic platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   6jlcvnb          ^[email protected]~dbus~debug~doc~examples~framework~gtk~gui~location~opengl~phonon+shared+sql+ssl+tools~webkit build_system=generic patches:=2081e9c,51aeba5,75bcb42,78c70fb,7f34d48,84b0991,9378afd platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   vduiqik          ^[email protected]~color build_system=autotools platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   a6otnph                          ^[email protected] build_system=autotools platform=linux os=almalinux9 target=skylake %[email protected]
 -   j7htybg      ^[email protected]~doc+openmp build_system=autotools platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   fkaqf2a          ^[email protected]+cxx~java~pic~python build_system=autotools patches:=33897ad platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   hgk7ifw                  ^[email protected] build_system=python_pip platform=linux os=almalinux9 target=skylake %[email protected]
[unified-env-oneapi] [dom@8W089Y3 spack-dev-20250904]$ cat log.concretize.unified-env-oneapi.v003.gcc
 -   pqs3glv      ^[email protected]~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=almalinux9 target=skylake %c,[email protected]
[e]  igrmpzp          ^[email protected]~binutils+bootstrap~graphite~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++' platform=linux os=almalinux9 target=skylake
[+]  agedjlg          ^[email protected] build_system=generic platform=linux os=almalinux9 target=skylake
[+]  b2hf3vf          ^[email protected]+optimize+pic+shared build_system=makefile platform=linux os=almalinux9 target=skylake %c,[email protected]
[+]  3nisfcs      ^[email protected]~guile build_system=generic platform=linux os=almalinux9 target=skylake %[email protected]
 -   7cl2daj              ^[email protected]~dev~docs+src build_system=generic platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   qwqo2s7          ^[email protected]~dbus~debug~doc~examples~framework~gtk+gui~location~opengl~phonon+shared+sql+ssl+tools~webkit build_system=generic patches:=2081e9c,51aeba5,75bcb42,78c70fb,7f34d48,84b0991,9378afd platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   gjxxgmt                      ^[email protected] build_system=autotools platform=linux os=almalinux9 target=skylake %[email protected]
 -   7jpjlbn                          ^[email protected] build_system=autotools platform=linux os=almalinux9 target=skylake %[email protected]
[+]  zqduveq          ^[email protected]~color build_system=autotools platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   7qlxb6f      ^[email protected]~doc+openmp build_system=autotools platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   fkaqf2a          ^[email protected]+cxx~java~pic~python build_system=autotools patches:=33897ad platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   qd6l3pf                  ^[email protected] build_system=python_pip platform=linux os=almalinux9 target=skylake %[email protected]
[unified-env-oneapi] [dom@8W089Y3 spack-dev-20250904]$ cat log.concretize.unified-env-oneapi.v004.gcc
 -   pqs3glv      ^[email protected]~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=almalinux9 target=skylake %c,[email protected]
[e]  igrmpzp          ^[email protected]~binutils+bootstrap~graphite~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++' platform=linux os=almalinux9 target=skylake
[+]  agedjlg          ^[email protected] build_system=generic platform=linux os=almalinux9 target=skylake
[+]  b2hf3vf          ^[email protected]+optimize+pic+shared build_system=makefile platform=linux os=almalinux9 target=skylake %c,[email protected]
[+]  3nisfcs      ^[email protected]~guile build_system=generic platform=linux os=almalinux9 target=skylake %[email protected]
 -   7cl2daj              ^[email protected]~dev~docs+src build_system=generic platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   qwqo2s7          ^[email protected]~dbus~debug~doc~examples~framework~gtk+gui~location~opengl~phonon+shared+sql+ssl+tools~webkit build_system=generic patches:=2081e9c,51aeba5,75bcb42,78c70fb,7f34d48,84b0991,9378afd platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   gjxxgmt                      ^[email protected] build_system=autotools platform=linux os=almalinux9 target=skylake %[email protected]
 -   7jpjlbn                          ^[email protected] build_system=autotools platform=linux os=almalinux9 target=skylake %[email protected]
[+]  zqduveq          ^[email protected]~color build_system=autotools platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   7qlxb6f      ^[email protected]~doc+openmp build_system=autotools platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   fkaqf2a          ^[email protected]+cxx~java~pic~python build_system=autotools patches:=33897ad platform=linux os=almalinux9 target=skylake %c,[email protected]
 -   qd6l3pf                  ^[email protected] build_system=python_pip platform=linux os=almalinux9 target=skylake %[email protected]
 - ```

@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Sep 9, 2025

I'll check the other issue in another PR. Currently we hardcoded a translation of oneapi in the context of direct dependencies. We should add proper package aliases to use it in config as a name.

@climbfuji
Copy link
Copy Markdown
Contributor

I'll check the other issue in another PR. Currently we hardcoded a translation of oneapi in the context of direct dependencies. We should add proper package aliases to use it in config as a name.

Personally, I'd be ok with exiting with an error when running into the legacy syntax given that the new syntax works with your changes, but of course that's up to you and the other spack developers.

Thanks for fixing this!

@haampie
Copy link
Copy Markdown
Member

haampie commented Sep 9, 2025

You could compare the minor differences between v002 and v003/v004. The latter should now be superior, because the preference for a dependency on oneapi makes the concretizer toggle variants to minimize the number of "data only" packages (that do not depend on any compiler).

@climbfuji
Copy link
Copy Markdown
Contributor

You could compare the minor differences between v002 and v003/v004. The latter should now be superior, because the preference for a dependency on oneapi makes the concretizer toggle variants to minimize the number of "data only" packages (that do not depend on any compiler).

Yes, I am happy with v003/v004, I don't think v002 is any better. In v002, it chooses to use a different libbsd that doesn't depend on libmd, but that doesn't affect us.

Copy link
Copy Markdown
Member

@haampie haampie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@haampie haampie merged commit 8d2ddc1 into spack:develop Sep 10, 2025
32 checks passed
@alalazo alalazo deleted the solver/account-for-strong-preferences branch September 10, 2025 06:02
@alalazo alalazo added the v1.0.3 PRs to backport for v1.0.3 label Sep 10, 2025
@becker33 becker33 mentioned this pull request Jan 10, 2026
@becker33 becker33 removed the v1.0.3 PRs to backport for v1.0.3 label Jan 30, 2026
@becker33
Copy link
Copy Markdown
Member

This PR does not apply cleanly to 1.0 and will be dropped from the 1.0.3 release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Something wasn't working, here's a fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Concretization differs greatly when using different "prefer" syntax

4 participants