Unify search logic for compilers and externals#45784
Merged
haampie merged 12 commits intospack:developfrom Aug 22, 2024
Merged
Conversation
spackbot-app
bot
added
commands
compilers
core
alalazo
force-pushed
the
deprecation/compilers-yaml-unified-search
branch
from
c1ab178 to
1896ab6
Compare
Member
|
This makes it 2.5x slower for me. |
Member
Author
|
The main issue for the slowdown you still see seems to be the overhead of loading a package (which we don't have on The ratio for |
alalazo
force-pushed
the
deprecation/compilers-yaml-unified-search
branch
from
888dab3 to
baae73c
Compare
alalazo
force-pushed
the
deprecation/compilers-yaml-unified-search
branch
from
bd98848 to
553ab6d
Compare
Member
Author
This commit makes spack compiler find write in packages.yaml, instead of compilers.yaml.
This excludes, e.g. llvm~clang etc. if found
add_compilers_to_config is not called anymore
alalazo
force-pushed
the
deprecation/compilers-yaml-unified-search
branch
from
553ab6d to
79bc08f
Compare
haampie
reviewed
Aug 21, 2024
haampie
reviewed
Aug 21, 2024
Member
|
ThreadpoolExecutor worked and works fine and is significantly faster: 3s -> 0.5s on macOS. I guess it's relatively safe although we don't control the package specific implementation... but that's unlikely to have global side effects. |
Member
Author
Seems there's some work to do to use a ThreadPoolExecutor: $ spack external find -t compiler
==> Warning: error detecting "llvm" from prefix /bin [trying to set variants "clang, lld, or lldb" in package "llvm", but the package has no such variants [happened when validating '[email protected]+clang~lld~lldb']]
==> Warning: error detecting "llvm" from prefix /usr/bin [trying to set variants "clang, lld, or lldb" in package "llvm", but the package has no such variants [happened when validating '[email protected]+clang+lld~lldb']]
==> No new external packages detectedvs: $ spack external find -t compiler
==> The following specs have been detected on this system and added to /home/culpo/.spack/packages.yaml
[email protected] [email protected] [email protected] [email protected] [email protected] [email protected] |
We could restrict to test this only on Darwin, but it's already tested in audits.
haampie
approved these changes
Aug 22, 2024
tldahlgren
added a commit
to nhanford/spack
that referenced
this pull request
Aug 22, 2024
This was referenced Aug 23, 2024
FrederickDeny
pushed a commit
to FrederickDeny/spack
that referenced
this pull request
Aug 26, 2024
so that there is no duplicate detection logic for compilers
paugier
pushed a commit
to paugier/spack
that referenced
this pull request
Aug 27, 2024
so that there is no duplicate detection logic for compilers
tldahlgren
added a commit
that referenced
this pull request
Aug 30, 2024
tldahlgren
added a commit
to nhanford/spack
that referenced
this pull request
Aug 30, 2024
becker33
pushed a commit
to nhanford/spack
that referenced
this pull request
Sep 5, 2024
Closed
3 tasks
becker33
pushed a commit
to nhanford/spack
that referenced
this pull request
Sep 28, 2024
becker33
pushed a commit
to nhanford/spack
that referenced
this pull request
Oct 10, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #39627
This PR is a slight step back from #44419, to refactor code in a way that doesn't affect where we write compilers.
The code used for:
$ spack compiler findis the same as the one used for:
$ spack external findDead code in
spack.compilershas been removed, and more can be removed later from each compiler class.Compared to #44419, compilers are still written in
compilers.yaml- so users shouldn't be negatively affected by this change.