Features/frontend default build#2292
Conversation
|
Quickly trying example discussed in #1980 : spack spec parmetis ^mpich
Input spec
--------------------------------
parmetis
^mpich
Normalized
--------------------------------
parmetis
^[email protected]:
^metis@5:
^mpich
Concretized
--------------------------------
[email protected]%[email protected]~debug~gdb+shared arch=bgq-cnk-powerpc
^[email protected]%[email protected]~doc+ncurses+openssl~ownlibs~qt arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected]~python arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^lz4@131%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected]+sigsegv arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected]~debug~gdb~idx64~real64+shared arch=bgq-cnk-powerpc
^[email protected]%[email protected]+hydra+pmi+romio~verbs arch=bgq-cnk-powerpcSo this looks good! thank you @scheibelp! |
|
Thanks for checking that! It looks like in this case this chose to compile everything with [email protected] I may be able to tweak this so things that arent build deps do their own separate resolution. In the meantime, specifying xl for the top-level dep should get the result you are looking for. |
|
I just noticed that I'm not detecting whether a package is a build dep correctly 100% of the time. Once I fix that I expect this to work |
|
Looking into this, I appear to have confused myself and stumbled upon a bug (namely I was explicitly mentioning cmake in a spec and it showed up as a link deptype). I'll report the issue shortly. In the meantime I expect this to work as intended. I think it slows down concretization by 20-30%. EDIT: I when I ran the tests for this branch compared to develop it didnt take longer - I guess that was in my head |
|
Looks like there is a bug now : ./bin/spack spec parmetis
Input spec
--------------------------------
parmetis
Normalized
--------------------------------
parmetis
^[email protected]:
^metis@5:
^mpi
Concretized
--------------------------------
==> Error: Multiple compilers satisfy spec [email protected]This is when I add packages:
all:
compiler: [xl,gcc]It's true that there are $ cat ~/.spack/bgq/compilers.yaml
compilers:
- compiler:
flags: {}
modules: []
operating_system: rhel6
paths:
cc: /usr/lib64/ccache/gcc
cxx: /usr/lib64/ccache/g++
f77: /usr/bin/gfortran
fc: /usr/bin/gfortran
spec: [email protected]
- compiler:
flags: {}
modules: []
operating_system: cnk
paths:
cc: /usr/lib64/ccache/gcc
cxx: /usr/lib64/ccache/g++
f77: /usr/bin/gfortran
fc: /usr/bin/gfortran
spec: [email protected]
- compiler:
flags: {}
modules: []
operating_system: rhel6
paths:
cc: /opt/ibmcmp/vacpp/bg/12.1/bin/xlc_r
cxx: /opt/ibmcmp/vacpp/bg/12.1/bin/xlc++
f77: /opt/ibmcmp/xlf/bg/14.1/bin/xlf_r
fc: /opt/ibmcmp/xlf/bg/14.1/bin/xlf2008
spec: [email protected]
- compiler:
flags: {}
modules: []
operating_system: cnk
paths:
cc: /opt/ibmcmp/vacpp/bg/12.1/bin/xlc_r
cxx: /opt/ibmcmp/vacpp/bg/12.1/bin/xlc++
f77: /opt/ibmcmp/xlf/bg/14.1/bin/xlf_r
fc: /opt/ibmcmp/xlf/bg/14.1/bin/xlf2008
spec: [email protected]Note that this doesn't happen with |
|
Thanks for checking this I will try to look at this tonight. |
|
@pramodk I'm having some trouble replicating this and also theorizing what is causing the problem. Would you be able to include what is printed by rerunning that command (I added some print statements to output more info)? One possibility: do you have a compilers.yaml in the spack installation directory (in addition to ~/.spack)? The printout should reveal this plus some extra information. Thanks! |
|
@scheibelp : I should mention few additional details :
Now trying ○ → rm -rf ~/.spack
○ → spack compiler find
spack arch==> Added 2 new compilers to /somepath/home/kumbhar/.spack/bgq/compilers.yaml
[email protected] [email protected]
○ → spack arch
bgq-cnk-ppc64
○ → spack spec parmetis
Input spec
--------------------------------
parmetis
Normalized
--------------------------------
parmetis
^[email protected]:
^metis@5:
^mpi
Concretized
--------------------------------
Traceback (most recent call last):
File "/somepath//spack/bin/spack", line 195, in <module>
main()
File "/somepath//spack/bin/spack", line 172, in main
return_val = command(parser, args)
File "/somepath//spack/lib/spack/spack/cmd/spec.py", line 78, in spec
spec.concretize()
File "/somepath//spack/lib/spack/spack/spec.py", line 1354, in concretize
self._concretize_helper())
File "/somepath//spack/lib/spack/spack/spec.py", line 1175, in _concretize_helper
changed |= dep.spec._concretize_helper(presets, visited)
File "/somepath//spack/lib/spack/spack/spec.py", line 1185, in _concretize_helper
(spack.concretizer.concretize_architecture(self),
File "/somepath//spack/lib/spack/spack/concretize.py", line 309, in concretize_architecture
self._concretize_operating_system(spec),
File "/somepath//spack/lib/spack/spack/concretize.py", line 249, in _concretize_operating_system
if spec.build_dep():
File "/somepath//spack/lib/spack/spack/spec.py", line 608, in __getattr__
raise AttributeError()
AttributeErrorI don't have any other changes in |
|
Oops....wait a minute, there is something I have in my config: $ rm -rf ~/.spack
$ ./bin/spack config get compilers
compilers:
- compiler:
modules: []
operating_system: CNK
paths:
cc: /usr/bin/bgxlc_r
cxx: /usr/bin/bgxlc++
f77: /usr/bin/bgxlf_r
fc: /usr/bin/bgxlf2008
spec: [email protected]So certainly Edit : Ok so there was an old entry in |
Based on the above it seems like the issue could in fact have been caused by duplicate compiler definition. I'll revert the debugging commit for now and I can bring it back if the problem returns. Thanks! |
|
@scheibelp : I was too quick to merge different PR's and I made mistake! Sorry! # with compiler: [xl, gcc]
$ ./bin/spack spec parmetis ^mpich
Input spec
--------------------------------
parmetis
^mpich
Normalized
--------------------------------
parmetis
^[email protected]:
^metis@5:
^mpich
Concretized
--------------------------------
[email protected]%[email protected]~debug~gdb+shared arch=bgq-cnk-ppc64
^[email protected]%[email protected]~doc+ncurses+openssl~ownlibs~qt arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected]~python arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^lz4@131%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected]+sigsegv arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected] arch=bgq-rhel6-power7
^[email protected]%[email protected]~debug~gdb~idx64~real64+shared arch=bgq-cnk-ppc64
^[email protected]%[email protected]+hydra+pmi+romio~verbs arch=bgq-cnk-ppc64 |
|
Great! Thanks again for checking this! I'll work on doing clean up and add tests for this. EDIT: hopefully by 7pm PST |
|
@xjrc Thanks for that! A cursory glance tells me it shouldn't be too bad: the more complex logic was in concretize_compiler and it doesnt appear that it will be affected. I'll just have to translate my logic to pick out the frontend os/target for build deps in In any case I should be able to get this up to date within a few hours of #2261 merging. |
scheibelp
force-pushed
the
features/frontend-default-build
branch
from
a9c620c to
2c9122f
Compare
|
I think this is in a good spot now so I'll remove the WIP tag. I can update and resolve conflicts when #2261 is merged |
scheibelp
force-pushed
the
features/frontend-default-build
branch
from
2c9122f to
1dd4ea5
Compare
|
@scheibelp: #2261 is merged now. Can you rebase this one? @becker33 See this PR for the |
|
@becker33 IIRC you mentioned taking a crack at merging this yourself. Did you start on that? If not I can do that by tomorrow. |
|
@scheibelp I think it may be easier for you to do it, since you know your code better than I will. I've started to familiarize myself with, but I'm not there yet.. |
|
Makes sense! I'll do the rebase |
scheibelp
force-pushed
the
features/frontend-default-build
branch
2 times, most recently
from
386be4c to
1763b5a
Compare
|
Rebase complete I'll note from the commit message:
I can alter that if desired |
scheibelp
force-pushed
the
features/frontend-default-build
branch
3 times, most recently
from
fc311d1 to
43f18d5
Compare
|
I believe we want run dependencies to default similarly to link dependencies. |
|
OK I'll update to that behavior (it will be ready this afternoon) |
Packages built targeting a backend may depend on packages like cmake which can be built against the frontend. With this commit, any build dependency or child of a build dependency will target the frontend by default. In compiler concretization when packages copy compilers from nearby packages, build dependencies use compiler information from other build dependencies, and link dependencies avoid using compiler information from build dependencies.
scheibelp
force-pushed
the
features/frontend-default-build
branch
from
43f18d5 to
c154909
Compare
|
Behavior is now updated to only prefer frontend for build deps (or dependencies of build deps that are not link/run dependencies for the root). Commit message also updated. |
|
@scheibelp: Thanks! |
|
|
||
| def __str__(self): | ||
| return self.name | ||
| return self.name + self.version |
There was a problem hiding this comment.
@scheibelp: I just realized post-merging that this might affect the Cray folks. @robertdfrench @mpbelhorn @mamelara @KineticTheory: is this going to confuse your spack instances? I believe we should have a number on the CNL OS, but previously we weren't appending a version to it.
|
@tgamblin Yes, this would probably confuse our Spack instances if we updated Spack in-place. But that's OK, because we don't update Spack in-place. For anyone who has concerns about this sort of potential problem, the following is how we avoid it. When we sync with upstream, we rebuild the entire software stack that we provide in a new location. Previously, this was an entirely different Spack instance in a new location, however the newly configurable Leaving the older installed packages in-place on disk, but ignored by Spack allows us to avoid discontinuities regarding provided software and no worry about name clashes/changes. Our Spack deploys are hidden from users. We point them to installed packages through customized modulefiles in a public place and only update them to point to the new stack once everything is built anew. |
This change can be reverted with no effect. I made it because while looking into something I noticed the compilers for CNL weren't found in the tests (the tests do not actually set the CNL as the backend anymore). I reverted this change at: #2526 |
|
@mpbelhorn @mamelara: Thanks -- that's helpful. Question: is it worth versioning the CNL OS? Do you have CNL upgrades on Titan/other machines that would change the binary compatibility of the OS, and can we get the version in a deterministic way? Currently the CNL OS is hard-coded at version |
Packages built targeting a backend may depend on packages like cmake which can be built against the frontend. With this commit, any build dependency or child of a build dependency will target the frontend by default. In compiler concretization when packages copy compilers from nearby packages, build dependencies use compiler information from other build dependencies, and link dependencies avoid using compiler information from build dependencies.
|
I'm aware that the version can change, but it doesn't change often in the lifetime of our machines. I'm not the one to ask - I don't even know what version of CNL is on our machines or exactly where to find it. Titan for example runs a modified SuSE11.3, "Cray Linux Environment" CLE 5.2 on the login/batch nodes. CLE is clearly versioned and releases ship with an associated (non-SuSE-based?) "Compute Node Linux" CNL for the compute nodes but I don't know what the 'official' CNL version is. I prefer to consider CLE simply SuSE11. |
The primary goal of spack#2292 was to use the frontend compiler to make build dependencies like cmake on HPC platforms. It turns out that while this works in some cases, it did not handle cases where a package was a link dependency of the root and of a build dependency (and could produce incorrect concretizations which would not build).
The primary goal of #2292 was to use the frontend compiler to make build dependencies like cmake on HPC platforms. It turns out that while this works in some cases, it did not handle cases where a package was a link dependency of the root and of a build dependency (and could produce incorrect concretizations which would not build).
The primary goal of spack#2292 was to use the frontend compiler to make build dependencies like cmake on HPC platforms. It turns out that while this works in some cases, it did not handle cases where a package was a link dependency of the root and of a build dependency (and could produce incorrect concretizations which would not build).
6 participants
A few notes:
-whether a dependency is a build dep is determined by checking whether one can reach the root by traversing link deps
-link dependencies of build deps should be built with the frontend compiler
-if a parent links to a dependency of a build dep however, that may not always be the case (I think this is mostly handled)
Also this appears to significantly boost concretization time as-is.