cray platform: detect platform properly after module purge#12989
cray platform: detect platform properly after module purge#12989
Conversation
alalazo
left a comment
alalazo
left a comment
There was a problem hiding this comment.
Just a few comments, based on testing this PR on Cori after a module purge.
|
The CCE changes look like they're based on the assumption that v9.0 and greater are using the new clang interface. This isn't necessarily the case as the the |
|
I suppose this goes back to a previous discussion we had regarding the cray detection. My question would be whether or not spack even should be detecting the cray platform if no modules are loaded? If no modules are loaded then should the machine not be treated as vanilla linux for the front end? |
becker33
force-pushed
the
bugfix/cray-no-module-preloaded
branch
from
d77406e to
59ed98b
Compare
Moved a couple of functions that were private to the cray.py module within the only other function that was using them
This commit improves the robustness of target detection on Cray by trying multiple strategies. The first one that produces results wins. If nothing is found only the generic family of the host is used as a target.
|
@tgamblin I think I've addressed all of your review comments. |
tgamblin
left a comment
tgamblin
left a comment
There was a problem hiding this comment.
Still have some requests here -- the os.environ thing needs another tweak.
lib/spack/spack/compiler.py
Outdated
| This method takes advantage of compiler modules to be cross-platform. | ||
| """ | ||
| # store environment to replace later | ||
| backup_env = os.environ |
There was a problem hiding this comment.
This still doesn't work because backup_env is a reference, not a copy. You want:
backup_env = os.environ.copy()
lib/spack/spack/compiler.py
Outdated
| modifications) to enable the compiler to run properly on any platform. | ||
| """ | ||
| # store environment to replace later | ||
| backup_env = os.environ.copy |
There was a problem hiding this comment.
Still wrong 😢
Can you please add a test for this?
|
This PR broke Spack support on Blue Waters: $ spack help
Traceback (most recent call last):
File "/projects/eot/bbcj/stewart1/spack/bin/spack", line 64, in <module>
sys.exit(spack.main.main())
File "/mnt/b/projects/eot/bbcj/stewart1/spack/lib/spack/spack/main.py", line 767, in main
if spack.config.get('config:debug'):
File "/mnt/b/projects/eot/bbcj/stewart1/spack/lib/spack/spack/config.py", line 671, in get
return config.get(path, default, scope)
File "/mnt/b/projects/eot/bbcj/stewart1/spack/lib/spack/llnl/util/lang.py", line 552, in __getattr__
return getattr(self.instance, name)
File "/mnt/b/projects/eot/bbcj/stewart1/spack/lib/spack/llnl/util/lang.py", line 548, in instance
self._instance = self.factory()
File "/mnt/b/projects/eot/bbcj/stewart1/spack/lib/spack/spack/config.py", line 653, in _config
_add_platform_scope(cfg, ConfigScope, name, path)
File "/mnt/b/projects/eot/bbcj/stewart1/spack/lib/spack/spack/config.py", line 606, in _add_platform_scope
platform = spack.architecture.platform().name
File "/mnt/b/projects/eot/bbcj/stewart1/spack/lib/spack/llnl/util/lang.py", line 178, in _memoized_function
func.cache[args] = func(*args)
File "/mnt/b/projects/eot/bbcj/stewart1/spack/lib/spack/spack/architecture.py", line 517, in platform
return platform_cls()
File "/mnt/b/projects/eot/bbcj/stewart1/spack/lib/spack/spack/platforms/cray.py", line 49, in __init__
for target in self._avail_targets():
File "/mnt/b/projects/eot/bbcj/stewart1/spack/lib/spack/spack/platforms/cray.py", line 188, in _avail_targets
craype_targets = target_names_from_modules(craype_modules)
File "/mnt/b/projects/eot/bbcj/stewart1/spack/lib/spack/spack/platforms/cray.py", line 164, in target_names_from_modules
for mod in modules:
TypeError: 'NoneType' object is not iterable |
|
Hopefully this is useful debugging information: $ module avail -t craype-
/u/sciteam/stewart1/spack/share/spack/modules/cray-cnl5-interlagos:
/sw/EasyBuild/modules/all:
/sw/bw/modulefiles:
/sw/xe/modulefiles:
/usr/local/modulefiles:
/opt/cray/craype/2.5.16/modulefiles:
craype-abudhabi
craype-abudhabi-cu
craype-accel-host
craype-accel-nvidia20
craype-accel-nvidia35
craype-accel-nvidia52
craype-accel-nvidia60
craype-barcelona
craype-hugepages128K
craype-hugepages16M
craype-hugepages2M
craype-hugepages512K
craype-hugepages64M
craype-hugepages8M
craype-interlagos
craype-interlagos-cu
craype-istanbul
craype-mc12
craype-mc8
craype-network-gemini
craype-network-none
craype-shanghai
/opt/cray/modulefiles:
craype-installer/1.16.2
craype-installer/1.17.0
craype-installer/1.18.0
craype-installer/1.20.0
craype-installer/1.24.2
craype-installer/1.24.3(default)
/opt/modulefiles:
/opt/cray/gem/modulefiles:
$ ls /opt/cray/pe/craype/default/modulefiles
ls: cannot access /opt/cray/pe/craype/default/modulefiles: No such file or directoryThe problem is the second check for |
|
P.S. Quick workaround for anyone else who encounters this: diff --git a/lib/spack/spack/platforms/cray.py b/lib/spack/spack/platforms/cray.py
index 2fccf2fe5..b036a507f 100644
--- a/lib/spack/spack/platforms/cray.py
+++ b/lib/spack/spack/platforms/cray.py
@@ -181,7 +181,7 @@ def modules_from_listdir():
if getattr(self, '_craype_targets', None) is None:
strategies = [
lambda: modules_in_output(module('avail', '-t', 'craype-')),
- modules_from_listdir
+ #modules_from_listdir
]
for available_craype_modules in strategies:
craype_modules = available_craype_modules() |
|
Also, note that before this PR, I got: $ spack arch
cray-cnl5-interlagosbut now I only get: $ spack arch
cray-cnl5-x86_64so I think this also broke microarchitecture detection. That'll probably require someone else to debug. |
|
This PR also broke module loading in - compiler:
paths:
cc: cc
cxx: CC
f77: ftn
fc: ftn
operating_system: cnl5
target: any
modules:
- PrgEnv-gnu
- gcc/5.3.0
environment: {}
extra_rpaths: []
flags: {}
spec: [email protected]This used to work fine. After updating to develop, I see an error message during installation: When I inspect the I've never seen this before, but I'm guessing that it's because Spack is unsetting the |
| bash = Executable('/bin/bash') | ||
| output = bash( | ||
| '-lc', 'echo $CRAY_CPU_TARGET', | ||
| '--norc', '--noprofile', '-lc', 'echo $CRAY_CPU_TARGET', |
There was a problem hiding this comment.
@becker33 I am a bit late with this but what output do you expect from this command? The environment is clean and bash is called in the login mode with --noprofile. Doesn't this mean that the output is always empty?
7 participants
Currently, we assume the cray platform can be identified by the default modules
We have discovered a cray machine on which the default modules are
No modules loaded.This caused some problems for Spack's platform detection and OS setup. This PR allows Spack to persevere despite those problems. In the worst case in which the
cle_releasefiles have been removed from the system, we will continue withos=cnlunknowninstead of failing immediately.NOTE 2019-10-01: The scope of this PR has expanded to include a set of hotfixes found/used at an internal hackathon at LLNL.