Do not load view into environment in generated spack ci jobs#17440
Do not load view into environment in generated spack ci jobs#17440becker33 merged 1 commit intospack:developfrom
Conversation
|
|
||
| job_scripts = [ | ||
| 'spack env activate .', | ||
| 'spack env activate --without-view .', |
There was a problem hiding this comment.
If one of the dependencies of a package is python, spack seems to be confusing the system python with the python version in the view during spack commands themselves.
I encountered it when trying to build [email protected] in one of the lasts steps (well, that's where python gets installed...). The main issue is that it cannot find botocore anymore which is a system dependency, and it fails to upload the binaries to the build cache. Probably the search paths for python modules are simplify confused by loading the other python into a view.
There was a problem hiding this comment.
So you needed this independent of compiler bootstrapping? I'm just surprised because I have run lots of pipelines with packages that have a dependency on python.
There was a problem hiding this comment.
Hm, I'm running it in a non-standard container runtime, so it could be that that somehow influences the environment a bit. Let me try and see tomorrow if it's reproducible when dropping this commit ^ but keeping all the other current patches that seem necessary.
There was a problem hiding this comment.
@haampie are you running this on Spack v0.14.2? We had a bug up until v0.15.0 that caused Spack to fail if an environment including python was partially installed and the view was activated. If you're seeing this behavior in v0.15.0 that is a much bigger concern to me.
@scottwittenburg On the other hand, there's not really any reason to add the view to the user environment as part of the CI job, so eliminating it should have a (probably negligible) performance benefit.
|
@haampie updated the title for accuracy. The view is created on the basis of the spack.yaml file (or the I don't think this clarification affects the purpose, effect, or beneficial nature of the PR. |
|
It's still reproducible by the way, with all other patches included. See https://gitlab.com/cscs-ci/ci-testing/spack-ci/-/pipelines/167717054 only happens when |
Solves a problem (#17432) where spack seems to be using the wrong python version when python was a build dependency. It seems enabling the environment without loading a view is sufficient to work around that.