rpaths with spack setup #12519
Description
I'm attempting to use spack setup to establish an out-of-stage build environment for cmake-based C++ projects, as described here. However, the binaries produced under this workflow have link errors for dependencies related to the compiler's runtime library.
Steps to reproduce the issue
$ spack setup $SPEC && mkdir build && cd build && ../spconfig.py .. && make && ./the-binaryError description
$ ldd the-binary
...
libc++.so.1 => not found
libc++abi.so.1 => not foundThese dependencies are for clang's libc++ runtime, which are configured by the extra_rpaths variable in compilers.yaml.
Information on your system
compilers.yaml:
compilers:
- compiler:
environment: {}
extra_rpaths:
- /spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/llvm-8.0.0-vcde4opt4xfhe5qcrktcagi6r7uhskgf/lib
flags: {}
modules:
- llvm-8.0.0-gcc-5.4.0-vcde4op
operating_system: ubuntu16.04
paths:
cc: /spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/llvm-8.0.0-vcde4opt4xfhe5qcrktcagi6r7uhskgf/bin/clang
cxx: /spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/llvm-8.0.0-vcde4opt4xfhe5qcrktcagi6r7uhskgf/bin/clang++
f77: /usr/bin/gfortran
fc: /usr/bin/gfortran
spec: [email protected]
target: x86_64Additional Information
My CMakeLists.txt is correctly configured to use rpaths, and does so for all package-level dependencies (i.e., dependencies declared in the package's package.py file). However, the rpaths for the missing dependencies are defined by the extra_rpaths variable in the compilers.yaml file, and are completely absent from the generated spconfig.py file (attached). The bug as I understand it is that rpaths declared for the compiler with extra_rpaths are not encoded in the CMAKE_INSTALL_RPATH variable in the generated spconfig.py file.
Workarounds
I know of two workarounds.
Loading the compiler's module
The issue can be avoided by loading llvm (spack load llvm) prior to executing the binary (./the-binary). However, if the binary is installed with make install, then the issue will resurface.
Explicitly depending upon LLVM
@chuckatkins suggested in a comment regarding issue #11582 that it might be fitting to make the compiler a dependency of all packages that it builds (i.e., with depends_on() in the package.py file). I've tried this fix and can confirm that it works, but with the caveat that I have to hard-code a dependency on LLVM without knowledge of whether that's actually the compiler that the user is using. Is it possible to gain access to the spec (or the compiler) in the class definition of a package (i.e., outside of a method)? Additionally, this typically ends up reinstalling the compiler to satisfy the need for the compiler dependency to be compiled by itself.
Attachments
spconfig.py:
#!/usr/bin/python
#
import sys
import os
import subprocess
def cmdlist(str):
return list(x.strip().replace("'",'') for x in str.split('\n') if x)
env = dict(os.environ)
env['CC'] = '/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/llvm-8.0.0-vcde4opt4xfhe5qcrktcagi6r7uhskgf/bin/clang'
env['CMAKE_PREFIX_PATH'] = ":".join(cmdlist("""
/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/cmake-3.12.3-mt7wnhrl2gwab3qb3bll4nvo2pmj7idw
"""))
env['CXX'] = '/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/llvm-8.0.0-vcde4opt4xfhe5qcrktcagi6r7uhskgf/bin/clang++'
env['FC'] = '/usr/bin/gfortran'
env['PATH'] = ":".join(cmdlist("""
/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/cmake-3.12.3-mt7wnhrl2gwab3qb3bll4nvo2pmj7idw/bin
/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/cmake-3.12.3-mt7wnhrl2gwab3qb3bll4nvo2pmj7idw/bin
/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/llvm-8.0.0-vcde4opt4xfhe5qcrktcagi6r7uhskgf/bin
/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/environment-modules-3.2.10-mb3jzyhb6r5wkb64ojkr5porf2zxjie6/Modules/bin
/spack/bin
/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin
/snap/bin
"""))
env['SPACK_TRANSITIVE_INCLUDE_PATH'] = ";".join(cmdlist("""
/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/cmake-3.12.3-mt7wnhrl2gwab3qb3bll4nvo2pmj7idw/include
"""))
cmd = cmdlist("""
/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/cmake-3.12.3-mt7wnhrl2gwab3qb3bll4nvo2pmj7idw/bin/cmake
-G
Unix Makefiles
-DCMAKE_INSTALL_PREFIX:PATH=/spack/opt/spack/linux-ubuntu16.04-x86_64/clang-8.0.0/hdiagsutil-master-c5egaqfc2lvd4jl5ragshnp6edowy3hd
-DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo
-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON
-DCMAKE_INSTALL_RPATH_USE_LINK_PATH:BOOL=FALSE
-DCMAKE_INSTALL_RPATH:STRING=/spack/opt/spack/linux-ubuntu16.04-x86_64/clang-8.0.0/hdiagsutil-master-c5egaqfc2lvd4jl5ragshnp6edowy3hd/lib;/spack/opt/spack/linux-ubuntu16.04-x86_64/clang-8.0.0/hdiagsutil-master-c5egaqfc2lvd4jl5ragshnp6edowy3hd/lib64
-DCMAKE_PREFIX_PATH:STRING=/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/cmake-3.12.3-mt7wnhrl2gwab3qb3bll4nvo2pmj7idw
-DCMAKE_C_FLAGS=
-DCMAKE_CXX_FLAGS=
""") + sys.argv[1:]
proc = subprocess.Popen(cmd, env=env)
proc.wait()