Specify GCC prefix in LLVM-based compilers by haampie · Pull Request #33146 · spack/spack

haampie · 2022-10-10T12:24:05Z

spack.compiler.Compiler: introduce prefix property

We currently don't really have something that gives the GCC install path, which
is used by many LLVM-based compilers (llvm, llvm-amdgpu, nvhpc, ...) to fix the
GCC toolchain once and for all.

This prefix property is dynamic in the sense that it queries the compiler
itself. This is necessary because it's not easy to deduce the install path from
the spack compiler cc property (might be a symlink, might be a filename like
gcc which works by having the compiler load a module that sets the PATH
variable, might be a generic compiler wrapper based on environment variables
like on cray...).

With this new property, we can clean up some recipes that have the logic
repeated for GCC.

Also makes intel-oneapi-compilers behave like llvm and nvhpc, it now
fixes the GCC toolchain based on what %gcc was used, instead of doing a
system search.

We currently don't really have something that gives the GCC install path, which is used by many LLVM-based compilers (llvm, llvm-amdgpu, nvhpc, ...) to fix the GCC toolchain once and for all. This `prefix` property is dynamic in the sense that it queries the compiler itself. This is necessary because it's not easy to deduce the install path from the `cc` property (might be a symlink, might be a filename like `gcc` which works by having the compiler load a module that sets the PATH variable, might be a generic compiler wrapper based on environment variables like on cray...). With this property introduced, we can clean up some recipes that have the logic repeated for GCC.

haampie · 2022-10-10T12:29:25Z

Notice that we're still picking up the wrong lib{gcc_s,stdc++}.so at runtime for C++ sources compiled with intel-oneapi-compilers, due to missing rpaths to the compiler support library dir.

Also note that intel-oneapi-compilers's libsyscl.so depends on libstdc++, and is lacking rpaths too.

In principle we can just use SPACK_COMPILER_IMPLICIT_RPATHS in def install etc and run patchelf --set-rpath ....

trws · 2022-10-10T18:44:35Z

var/spack/repos/builtin/packages/intel-oneapi-compilers/package.py

+    # cannot express that properly. For now, add conflicts for non-gcc compilers
+    # instead.
+    for __compiler in spack.compilers.supported_compilers():
+        if __compiler != "gcc":


I see where you're coming from, this may be the best we can do for now, but is it that gcc is a direct dependency or that we need the system toolchain that's in use? The latter might let us propagate it transitively through other compilers as long as we can query them for the appropriate thing.

More concretely, for clang-based compilers for example we could use the Selected GCC installation from clang -v, or the result of dirname $(clang -print-libgcc-file-name).

It's also a direct dependency of the runtime libraries, although that's not covered by this PR:

$ readelf -d ./compiler/2022.2.0/linux/lib/libsycl.so | grep 'libgcc_s.so' 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]

So, I was thinking to submit another PR that would patchelf --add-rpath <gcc compiler support library dir> <intel oneapi compiler support library>

That makes sense, but all the places this is happening only add the GCC stuff if the compiler is gcc, so compiling with clang we still don't get a toolchain set right?

Yeah, in the PR I've added conflicts to only ever allow GCC as a "compiler" for intel-oneapi-compilers (just s.t. we can express the dependency on GCC support libs).

Regarding llvm%clang, we could introspect clang and have it report its chosen gcc toolchain, and use that to fix GCC_INSTALL_PREFIX once and for all too... but that would another PR.

haampie · 2022-10-11T10:38:51Z

@spackbot run pipeline

spackbot-app · 2022-10-11T10:38:55Z

I've started that pipeline for you!

* spack.compiler.Compiler: introduce prefix property We currently don't really have something that gives the GCC install path, which is used by many LLVM-based compilers (llvm, llvm-amdgpu, nvhpc, ...) to fix the GCC toolchain once and for all. This `prefix` property is dynamic in the sense that it queries the compiler itself. This is necessary because it's not easy to deduce the install path from the `cc` property (might be a symlink, might be a filename like `gcc` which works by having the compiler load a module that sets the PATH variable, might be a generic compiler wrapper based on environment variables like on cray...). With this property introduced, we can clean up some recipes that have the logic repeated for GCC. * intel-oneapi-compilers: set --gcc-sysroot to %gcc prefix

stephenmsachs · 2022-10-14T12:10:13Z

This PR breaks cmake%intel. Reproducer:

git clone https://github.com/spack/spack.git
cd spack/
source share/spack/setup-env.sh 
spack install intel-oneapi-compilers
. "$(spack location -i intel-oneapi-compilers)/setvars.sh"
spack compiler add --scope site
spack install cmake%intel

Output:

==> Installing cmake-3.24.2-kagbqz6qrgbdgwfwqoa45yloqn5fzrzd
==> No binary for cmake-3.24.2-kagbqz6qrgbdgwfwqoa45yloqn5fzrzd found: installing from source
==> Warning: Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors
==> Fetching https://github.com/Kitware/CMake/releases/download/v3.24.2/cmake-3.24.2.tar.gz
==> No patches needed for cmake   
==> cmake: Executing phase: 'bootstrap'
==> Error: ProcessError: Command exited with status 11:
    './bootstrap' '--prefix=/tmp/spack/opt/spack/linux-amzn2-zen2/intel-2021.7.0/cmake-3.24.2-kagbqz6qrgbdgwfwqoa45yloqn5fzrzd' '--parallel=16' '--no-system-libs' '--no-qt-gui' '--' '-DCMAKE_BUILD_TYPE=Release' '-DCMake_TEST_INSTALL=OFF' '-DCMAKE_USE_OPENSSL=ON' '-DBUILD_CursesDialog=True' '-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=ON' '-DCMAKE_INSTALL_RPATH=/tmp/spack/opt/spack/linux-amzn2-zen2/intel-2021.7.0/cmake-3.24.2-kagbqz6qrgbdgwfwqoa45yloqn5fzrzd/lib;/tmp/spack/opt/spack/linux-amzn2-zen2/intel-2021.7.0/cmake-3.24.2-kagbqz6qrgbdgwfwqoa45yloqn5fzrzd/lib64;/tmp/spack/opt/spack/linux-amzn2-zen2/intel-2021.7.0/ncurses-6.3-5sdhoxvcggmeysvawbmz2jmzd3z5u3fd/lib;/tmp/spack/opt/spack/linux-amzn2-zen2/intel-2021.7.0/openssl-1.1.1q-lyvkhdkeemntujkxfzw2ux4iyjujce6c/lib;/tmp/spack/opt/spack/linux-amzn2-zen2/intel-2021.7.0/zlib-1.2.12-uoldnmbrzup76pr4lfwfwh7a2d2kzk66/lib' '-DCMAKE_PREFIX_PATH=/tmp/spack/opt/spack/linux-amzn2-zen2/intel-2021.7.0/openssl-1.1.1q-lyvkhdkeemntujkxfzw2ux4iyjujce6c;/tmp/spack/opt/spack/linux-amzn2-zen2/intel-2021.7.0/zlib-1.2.12-uoldnmbrzup76pr4lfwfwh7a2d2kzk66;/tmp/spack/opt/spack/linux-amzn2-zen2/intel-2021.7.0/ncurses-6.3-5sdhoxvcggmeysvawbmz2jmzd3z5u3fd'

2 errors found in build log:
     839    -- Checking if compiler supports C++ make_unique
     840    -- Checking if compiler supports C++ make_unique - no
     841    -- Checking if compiler supports C++ unique_ptr
     842    -- Checking if compiler supports C++ unique_ptr - no
     843    -- Checking if compiler supports C++ filesystem
     844    -- Checking if compiler supports C++ filesystem - no
  >> 845    CMake Error at CMakeLists.txt:112 (message):
     846      The C++ compiler does not support C++11 (e.g.  std::unique_ptr).
     847
     848
     849    -- Configuring incomplete, errors occurred!
     850    See also "/tmp/ec2-user/spack-stage/spack-stage-cmake-3.24.2-kagbqz6qrgbdgwfwqoa45yloqn5fzrzd/spack-src/CMakeFiles/CMakeOutput.log".
     851    See also "/tmp/ec2-user/spack-stage/spack-stage-cmake-3.24.2-kagbqz6qrgbdgwfwqoa45yloqn5fzrzd/spack-src/CMakeFiles/CMakeError.log".
     852    ---------------------------------------------
  >> 853    Error when bootstrapping CMake:
     854    Problem while running initial CMake
     855    ---------------------------------------------

The cmake checks for make_unique, etc. also produce 'yes' before this PR.

rscohn2 · 2022-10-14T12:36:39Z

Does #33281 (comment) fix it?

I can't reproduce the problem. I guess it depends on the system gcc available.

stephenmsachs · 2022-10-14T12:49:24Z

Does #33281 (comment) fix it?

Unfortunately no.

I can't reproduce the problem. I guess it depends on the system gcc available.

Yes, you are probably right. This is gcc-7.3.1

rscohn2 · 2022-10-14T13:05:54Z

I expected that your PR would undo the problem that this PR caused for icc and restore old behavior, but something else seems to be happening. I am installing old compilers so I can try to reproduce the problem. Can you upload the logs and env files for the cmake build that failed?

stephenmsachs · 2022-10-14T13:11:56Z

I'll get to it right away. I am installing intel-oneapi-compilers on top of newer gcc right now to see whether it eliminates the cmake issue.

stephenmsachs · 2022-10-14T13:34:47Z

I get the same error for :

-- linux-amzn2-zen2 / [email protected] --------------------------------
[email protected]

here are the logs:
CMakeError.log
CMakeOutput.log
spack-build-env.txt
spack-build-out.txt

rscohn2 · 2022-10-14T13:54:44Z

I think I understand the problem:

/tmp/spack/lib/spack/env/intel/icpc   -std=gnu++17 -MD -MT CMakeFiles/cmTC_eb3ad.dir/cm_cxx_filesystem.cxx.o -MF CMakeFiles/cmTC_eb3ad.dir/cm_cxx_filesystem.cxx.o.d -o CMakeFiles/cmTC_eb3ad.dir/cm_cxx_filesystem.cxx.o -c /tmp/ec2-user/spack-stage/spack-stage-cmake-3.24.2-kagbqz6qrgbdgwfwqoa45yloqn5fzrzd/spack-src/Source/Checks/cm_cxx_filesystem.cxx
icpc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
icpc: command line warning #10148: option '--gcc-toolchain=/tmp/spack/opt/spack/linux-amzn2-zen/gcc-7.3.1/gcc-12.2.0-rzixe57jqsugv3xxll4nwglzkix4bst2' not supported
/tmp/ec2-user/spack-stage/spack-stage-cmake-3.24.2-kagbqz6qrgbdgwfwqoa45yloqn5fzrzd/spack-src/Source/Checks/cm_cxx_filesystem.cxx(2): catastrophic error: cannot open source file "filesystem"
  #include <filesystem>

It is trying to test for c++17 support and failing. The --gcc-toolchain would have helped, but it works for icpx and not icpc. It needs to use a different option when the compiler is icpc.

To work around it, I think you can spack load [email protected] so a more modern gcc is in your path. You can verify which compiler icpc is using this way:

rscohn1@anpfclxlin02:spack$ icpc ../test.cpp -v
icpc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second ha\
lf of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this c\
ompiler. Use '-diag-disable=10441' to disable this message.
icpc version 2021.7.0 (gcc version 7.3.0 compatibility)

stephenmsachs · 2022-10-17T13:10:14Z

You are right. I added this use case to #33281.

haampie added 2 commits October 10, 2022 14:20

intel-oneapi-compilers: set --gcc-sysroot to %gcc prefix

cc9ac59

spackbot-app bot added compilers conflicts core PR affects Spack core functionality intel update-package labels Oct 10, 2022

spackbot-app bot requested review from renjithravindrankannath, rscohn2, srekolam, trws and vlkale October 10, 2022 12:24

fix style

0ada281

tldahlgren assigned haampie Oct 10, 2022

trws reviewed Oct 10, 2022

View reviewed changes

trws mentioned this pull request Oct 10, 2022

Iintrospect clang and have it report its chosen gcc toolchain #33176

Open

trws approved these changes Oct 10, 2022

View reviewed changes

haampie enabled auto-merge (squash) October 11, 2022 10:39

haampie mentioned this pull request Oct 11, 2022

Intel oneapi compilers: silence linker flag warnings for old versions #33192

Merged

haampie merged commit 926dca9 into spack:develop Oct 11, 2022

haampie deleted the fix/gcc-prefix-in-llvm-based-compilers branch October 12, 2022 11:37

Conversation

haampie commented Oct 10, 2022

Uh oh!

haampie commented Oct 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trws Oct 10, 2022

Choose a reason for hiding this comment

Uh oh!

trws Oct 10, 2022

Choose a reason for hiding this comment

Uh oh!

haampie Oct 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

haampie Oct 10, 2022

Choose a reason for hiding this comment

Uh oh!

trws Oct 10, 2022

Choose a reason for hiding this comment

Uh oh!

haampie Oct 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

haampie Oct 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

haampie commented Oct 11, 2022

Uh oh!

spackbot-app bot commented Oct 11, 2022

Uh oh!

stephenmsachs commented Oct 14, 2022

Uh oh!

rscohn2 commented Oct 14, 2022

Uh oh!

stephenmsachs commented Oct 14, 2022

Uh oh!

rscohn2 commented Oct 14, 2022

Uh oh!

stephenmsachs commented Oct 14, 2022

Uh oh!

stephenmsachs commented Oct 14, 2022

Uh oh!

rscohn2 commented Oct 14, 2022

Uh oh!

stephenmsachs commented Oct 17, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

haampie commented Oct 10, 2022 •

edited

Loading

haampie Oct 10, 2022 •

edited

Loading

haampie Oct 10, 2022 •

edited

Loading

haampie Oct 10, 2022 •

edited

Loading