e4s ci: add umpire +rocm by eugeneswalker · Pull Request #32504 · spack/spack

eugeneswalker · 2022-09-03T10:15:56Z

Hoping that the merge of #32469 has fixed the issue building umpire +rocm

eugeneswalker · 2022-09-08T16:52:54Z

umpire +rocm is failing :

==> Installing umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr
==> No binary for umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr found: installing from source
==> No patches needed for umpire
==> umpire: Executing phase: 'initconfig'
==> umpire: Executing phase: 'cmake'
==> Error: ProcessError: Command exited with status 1:
    '/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/cmake-3.20.6-vymo2adfcdz2lx7647b2qw2c3farbkrb/bin/cmake' '-G' 'Unix Makefiles' '-DCMAKE_INSTALL_PREFIX:STRING=/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr' '-DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo' '-DBUILD_TESTING:BOOL=OFF' '-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF' '-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON' '-DCMAKE_INSTALL_RPATH_USE_LINK_PATH:BOOL=ON' '-DCMAKE_INSTALL_RPATH:STRING=/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/lib64;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/camp-2022.03.2-ecqqe5utykau2mdqqq7zsjg6tgwoyv5k/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/hip-5.2.0-bv7urfoqfdriepvm6ytgd3wubhv4smbv/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/comgr-5.2.0-vxehasfollvkwk5yfx47wexxe2rm3rar/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/llvm-amdgpu-5.2.0-v5fipzrqmx5aiwvymed7i5qjngj4ooah/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/ncurses-6.3-hbmtrd4qgjwpmulbo3oq3gijcx6l6dbt/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/z3-4.8.16-l5sunkjouxf7gyx32iyceyin7lf7wdao/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/zlib-1.2.12-edugrkv3cwwfvj7um4p4pcb77joh6bun/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/mesa-22.1.2-qlaqystynt5cduyrbrdelyv5ekb2pj7x/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/expat-2.4.8-4nhsan54zuhn5xyrhvfcswwyspzmk4fz/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libbsd-0.11.5-z4wmj5q2x6qmyhsu2adra2g4klzs2w5t/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libmd-1.0.4-s4jxh5c7u552tyyabmulx26zbpbsvcme/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/glproto-1.4.17-4nsecaypsbkow7b526uftffnxprkchdy/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libunwind-1.6.2-tkuyd6pc5mbhv5xkx5fm2iazhi7p4rog/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/xz-5.2.5-vs3p73wz5lhzwf66lt3bjlfx6zbudpze/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libx11-1.7.0-aqpxumqoryhymuulfl6lvxx5asibfi2f/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/inputproto-2.3.2-nccbhlzv47qugswdsem3baqjeoafjhvy/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/kbproto-1.0.7-zsoykodpudmncvbclmr7a2ero6khhnyn/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxcb-1.14-q4ubmiiuctmntaujpjk5qo2ayklvjjio/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libpthread-stubs-0.4-gbdhl7qdkuyctqs7xgnzet6nkv7ib36i/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxau-1.0.8-2nobcfiynx2sahk6g55qjaqur4rh5m5j/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/xproto-7.0.31-qj7l6eeauiiwsuk5kx426feo4qsjdqg7/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxdmcp-1.1.2-y5ebqkxo2je7uyy2pceonkfudvmzovzm/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/xcb-proto-1.14.1-64ag6yv3scoh3y3zq224tti2iqcxvfye/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/xextproto-7.3.0-ale4anqj6talw5375zowgx5gvfqy7huo/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxext-1.3.3-2kf7jc2ewswrronlalwpsnekfecvesu2/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxt-1.1.5-bbg3dtdxpqaty62w3r37efkmijen5jhv/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libice-1.0.9-msx6pxsrvca7e2aijb5uczohmfj3e27e/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libsm-1.2.3-y3d4mdcktfu4sazh6dz7syzn3xh5du6f/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/util-linux-uuid-2.37.4-biegun4vnqnkmvanpn3o25pasfmp3w6v/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxrandr-1.5.0-3susvjonas2gcqbtpmi6xuv5e7jy35wy/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxrender-0.9.10-5evwzcccjc7ck3ee75cgm7wkaaf2cdy6/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/renderproto-0.11.1-y6hlb667lwb4ll6tdxliz6ghik7n2zpp/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/randrproto-1.5.0-jtsawq65lnncp7ucywabo7dguf2vamrq/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/hsa-rocr-dev-5.2.0-2ijb2sokr6r5b2ripmqx7qkmixy6q7k2/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/elfutils-0.186-6z236ur4sdwrlv4ao2z6zetnrp33vlyh/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/bzip2-1.0.8-ufk3ruarnyb7d5vkutpwgozseljwzmlw/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/hsakmt-roct-5.2.0-lvzsuz2rowqky2vpajmaj7aihbkekquj/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libdrm-2.4.110-ftrbvcmpiaclh2g5zupw225ayiygp65n/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libpciaccess-0.16-qdz4q24wgopy4sj4w4ohihrxprrpu5ti/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/numactl-2.0.14-rykeckz6wf6edsmp6offpic56t5yiln3/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/perl-file-which-1.22-itncgpi5ayb4222jdgowy5bpphjgbzgz/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/perl-uri-encode-1.1.1-lalgetywo326r3kladdidb2utec6lmye/lib;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/openssl-1.1.1q-pozs2xtnqopa3t2vj5alepgc23bnd36o/lib' '-DCMAKE_PREFIX_PATH:STRING=/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/camp-2022.03.2-ecqqe5utykau2mdqqq7zsjg6tgwoyv5k;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/hip-5.2.0-bv7urfoqfdriepvm6ytgd3wubhv4smbv;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/roctracer-dev-api-5.2.0-7l7i7mh7d5eczqd42w3zi3lmzkm6456t;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/rocminfo-5.2.0-mbgbwafz2zbvtg7fa3lfcjtipzd3rmbp;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/perl-uri-encode-1.1.1-lalgetywo326r3kladdidb2utec6lmye;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/perl-file-which-1.22-itncgpi5ayb4222jdgowy5bpphjgbzgz;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/hsa-rocr-dev-5.2.0-2ijb2sokr6r5b2ripmqx7qkmixy6q7k2;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/hsakmt-roct-5.2.0-lvzsuz2rowqky2vpajmaj7aihbkekquj;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/numactl-2.0.14-rykeckz6wf6edsmp6offpic56t5yiln3;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libdrm-2.4.110-ftrbvcmpiaclh2g5zupw225ayiygp65n;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libpciaccess-0.16-qdz4q24wgopy4sj4w4ohihrxprrpu5ti;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/elfutils-0.186-6z236ur4sdwrlv4ao2z6zetnrp33vlyh;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/glx-1.4-y5nrf25ypsqmu7e52dexbxh2cwvnayb2;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/mesa-22.1.2-qlaqystynt5cduyrbrdelyv5ekb2pj7x;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/xrandr-1.5.0-4mhm43d7pt7o2cix44u7mkkg2suw3uop;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxrandr-1.5.0-3susvjonas2gcqbtpmi6xuv5e7jy35wy;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/randrproto-1.5.0-jtsawq65lnncp7ucywabo7dguf2vamrq;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxrender-0.9.10-5evwzcccjc7ck3ee75cgm7wkaaf2cdy6;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/renderproto-0.11.1-y6hlb667lwb4ll6tdxliz6ghik7n2zpp;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxt-1.1.5-bbg3dtdxpqaty62w3r37efkmijen5jhv;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libsm-1.2.3-y3d4mdcktfu4sazh6dz7syzn3xh5du6f;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libice-1.0.9-msx6pxsrvca7e2aijb5uczohmfj3e27e;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxext-1.3.3-2kf7jc2ewswrronlalwpsnekfecvesu2;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libx11-1.7.0-aqpxumqoryhymuulfl6lvxx5asibfi2f;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/xtrans-1.3.5-wexmmr2ztsid4m7iemqw5x45nyzs7gng;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/xextproto-7.3.0-ale4anqj6talw5375zowgx5gvfqy7huo;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxcb-1.14-q4ubmiiuctmntaujpjk5qo2ayklvjjio;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/xcb-proto-1.14.1-64ag6yv3scoh3y3zq224tti2iqcxvfye;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxdmcp-1.1.2-y5ebqkxo2je7uyy2pceonkfudvmzovzm;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libxau-1.0.8-2nobcfiynx2sahk6g55qjaqur4rh5m5j;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/xproto-7.0.31-qj7l6eeauiiwsuk5kx426feo4qsjdqg7;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libpthread-stubs-0.4-gbdhl7qdkuyctqs7xgnzet6nkv7ib36i;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/kbproto-1.0.7-zsoykodpudmncvbclmr7a2ero6khhnyn;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/inputproto-2.3.2-nccbhlzv47qugswdsem3baqjeoafjhvy;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libunwind-1.6.2-tkuyd6pc5mbhv5xkx5fm2iazhi7p4rog;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/glproto-1.4.17-4nsecaypsbkow7b526uftffnxprkchdy;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/comgr-5.2.0-vxehasfollvkwk5yfx47wexxe2rm3rar;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/llvm-amdgpu-5.2.0-v5fipzrqmx5aiwvymed7i5qjngj4ooah;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/z3-4.8.16-l5sunkjouxf7gyx32iyceyin7lf7wdao;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/util-linux-uuid-2.37.4-biegun4vnqnkmvanpn3o25pasfmp3w6v;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/xz-5.2.5-vs3p73wz5lhzwf66lt3bjlfx6zbudpze;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/expat-2.4.8-4nhsan54zuhn5xyrhvfcswwyspzmk4fz;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libbsd-0.11.5-z4wmj5q2x6qmyhsu2adra2g4klzs2w5t;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/libmd-1.0.4-s4jxh5c7u552tyyabmulx26zbpbsvcme;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/blt-0.5.1-zpuz2d4fpb6zreileo7k4tebkhimhje6;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/cmake-3.20.6-vymo2adfcdz2lx7647b2qw2c3farbkrb;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/openssl-1.1.1q-pozs2xtnqopa3t2vj5alepgc23bnd36o;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/zlib-1.2.12-edugrkv3cwwfvj7um4p4pcb77joh6bun;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/bzip2-1.0.8-ufk3ruarnyb7d5vkutpwgozseljwzmlw;/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/ncurses-6.3-hbmtrd4qgjwpmulbo3oq3gijcx6l6dbt' '-C' '/tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src/[email protected]' '/tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src'

2 errors found in build log:
  >> 5     CMake Error at 3f5e0dd1c1fa-linux-ubuntu22.04-x86_64-gcc@@11.2.0.cmake:44:
     6       Parse error.  Expected a command name, got unquoted argument with text
     7       "-DHIP_CLANG_INCLUDE_PATH=/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/llvm-amdgpu-5.2.0-v5fipzrqmx5aiwvymed7i5qjngj4ooah/lib/clang/14.0.0/include".
     8
     9
  >> 10    CMake Error: Error processing file: /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src/3f5e0dd1c1fa-linux-ubuntu22.04
           -x86_64-gcc@@11.2.0.cmake
     11    -- Configuring incomplete, errors occurred!

Do you know what is going on @davidbeckingsale ?

davidbeckingsale · 2022-09-08T16:58:26Z

Yup, the hip_repair_options method in camp (

spack/var/spack/repos/builtin/packages/camp/package.py

Line 11 in 2cfac33

def hip_repair_options(options, spec):

) is not written to work with a CachedCMakePackage.

CC @trws

trws · 2022-09-08T18:17:39Z

Dang, yes, I see that. It needs to add the appropriate set command I suppose, very strange gitlab didn't catch that. It should be as simple as either making a second copy of the function or adding an option and using cmake_cache_path with the variable name and computed path in the existing one.

eugeneswalker · 2022-09-09T15:24:04Z

Dang, yes, I see that. It needs to add the appropriate set command I suppose, very strange gitlab didn't catch that. It should be as simple as either making a second copy of the function or adding an option and using cmake_cache_path with the variable name and computed path in the existing one.

I must admit I don't understand the issue. Is there an example I could reference? We could test the proposed change out in this PR here.

eugeneswalker · 2022-09-09T19:10:03Z

I've updated the umpire package in this PR (see 1aaf096) and will see how that works out here.

Based on my offline trial of this change, this gets us passed the cmake error and onto some bona fide errors?

==> Installing umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr
==> No binary for umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr found: installing from source
==> Using cached archive: /spack/var/spack/cache/_source-cache/git//LLNL/Umpire.git/v2022.03.1.tar.gz
==> Warning: Fetching from mirror without a checksum!
  This package is normally checked out from a version control system, but it has been archived on a spack mirror.  This means we cannot know a checksum for the tarball in advance. Be sure that your connection to this mirror is secure!
==> No patches needed for umpire
==> umpire: Executing phase: 'initconfig'
==> umpire: Executing phase: 'cmake'
==> umpire: Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    'make' '-j16'

10 errors found in build log:
     161    [ 13%] Built target umpire_tpl_judy
     162    [ 13%] Building CXX object src/umpire/util/CMakeFiles/umpire_util.dir/Logger.cpp.o
     163    cd /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-build-sinzdam/src/umpire/util && /spack/lib/spack/env/gcc/g++ -DCAMP_HAVE_HI
            P -I/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/camp-2022.03.2-ecqqe5utykau2mdqqq7zsjg6tgwoyv5k/include -I/tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sin
            zdam5gqzlttwty3uhzumovaoyrtlr/spack-src/src -I/tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-build-sinzdam/include -Wall -Wext
            ra      -O2 -g -DNDEBUG -fPIC -std=c++14 -MD -MT src/umpire/util/CMakeFiles/umpire_util.dir/Logger.cpp.o -MF CMakeFiles/umpire_util.dir/Logger.cpp.o.d -o CMakeFiles/umpi
            re_util.dir/Logger.cpp.o -c /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src/src/umpire/util/Logger.cpp
     164    [ 14%] Building CXX object src/umpire/resource/CMakeFiles/umpire_resource.dir/NullMemoryResource.cpp.o
     165    cd /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-build-sinzdam/src/umpire/resource && /spack/lib/spack/env/gcc/g++ -DCAMP_HAV
            E_HIP -D__HIP_PLATFORM_AMD__=1 -D__HIP_PLATFORM_HCC__=1 -I/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/camp-2022.03.2-ecqqe5utykau2mdqqq7zsjg6tgwoyv5k/include -I
            /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src/src -I/tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3
            uhzumovaoyrtlr/spack-build-sinzdam/include -isystem /spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/hip-5.2.0-bv7urfoqfdriepvm6ytgd3wubhv4smbv/include -Wall -Wextra
                  -O2 -g -DNDEBUG -fPIC -std=c++14 -MD -MT src/umpire/resource/CMakeFiles/umpire_resource.dir/NullMemoryResource.cpp.o -MF CMakeFiles/umpire_resource.dir/NullMemoryR
            esource.cpp.o.d -o CMakeFiles/umpire_resource.dir/NullMemoryResource.cpp.o -c /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-s
            rc/src/umpire/resource/NullMemoryResource.cpp
     166    /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src/src/umpire/util/io.cpp: In function 'void umpire::util::initialize_io(bool,
             bool)':
  >> 167    /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src/src/umpire/util/io.cpp:103:14: error: 'std::filesystem' has not been declar
            ed
     168      103 |         std::filesystem::path root_io_dir_path{root_io_dir};
     169          |              ^~~~~~~~~~
  >> 170    /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src/src/umpire/util/io.cpp:105:19: error: 'std::filesystem' has not been declar
            ed
     171      105 |         if (!std::filesystem::exists(root_io_dir_path) && (enable_log || enable_replay)) {
     172          |                   ^~~~~~~~~~
  >> 173    /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src/src/umpire/util/io.cpp:105:38: error: 'root_io_dir_path' was not declared i
            n this scope; did you mean 'root_io_dir'?
     174      105 |         if (!std::filesystem::exists(root_io_dir_path) && (enable_log || enable_replay)) {
     175          |                                      ^~~~~~~~~~~~~~~~
     176          |                                      root_io_dir
  >> 177    /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src/src/umpire/util/io.cpp:106:16: error: 'std::filesystem' has not been declar
            ed
     178      106 |           std::filesystem::create_directories(root_io_dir_path);
     179          |                ^~~~~~~~~~
     180    /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src/src/umpire/util/io.cpp: In function 'bool umpire::util::directory_exists(co
            nst string&)':
  >> 181    /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src/src/umpire/util/io.cpp:187:8: error: 'std::filesystem' has not been declare
            d
     182      187 |   std::filesystem::path fspath_path(path);
     183          |        ^~~~~~~~~~
  >> 184    /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src/src/umpire/util/io.cpp:188:15: error: 'std::filesystem' has not been declar
            ed
     185      188 |   return std::filesystem::exists(fspath_path);
     186          |               ^~~~~~~~~~
  >> 187    /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-sinzdam5gqzlttwty3uhzumovaoyrtlr/spack-src/src/umpire/util/io.cpp:188:34: error: 'fspath_path' was not declared in thi
            s scope
     188      188 |   return std::filesystem::exists(fspath_path);

eugeneswalker · 2022-09-14T19:50:30Z

@trws Can you advise, did I do this correct? Forgetting whether this is how we would ultimately want to do it, I'm trying to figure out whether the error I get after doing this is because my change is wrong, or whether it is a bona fide error that we need to deal with even if I did this right?

Change I made:

d7d78a3

Resulting error:

See log here: e4s ci: add umpire +rocm #32504 (comment)

trws · 2022-09-14T20:12:54Z

This change is fine, though it would probably be better to have a function for it since it's going to end up getting used in multiple packages. That error is from umpire's cmake mis-detecting the availability of std::filesystem though I think, which should be unrelated to this. @mcfadden8, any idea?

mcfadden8 · 2022-09-14T22:55:37Z

This change is fine, though it would probably be better to have a function for it since it's going to end up getting used in multiple packages. That error is from umpire's cmake mis-detecting the availability of std::filesystem though I think, which should be unrelated to this. @mcfadden8, any idea?

I agree that the std::filesystem error should not be related. Does this error happen without these changes? I've heard of std::filesystem errors being reported previously, but I thought that @davidbeckingsale may have dealt with it previously.

AFAIK, we are not seeing these in the Umpire CI, but that may be due to a missing configuration to be tested. Can someone provide me the spack spec being used so that I may see if I can reproduce this locally?

eugeneswalker · 2022-09-15T20:48:32Z

@mcfadden8

Can someone provide me the spack spec being used so that I may see if I can reproduce this locally?

The spec is umpire +rocm amdgpu_target=gfx90a - I'm attaching a full Spack environment that mimics the build as it is done here in the CI:

spack.yaml.txt

The CI build environment:

Uses Docker container image ecpe4s/ubuntu22.04-runner-x86_64:2022-08-01
Uses %[email protected]
Allows Spack to build the ROCm stack

Thank you!

eugeneswalker · 2022-09-22T22:03:51Z

Can someone provide me the spack spec being used so that I may see if I can reproduce this locally?

Any luck with this @mcfadden8 ?

mcfadden8 · 2022-09-22T22:51:46Z

Sorry @eugeneswalker, I have not yet had a chance to look. I'll try to take a look at this early next week.

mcfadden8 · 2022-09-28T20:53:51Z

Can someone provide me the spack spec being used so that I may see if I can reproduce this locally?

Any luck with this @mcfadden8 ?

I'm able to reproduce this with a simpler setup using a gcc11 container. I believe that @davidbeckingsale also reported seeing this a earlier. I'll look at it more closely and talk with David about it.

mcfadden8 · 2022-09-29T19:53:12Z

Update: (@trws, @davidbeckingsale, and @eugeneswalker)

The std::filesystem compile error seems to be happening because cmake says that std::filesystem is supported. It checks for std::filesystem through the use of check_cxx_source_compiles on a snippet of source code. Looking at the cmake logs I saw that -std=c++14 was missing from the compile line that cmake is using but is present in the compilation commands used for umpire. When I added -std=c++14 to the compile line that check_cxx_source_compiles generates, it fails too.

So what's left to figure out is why check_cxx_source_compiles is not including the same compilation flags that Umpire is using.

We have documented this in our internal bug tracking system and have begun working on a fix.

This is being worked on here: llnl/blt#599

eugeneswalker · 2022-09-30T19:58:47Z

We have documented this in our internal bug tracking system and have begun working on a fix.

This is being worked on here: LLNL/blt#599

Manually patching in LLNL/blt#599 resolves the std::filesystem error for me, and that leads me to:

...
  >> 387    g++: error: unrecognized command-line option '--rocm-path=/spack/opt/spack/linux-ubuntu22.04-cascadelake/gcc-11.2.0/hip-5.2.3-unhizofrmbtqv4nvknubhsw4t4wpeqnj'
  >> 388    g++: error: unrecognized command-line option '--offload-arch=gfx900'
...

See:

==> Installing umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn
==> No binary for umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn found: installing from source
==> No patches needed for umpire
==> umpire: Executing phase: 'initconfig'
==> umpire: Executing phase: 'cmake'
==> umpire: Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    'make' '-j16'

5 errors found in build log:
     381    cd /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-build-tmss4gq/examples/tutorial && /spack/lib/spack/env/gcc/g++ -DCAMP_HAVE_HIP -D__HIP_PL
            ATFORM_AMD__=1 -D__HIP_PLATFORM_HCC__=1 -I/tmp/root/spack-stage/spack-stage-umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-src/src -I/tmp/root/spack-stage/spack-stage-umpire-
            2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-build-tmss4gq/include -isystem /spack/opt/spack/linux-ubuntu22.04-cascadelake/gcc-11.2.0/hip-5.2.3-unhizofrmbtqv4nvknubhsw4t4wpeqnj/in
            clude -isystem /spack/opt/spack/linux-ubuntu22.04-cascadelake/gcc-11.2.0/camp-2022.03.2-mwar3y5ghukmikpv7zrc3lhvoepwnhq3/include -isystem /spack/opt/spack/linux-ubuntu22.04-cascadelak
            e/gcc-11.2.0/hsa-rocr-dev-5.2.3-ozaey6ffk7wukbe5mpbaqb6ujaallicz/include -Wall -Wextra   -std=c++14  -O2 -g -DNDEBUG -fPIE -std=c++14 -MD -MT examples/tutorial/CMakeFiles/tut_copy.dir
            /tut_copy.cpp.o -MF CMakeFiles/tut_copy.dir/tut_copy.cpp.o.d -o CMakeFiles/tut_copy.dir/tut_copy.cpp.o -c /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsr
            adyizn/spack-src/examples/tutorial/tut_copy.cpp
     382    [ 54%] Building CXX object examples/tutorial/CMakeFiles/tut_introspection.dir/tut_introspection.cpp.o
     383    cd /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-build-tmss4gq/examples && /spack/lib/spack/env/gcc/g++ -DCAMP_HAVE_HIP -D__HIP_PLATFORM_AM
            D__=1 -D__HIP_PLATFORM_HCC__=1 -I/tmp/root/spack-stage/spack-stage-umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-src/src -I/tmp/root/spack-stage/spack-stage-umpire-2022.03.1
            -tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-build-tmss4gq/include -isystem /spack/opt/spack/linux-ubuntu22.04-cascadelake/gcc-11.2.0/hip-5.2.3-unhizofrmbtqv4nvknubhsw4t4wpeqnj/include -is
            ystem /spack/opt/spack/linux-ubuntu22.04-cascadelake/gcc-11.2.0/camp-2022.03.2-mwar3y5ghukmikpv7zrc3lhvoepwnhq3/include -isystem /spack/opt/spack/linux-ubuntu22.04-cascadelake/gcc-11.
            2.0/hsa-rocr-dev-5.2.3-ozaey6ffk7wukbe5mpbaqb6ujaallicz/include -Wall -Wextra   -std=c++14  -O2 -g -DNDEBUG -fPIE -std=c++14 -MD -MT examples/CMakeFiles/use_after_free.dir/use_after_f
            ree.cpp.o -MF CMakeFiles/use_after_free.dir/use_after_free.cpp.o.d -o CMakeFiles/use_after_free.dir/use_after_free.cpp.o -c /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-tmss4gqp
            4flhlie5rn3f3cwzsradyizn/spack-src/examples/use_after_free.cpp
     384    cd /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-build-tmss4gq/examples/tutorial && /spack/lib/spack/env/gcc/g++ -DCAMP_HAVE_HIP -D__HIP_PL
            ATFORM_AMD__=1 -D__HIP_PLATFORM_HCC__=1 -I/tmp/root/spack-stage/spack-stage-umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-src/src -I/tmp/root/spack-stage/spack-stage-umpire-
            2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-build-tmss4gq/include -isystem /spack/opt/spack/linux-ubuntu22.04-cascadelake/gcc-11.2.0/hip-5.2.3-unhizofrmbtqv4nvknubhsw4t4wpeqnj/in
            clude -isystem /spack/opt/spack/linux-ubuntu22.04-cascadelake/gcc-11.2.0/camp-2022.03.2-mwar3y5ghukmikpv7zrc3lhvoepwnhq3/include -isystem /spack/opt/spack/linux-ubuntu22.04-cascadelak
            e/gcc-11.2.0/hsa-rocr-dev-5.2.3-ozaey6ffk7wukbe5mpbaqb6ujaallicz/include -Wall -Wextra   -std=c++14  -O2 -g -DNDEBUG -fPIE -std=c++14 -MD -MT examples/tutorial/CMakeFiles/tut_introspe
            ction.dir/tut_introspection.cpp.o -MF CMakeFiles/tut_introspection.dir/tut_introspection.cpp.o.d -o CMakeFiles/tut_introspection.dir/tut_introspection.cpp.o -c /tmp/root/spack-stage/s
            pack-stage-umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-src/examples/tutorial/tut_introspection.cpp
     385    [ 54%] Building CXX object examples/tutorial/CMakeFiles/tut_dynamic_pool_1.dir/tut_dynamic_pool_1.cpp.o
     386    cd /tmp/root/spack-stage/spack-stage-umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-build-tmss4gq/examples/tutorial && /spack/lib/spack/env/gcc/g++ -DCAMP_HAVE_HIP -D__HIP_PL
            ATFORM_AMD__=1 -D__HIP_PLATFORM_HCC__=1 -I/tmp/root/spack-stage/spack-stage-umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-src/src -I/tmp/root/spack-stage/spack-stage-umpire-
            2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-build-tmss4gq/include -isystem /spack/opt/spack/linux-ubuntu22.04-cascadelake/gcc-11.2.0/hip-5.2.3-unhizofrmbtqv4nvknubhsw4t4wpeqnj/in
            clude -isystem /spack/opt/spack/linux-ubuntu22.04-cascadelake/gcc-11.2.0/camp-2022.03.2-mwar3y5ghukmikpv7zrc3lhvoepwnhq3/include -isystem /spack/opt/spack/linux-ubuntu22.04-cascadelak
            e/gcc-11.2.0/hsa-rocr-dev-5.2.3-ozaey6ffk7wukbe5mpbaqb6ujaallicz/include -Wall -Wextra   -std=c++14  -O2 -g -DNDEBUG -fPIE -std=c++14 -MD -MT examples/tutorial/CMakeFiles/tut_dynamic_
            pool_1.dir/tut_dynamic_pool_1.cpp.o -MF CMakeFiles/tut_dynamic_pool_1.dir/tut_dynamic_pool_1.cpp.o.d -o CMakeFiles/tut_dynamic_pool_1.dir/tut_dynamic_pool_1.cpp.o -c /tmp/root/spack-s
            tage/spack-stage-umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-src/examples/tutorial/tut_dynamic_pool_1.cpp
  >> 387    g++: error: unrecognized command-line option '--rocm-path=/spack/opt/spack/linux-ubuntu22.04-cascadelake/gcc-11.2.0/hip-5.2.3-unhizofrmbtqv4nvknubhsw4t4wpeqnj'
  >> 388    g++: error: unrecognized command-line option '--offload-arch=gfx900'
  >> 389    make[2]: *** [examples/CMakeFiles/multi_device.dir/build.make:79: examples/CMakeFiles/multi_device.dir/multi_device.cpp.o] Error 1
     390    make[2]: Leaving directory '/tmp/root/spack-stage/spack-stage-umpire-2022.03.1-tmss4gqp4flhlie5rn3f3cwzsradyizn/spack-build-tmss4gq'
  >> 391    make[1]: *** [CMakeFiles/Makefile2:1515: examples/CMakeFiles/multi_device.dir/all] Error 2
     392    make[1]: *** Waiting for unfinished jobs....

eugeneswalker added e4s ROCm/hip Support for ROCm/hip labels Sep 3, 2022

spackbot-app bot added core PR affects Spack core functionality gitlab Issues related to gitlab integration labels Sep 3, 2022

eugeneswalker force-pushed the e4s-add-umpire-rocm branch from 61efb04 to cb78268 Compare September 9, 2022 19:08

spackbot-app bot added the update-package label Sep 9, 2022

eugeneswalker force-pushed the e4s-add-umpire-rocm branch 3 times, most recently from c9f18ef to 1aaf096 Compare September 9, 2022 19:25

alalazo force-pushed the e4s-add-umpire-rocm branch from 1aaf096 to d7d78a3 Compare September 12, 2022 09:09

This was referenced Sep 12, 2022

e4s ci: add chai +rocm #32506

Merged

e4s ci: add raja +rocm #32505

Merged

This was referenced Sep 14, 2022

Apply hip workaround for raja-framework #32469

Merged

Fix ROCm constraints for ginkgo@glu_experimental in HiOp #32499

Merged

eugeneswalker mentioned this pull request Sep 26, 2022

[email protected] (and raja) +rocm build fails: camp.hip_repair_option doesn't work with CachedCMakePackage #32830

Closed

4 tasks

msimberg mentioned this pull request Sep 27, 2022

Attempt to reenable HIP CI eth-cscs/DLA-Future#650

Closed

eugeneswalker force-pushed the e4s-add-umpire-rocm branch from d7d78a3 to f168d11 Compare September 28, 2022 18:15

mcfadden8 mentioned this pull request Sep 30, 2022

Explicitly set cxx standard for GNU compiler llnl/blt#599

Closed

eugeneswalker mentioned this pull request Sep 30, 2022

[email protected] bug affecting build of [email protected] +rocm: umpire/util/io.cpp:103:14: error: 'std::filesystem' has not been declared #32932

Closed

4 tasks

eugeneswalker force-pushed the e4s-add-umpire-rocm branch 2 times, most recently from 8859d52 to 60284ed Compare October 10, 2022 19:46

e4s ci: add umpire +rocm

09bbf2c

eugeneswalker force-pushed the e4s-add-umpire-rocm branch from 60284ed to 09bbf2c Compare October 19, 2022 19:56

eugeneswalker enabled auto-merge (squash) October 19, 2022 21:49

eugeneswalker requested a review from wspear October 19, 2022 21:49

wspear approved these changes Oct 19, 2022

View reviewed changes

eugeneswalker merged commit 8aac0d0 into spack:develop Oct 19, 2022

Conversation

eugeneswalker commented Sep 3, 2022

Uh oh!

eugeneswalker commented Sep 8, 2022

Uh oh!

davidbeckingsale commented Sep 8, 2022

Uh oh!

trws commented Sep 8, 2022

Uh oh!

eugeneswalker commented Sep 9, 2022

Uh oh!

eugeneswalker commented Sep 9, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eugeneswalker commented Sep 14, 2022

Uh oh!

trws commented Sep 14, 2022

Uh oh!

mcfadden8 commented Sep 14, 2022

Uh oh!

eugeneswalker commented Sep 15, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eugeneswalker commented Sep 22, 2022

Uh oh!

mcfadden8 commented Sep 22, 2022

Uh oh!

mcfadden8 commented Sep 28, 2022

Uh oh!

mcfadden8 commented Sep 29, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eugeneswalker commented Sep 30, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

eugeneswalker commented Sep 9, 2022 •

edited

Loading

eugeneswalker commented Sep 15, 2022 •

edited

Loading

mcfadden8 commented Sep 29, 2022 •

edited

Loading