Skip to content

mgard@2023-03-31 +cuda cuda_arch=90 fails build: Compressor_1D_double_CUDA.cpp.o] Error 139: Segmentation fault (core dumped) #38558

@eugeneswalker

Description

@eugeneswalker

Steps to reproduce the issue

mgard@2023-03-31 +serial +openmp +timing +unstructured +cuda cuda_arch=90 fails to build using:

  • spack@develop (5c6c3b4 from Sun Jun 18 07:17:19 2023 -0400)
  • GCC 11.1.0
  • CUDA 11.8.0
  • Ubuntu 20.04, X86_64

Concrete spec: mgard.spec.json.txt

Concretization
 -   mgard@2023-03-31%[email protected]+cuda~ipo+openmp+serial+timing+unstructured build_system=cmake build_type=Release cuda_arch=90 generator=make arch=linux-ubuntu20.04-x86_64
[+]      ^[email protected]%[email protected]~doc+ncurses+ownlibs~qt build_system=generic build_type=Release arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected]~symlinks+termlib abi=none build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected]~docs~shared build_system=generic certs=mozilla arch=linux-ubuntu20.04-x86_64
[+]              ^ca-certificates-mozilla@2023-01-10%[email protected] build_system=generic arch=linux-ubuntu20.04-x86_64
[+]              ^[email protected]%[email protected]+cpanm+open+shared+threads build_system=generic arch=linux-ubuntu20.04-x86_64
[+]                  ^[email protected]%[email protected]+cxx~docs+stl build_system=autotools patches=26090f4,b231fcc arch=linux-ubuntu20.04-x86_64
[+]      ^[email protected]%[email protected]~allow-unsupported-compilers~dev build_system=generic arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected]~python build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]      ^[email protected]%[email protected]~guile build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]      ^[email protected]%[email protected]+iconv build_system=autotools compression=bz2lib,lz4,lzma,lzo2,zlib,zstd crypto=mbedtls libs=shared,static programs=none xar=libxml2 arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected]~debug~pic+shared build_system=generic arch=linux-ubuntu20.04-x86_64
[+]              ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected] build_system=autotools libs=shared,static arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected] build_system=makefile libs=shared,static arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected] build_system=autotools libs=shared,static arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected]+pic build_system=makefile build_type=Release libs=static arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected]+pic build_system=autotools libs=shared,static arch=linux-ubuntu20.04-x86_64
[+]      ^[email protected]%[email protected]+cuda~ipo build_system=cmake build_type=Release cuda_arch=none generator=make arch=linux-ubuntu20.04-x86_64
[+]      ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]      ^[email protected]%[email protected]~ipo+shared build_system=cmake build_type=Release generator=make arch=linux-ubuntu20.04-x86_64
[+]      ^[email protected]%[email protected]+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=0d98e93,f2fd060 arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected]+libbsd build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]              ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]                  ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected]+bzip2+curses+git~libunistring+libxml2+tar+xz build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]              ^[email protected]%[email protected] build_system=autotools zip=pigz arch=linux-ubuntu20.04-x86_64
[+]                  ^[email protected]%[email protected] build_system=makefile arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected]~obsolete_api build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected] build_system=autotools patches=bbf97f1 arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected]+column_metadata+dynamic_extensions+fts~functions+rtree build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]          ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]      ^[email protected]%[email protected] build_system=autotools arch=linux-ubuntu20.04-x86_64
[+]      ^[email protected]%[email protected]~ipo+pic+shared~tests build_system=cmake build_type=Release generator=make arch=linux-ubuntu20.04-x86_64
[+]      ^[email protected]%[email protected]+optimize+pic+shared build_system=makefile arch=linux-ubuntu20.04-x86_64
[+]      ^[email protected]%[email protected]+programs build_system=makefile compression=none libs=shared,static arch=linux-ubuntu20.04-x86_64

Error:

$> spack install -f ./mgard.spec.json
...
==> Installing mgard-2023-03-31-wvacdfy4ajcqoscn2ilmsffb27sazhhk
==> No binary for mgard-2023-03-31-wvacdfy4ajcqoscn2ilmsffb27sazhhk found: installing from source
==> No patches needed for mgard
==> mgard: Executing phase: 'cmake'
==> mgard: Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    '/spack/opt/spack/linux-ubuntu20.04-x86_64/gcc-11.1.0/gmake-4.4.1-lp2ol6q2quzz7sjtfkrxeczlsvzukkzk/bin/make' '-j16'

26 errors found in build log:
     1165    [ 98%] Building CUDA object CMakeFiles/mgard-library.dir/src/mgard-x/CompressionHighLevel/Compress_CUDA.cpp.o
     1166    /spack/opt/spack/linux-ubuntu20.04-x86_64/gcc-11.1.0/cuda-11.8.0-e5qmi3ynerocnxupx3epemw25cezxkoo/bin/nvcc -forward-unknown-to-host-compiler -ccbin=/usr/bin/g++ -DMGARD_ENABLE_CUDA -D
             MGARD_ENABLE_OPENMP -DMGARD_ENABLE_SERIAL -DMGARD_TIMING -DMGARD_ZSTD -Dmgard_library_EXPORTS --options-file CMakeFiles/mgard-library.dir/includes_CUDA.rsp --expt-relaxed-constexpr -w
              --generate-line-info -O3 -DNDEBUG -std=c++17 --generate-code=arch=compute_90,code=[compute_90,sm_90] -Xcompiler=-fPIC -Xcompiler -fopenmp -MD -MT CMakeFiles/mgard-library.dir/src/mga
             rd-x/CompressionHighLevel/Compress_CUDA.cpp.o -MF CMakeFiles/mgard-library.dir/src/mgard-x/CompressionHighLevel/Compress_CUDA.cpp.o.d -x cu -rdc=true -c /tmp/root/spack-stage/spack-st
             age-mgard-2023-03-31-wvacdfy4ajcqoscn2ilmsffb27sazhhk/spack-build-wvacdfy/src/mgard-x/CompressionHighLevel/Compress_CUDA.cpp -o CMakeFiles/mgard-library.dir/src/mgard-x/CompressionHig
             hLevel/Compress_CUDA.cpp.o
     1167    [ 98%] Building CUDA object CMakeFiles/mgard-library.dir/src/mgard-x/CompressionHighLevel/Decompress_CUDA.cpp.o
     1168    /spack/opt/spack/linux-ubuntu20.04-x86_64/gcc-11.1.0/cuda-11.8.0-e5qmi3ynerocnxupx3epemw25cezxkoo/bin/nvcc -forward-unknown-to-host-compiler -ccbin=/usr/bin/g++ -DMGARD_ENABLE_CUDA -D
             MGARD_ENABLE_OPENMP -DMGARD_ENABLE_SERIAL -DMGARD_TIMING -DMGARD_ZSTD -Dmgard_library_EXPORTS --options-file CMakeFiles/mgard-library.dir/includes_CUDA.rsp --expt-relaxed-constexpr -w
              --generate-line-info -O3 -DNDEBUG -std=c++17 --generate-code=arch=compute_90,code=[compute_90,sm_90] -Xcompiler=-fPIC -Xcompiler -fopenmp -MD -MT CMakeFiles/mgard-library.dir/src/mga
             rd-x/CompressionHighLevel/Decompress_CUDA.cpp.o -MF CMakeFiles/mgard-library.dir/src/mgard-x/CompressionHighLevel/Decompress_CUDA.cpp.o.d -x cu -rdc=true -c /tmp/root/spack-stage/spac
             k-stage-mgard-2023-03-31-wvacdfy4ajcqoscn2ilmsffb27sazhhk/spack-build-wvacdfy/src/mgard-x/CompressionHighLevel/Decompress_CUDA.cpp -o CMakeFiles/mgard-library.dir/src/mgard-x/Compress
             ionHighLevel/Decompress_CUDA.cpp.o
     1169    [ 98%] Building CUDA object CMakeFiles/mgard-library.dir/src/mgard-x/RuntimeX/DeviceAdapters/DeviceAdapterCuda.cu.o
     1170    /spack/opt/spack/linux-ubuntu20.04-x86_64/gcc-11.1.0/cuda-11.8.0-e5qmi3ynerocnxupx3epemw25cezxkoo/bin/nvcc -forward-unknown-to-host-compiler -ccbin=/usr/bin/g++ -DMGARD_ENABLE_CUDA -D
             MGARD_ENABLE_OPENMP -DMGARD_ENABLE_SERIAL -DMGARD_TIMING -DMGARD_ZSTD -Dmgard_library_EXPORTS --options-file CMakeFiles/mgard-library.dir/includes_CUDA.rsp --expt-relaxed-constexpr -w
              --generate-line-info -O3 -DNDEBUG -std=c++17 --generate-code=arch=compute_90,code=[compute_90,sm_90] -Xcompiler=-fPIC -Xcompiler -fopenmp -MD -MT CMakeFiles/mgard-library.dir/src/mga
             rd-x/RuntimeX/DeviceAdapters/DeviceAdapterCuda.cu.o -MF CMakeFiles/mgard-library.dir/src/mgard-x/RuntimeX/DeviceAdapters/DeviceAdapterCuda.cu.o.d -x cu -rdc=true -c /tmp/root/spack-st
             age/spack-stage-mgard-2023-03-31-wvacdfy4ajcqoscn2ilmsffb27sazhhk/spack-src/src/mgard-x/RuntimeX/DeviceAdapters/DeviceAdapterCuda.cu -o CMakeFiles/mgard-library.dir/src/mgard-x/Runtim
             eX/DeviceAdapters/DeviceAdapterCuda.cu.o
  >> 1171    Segmentation fault (core dumped)
  >> 1172    make[2]: *** [CMakeFiles/mgard-library.dir/build.make:7504: CMakeFiles/mgard-library.dir/src/mgard-x/CompressionLowLevel/Compressor_1D_double_CUDA.cpp.o] Error 139
     1173    make[2]: *** Waiting for unfinished jobs....
  >> 1174    Segmentation fault (core dumped)
  >> 1175    make[2]: *** [CMakeFiles/mgard-library.dir/build.make:7534: CMakeFiles/mgard-library.dir/src/mgard-x/CompressionLowLevel/Compressor_3D_double_CUDA.cpp.o] Error 139
  >> 1176    Segmentation fault (core dumped)
  >> 1177    make[2]: *** [CMakeFiles/mgard-library.dir/build.make:7519: CMakeFiles/mgard-library.dir/src/mgard-x/CompressionLowLevel/Compressor_2D_double_CUDA.cpp.o] Error 139
  >> 1178    Segmentation fault (core dumped)
  >> 1179    make[2]: *** [CMakeFiles/mgard-library.dir/build.make:7549: CMakeFiles/mgard-library.dir/src/mgard-x/CompressionLowLevel/Compressor_4D_double_CUDA.cpp.o] Error 139
  >> 1180    Segmentation fault (core dumped)
  >> 1181    make[2]: *** [CMakeFiles/mgard-library.dir/build.make:7564: CMakeFiles/mgard-library.dir/src/mgard-x/CompressionLowLevel/Compressor_5D_double_CUDA.cpp.o] Error 139
  >> 1182    Segmentation fault (core dumped)
  >> 1183    make[2]: *** [CMakeFiles/mgard-library.dir/build.make:7579: CMakeFiles/mgard-library.dir/src/mgard-x/CompressionLowLevel/Compressor_1D_float_CUDA.cpp.o] Error 139
  >> 1184    Segmentation fault (core dumped)
  >> 1185    make[2]: *** [CMakeFiles/mgard-library.dir/build.make:7594: CMakeFiles/mgard-library.dir/src/mgard-x/CompressionLowLevel/Compressor_2D_float_CUDA.cpp.o] Error 139
  >> 1186    Segmentation fault (core dumped)
  >> 1187    make[2]: *** [CMakeFiles/mgard-library.dir/build.make:7639: CMakeFiles/mgard-library.dir/src/mgard-x/CompressionLowLevel/Compressor_5D_float_CUDA.cpp.o] Error 139
  >> 1188    Segmentation fault (core dumped)
  >> 1189    make[2]: *** [CMakeFiles/mgard-library.dir/build.make:7624: CMakeFiles/mgard-library.dir/src/mgard-x/CompressionLowLevel/Compressor_4D_float_CUDA.cpp.o] Error 139
  >> 1190    Segmentation fault (core dumped)
  >> 1191    make[2]: *** [CMakeFiles/mgard-library.dir/build.make:7609: CMakeFiles/mgard-library.dir/src/mgard-x/CompressionLowLevel/Compressor_3D_float_CUDA.cpp.o] Error 139
  >> 1192    Segmentation fault (core dumped)
  >> 1193    make[2]: *** [CMakeFiles/mgard-library.dir/build.make:7669: CMakeFiles/mgard-library.dir/src/mgard-x/CompressionHighLevel/Decompress_CUDA.cpp.o] Error 139
  >> 1194    Segmentation fault (core dumped)
  >> 1195    make[2]: *** [CMakeFiles/mgard-library.dir/build.make:7654: CMakeFiles/mgard-library.dir/src/mgard-x/CompressionHighLevel/Compress_CUDA.cpp.o] Error 139
     1196    make[2]: Leaving directory '/tmp/root/spack-stage/spack-stage-mgard-2023-03-31-wvacdfy4ajcqoscn2ilmsffb27sazhhk/spack-build-wvacdfy'
  >> 1197    make[1]: *** [CMakeFiles/Makefile2:954: CMakeFiles/mgard-library.dir/all] Error 2
     1198    make[1]: Leaving directory '/tmp/root/spack-stage/spack-stage-mgard-2023-03-31-wvacdfy4ajcqoscn2ilmsffb27sazhhk/spack-build-wvacdfy'
  >> 1199    make: *** [Makefile:139: all] Error 2

Error message

Error message
see above

Information on your system

  • Spack: 0.21.0.dev0 (10fd13d)
  • Python: 3.8.10
  • Platform: linux-ubuntu20.04-cascadelake
  • Concretizer: clingo

Additional information

spack-build-env.txt
spack-build-out.txt

@robertu94 @wspear @nchaimov

General information

  • I have run spack debug report and reported the version of Spack/Python/Platform
  • I have run spack maintainers <name-of-the-package> and @mentioned any maintainers
  • I have uploaded the build log and environment files
  • I have searched the issues of this repo and believe this is not a duplicate

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions