Skip to content

plink-ng: new package#16750

Merged
alalazo merged 9 commits intospack:developfrom
ResearchIT:package/plink-ng
May 27, 2020
Merged

plink-ng: new package#16750
alalazo merged 9 commits intospack:developfrom
ResearchIT:package/plink-ng

Conversation

@codeandkey
Copy link
Copy Markdown
Contributor

@codeandkey codeandkey commented May 21, 2020

This package makes use of some standard BLAS and LAPACK functions but I was unable to get it working using the blas and lapack virtual dependencies. This recipe uses openblas to resolve all the needed functions but if the virtual dependencies are preferable then I can try more to get them working.

The two major problems I ran into were atlas build problems and it seemed that lapack did not actually bring any packages into the spec.

alalazo
alalazo requested changes May 22, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@alalazo alalazo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two major problems I ran into were atlas build problems and it seemed that lapack did not actually bring any packages into the spec.

Can you elaborate on the second issue? Many packages provide both lapack and blas so both dependencies are satisfied with a single package.

@codeandkey
Copy link
Copy Markdown
Contributor Author

codeandkey commented May 26, 2020

Here is the build log when using both virtual dependencies:

==> Error: Failed to install plink-ng due to ChildError: ProcessError: Command exited with status 2:
    '/usr/bin/sh' 'build.sh'
4 errors found in build log:
     58    /home/las_jtst/spack/lib/spack/env/gcc/g++ -c  -DNDEBUG -DZSTD_MULTITHREAD -std=c++14 -g -mavx2 -mbmi -mbmi2 -mlzcnt -DZSTD_MULTITHREAD -Wall -Wextra -Wshadow
            -Wformat-security -Wdouble-promotion -Wfloat-conversion -Wold-style-cast  -o plink2_psam.o plink2_psam.cc
     59    /home/las_jtst/spack/lib/spack/env/gcc/g++ -c  -DNDEBUG -DZSTD_MULTITHREAD -std=c++14 -g -mavx2 -mbmi -mbmi2 -mlzcnt -DZSTD_MULTITHREAD -Wall -Wextra -Wshadow
            -Wformat-security -Wdouble-promotion -Wfloat-conversion -Wold-style-cast  -o plink2_pvar.o plink2_pvar.cc
     60    /home/las_jtst/spack/lib/spack/env/gcc/g++ -c  -DNDEBUG -DZSTD_MULTITHREAD -std=c++14 -g -mavx2 -mbmi -mbmi2 -mlzcnt -DZSTD_MULTITHREAD -Wall -Wextra -Wshadow
            -Wformat-security -Wdouble-promotion -Wfloat-conversion -Wold-style-cast  -o plink2_random.o plink2_random.cc
     61    /home/las_jtst/spack/lib/spack/env/gcc/g++ -c  -DNDEBUG -DZSTD_MULTITHREAD -std=c++14 -g -mavx2 -mbmi -mbmi2 -mlzcnt -DZSTD_MULTITHREAD -Wall -Wextra -Wshadow
            -Wformat-security -Wdouble-promotion -Wfloat-conversion -Wold-style-cast  -o plink2_set.o plink2_set.cc
     62    mkdir -p bin
     63    /home/las_jtst/spack/lib/spack/env/gcc/g++  include/SFMT.o libdeflate/lib/adler32.o libdeflate/lib/aligned_malloc.o libdeflate/lib/crc32.o libdeflate/lib/defl
           ate_compress.o libdeflate/lib/deflate_decompress.o libdeflate/lib/gzip_compress.o libdeflate/lib/gzip_decompress.o libdeflate/lib/zlib_compress.o libdeflate/l
           ib/zlib_decompress.o libdeflate/lib/x86/cpu_features.o include/plink2_base.o include/plink2_bits.o include/pgenlib_misc.o include/pgenlib_read.o include/pgenl
           ib_write.o include/plink2_bgzf.o include/plink2_stats.o include/plink2_string.o include/plink2_text.o include/plink2_thread.o include/plink2_zstfile.o plink2.
           o plink2_adjust.o plink2_cmdline.o plink2_common.o plink2_compress_stream.o plink2_data.o plink2_decompress.o plink2_export.o plink2_fasta.o plink2_filter.o p
           link2_glm.o plink2_help.o plink2_import.o plink2_ld.o plink2_matrix.o plink2_matrix_calc.o plink2_misc.o plink2_psam.o plink2_pvar.o plink2_random.o plink2_se
           t.o -o bin/plink2  -llapack -lcblas -lblas -lm -lpthread -L. /home/las_jtst/spack/opt/spack/linux-rhel7-skylake_avx512/gcc-7.3.0/zlib-1.2.11-ssbqvu25vl6f4ahnm
           p4qayoyrstjrore/lib/libz.a -lzstd
  >> 64    /usr/bin/ld: cannot find -llapack
  >> 65    /usr/bin/ld: cannot find -lblas
  >> 66    collect2: error: ld returned 1 exit status
  >> 67    make: *** [plink2] Error 1

And the spec:

Input spec
--------------------------------
plink-ng%[email protected]

Concretized
--------------------------------
plink-ng@200511%[email protected] arch=linux-rhel7-skylake_avx512
    ^cblas@2015-06-06%[email protected] arch=linux-rhel7-skylake_avx512
        ^[email protected]%[email protected]~consistent_fpcsr~ilp64+pic+shared threads=none arch=linux-rhel7-skylake_avx512
    ^[email protected]%[email protected]+optimize+pic+shared arch=linux-rhel7-skylake_avx512
    ^[email protected]%[email protected]+pic arch=linux-rhel7-skylake_avx512

It seems like openblas provides the virtual dependencies but does not provide the libraries for -lblas and -llapack.

alalazo
alalazo reviewed May 26, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@alalazo alalazo left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation. Can you check if the changes posted below work for you? I didn't test them but, modulo typos, they should allow for this package to depend on lapack and blas.

@codeandkey
Copy link
Copy Markdown
Contributor Author

codeandkey commented May 26, 2020 •
edited
Loading

This works great! Just wondering, where is the libs property defined? I couldn't find a definition in the openblas recipe.

@alalazo
Copy link
Copy Markdown
Member

alalazo commented May 27, 2020

@codeandkey It's admittedly a poorly documented feature. openblas is using a default implementation to find libraries:

spack/lib/spack/spack/spec.py

Lines 751 to 799 in ac7f6b9

def _libs_default_handler(descriptor, spec, cls):
"""Default handler when looking for the 'libs' attribute.
Tries to search for ``lib{spec.name}`` recursively starting from
``spec.prefix``. If ``spec.name`` starts with ``lib``, searches for
``{spec.name}`` instead.
Parameters:
descriptor (ForwardQueryToPackage): descriptor that triggered the call
spec (Spec): spec that is being queried
cls (type(spec)): type of spec, to match the signature of the
descriptor ``__get__`` method
Returns:
LibraryList: The libraries found
Raises:
NoLibrariesError: If no libraries are found
"""
# Variable 'name' is passed to function 'find_libraries', which supports
# glob characters. For example, we have a package with a name 'abc-abc'.
# Now, we don't know if the original name of the package is 'abc_abc'
# (and it generates a library 'libabc_abc.so') or 'abc-abc' (and it
# generates a library 'libabc-abc.so'). So, we tell the function
# 'find_libraries' to give us anything that matches 'libabc?abc' and it
# gives us either 'libabc-abc.so' or 'libabc_abc.so' (or an error)
# depending on which one exists (there is a possibility, of course, to
# get something like 'libabcXabc.so, but for now we consider this
# unlikely).
name = spec.name.replace('-', '?')
# Avoid double 'lib' for packages whose names already start with lib
if not name.startswith('lib'):
name = 'lib' + name
# If '+shared' search only for shared library; if '~shared' search only for
# static library; otherwise, first search for shared and then for static.
search_shared = [True] if ('+shared' in spec) else \
([False] if ('~shared' in spec) else [True, False])
for shared in search_shared:
libs = fs.find_libraries(
name, spec.prefix, shared=shared, recursive=True)
if libs:
return libs
msg = 'Unable to recursively locate {0} libraries in {1}'
raise spack.error.NoLibrariesError(msg.format(spec.name, spec.prefix))

@alalazo alalazo merged commit fcacf8d into spack:develop May 27, 2020
@snehring snehring deleted the package/plink-ng branch April 26, 2023 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alalazo alalazo alalazo approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@codeandkey @alalazo