Skip to content

Add pic variant to cflags in boost.#9750

Merged
adamjstewart merged 2 commits intospack:developfrom
exactlab:exactlab_boost_add_pic_flag
Nov 13, 2018
Merged

Add pic variant to cflags in boost.#9750
adamjstewart merged 2 commits intospack:developfrom
exactlab:exactlab_boost_add_pic_flag

Conversation

@bebosudo
Copy link
Copy Markdown
Contributor

@bebosudo bebosudo commented Nov 7, 2018

While installing a package depending on the boost library, I've got the following error:

     24046    /usr/bin/ld: /opt/cluster/spack/opt/spack/linux-centos7-x86_64/gcc-8.2.0/boost-1.68.0-xkyf5icz7yn62yjvduwrs3mfru4merj4/lib/libboost_serialization-mt.a(archive_exception.o): re
              location R_X86_64_32S against symbol `_ZTVN5boost7archive17archive_exceptionE' can not be used when making a shared object; recompile with -fPIC
     24047    /usr/bin/ld: /opt/cluster/spack/opt/spack/linux-centos7-x86_64/gcc-8.2.0/boost-1.68.0-xkyf5icz7yn62yjvduwrs3mfru4merj4/lib/libboost_serialization-mt.a(basic_archive.o): reloca
              tion R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
     24048    /usr/bin/ld: /opt/cluster/spack/opt/spack/linux-centos7-x86_64/gcc-8.2.0/boost-1.68.0-xkyf5icz7yn62yjvduwrs3mfru4merj4/lib/libboost_filesystem-mt.a(operations.o): relocation R
              _X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
     24049    /usr/bin/ld: /opt/cluster/spack/opt/spack/linux-centos7-x86_64/gcc-8.2.0/boost-1.68.0-xkyf5icz7yn62yjvduwrs3mfru4merj4/lib/libboost_filesystem-mt.a(path.o): relocation R_X86_6
              4_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
     24050    /usr/bin/ld: /opt/cluster/spack/opt/spack/linux-centos7-x86_64/gcc-8.2.0/boost-1.68.0-xkyf5icz7yn62yjvduwrs3mfru4merj4/lib/libboost_date_time-mt.a(greg_month.o): relocation R_
              X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
     24051    /usr/bin/ld: /opt/cluster/spack/opt/spack/linux-centos7-x86_64/gcc-8.2.0/boost-1.68.0-xkyf5icz7yn62yjvduwrs3mfru4merj4/lib/libboost_system-mt.a(error_code.o): relocation R_X86
              _64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
  >> 24052    /usr/bin/ld: final link failed: Nonrepresentable section on output
  >> 24053    collect2: error: ld returned 1 exit status
  >> 24054    make[2]: *** [Pyext/ecflow.so] Error 1
     24055    make[2]: Leaving directory `/tmp/root/spack-stage/spack-stage-8usvfauy/ecFlow-4.11.1-Source/spack-build'
  >> 24056    make[1]: *** [Pyext/CMakeFiles/ecflow.dir/all] Error 2
     24057    make[1]: *** Waiting for unfinished jobs....

Therefore I've added the pic variant, as in #9605.

@mwkrentel
Copy link
Copy Markdown
Member

Your build faild because you're trying to link with the boost static
.a archives instead of the shared .so libraries. Either that's what
your build scripts are asking for, or else maybe your boost directory
didn't install the shared libs.

Is that what you're trying to do? Statically link the boost libs
(with fPIC) into another shared library?

That's a valid use case, but a bit unusual. It would be more common
to link with the shared boost libs. So, I suggest the default should
be False.

Either way, the description is misleading and should instead say that
the variant adds fPIC to the static libraries. Shared libraries are
always built with fPIC and the spack boost recipe already builds both
static and shared libraries.

@bebosudo
Copy link
Copy Markdown
Contributor Author

bebosudo commented Nov 9, 2018

Hi Mark, you were right. For some reasons ecFlow (https://confluence.ecmwf.int/display/ECFLOW/, a library I'm compiling) builds its libraries statically:

# cd /../spack/opt/spack/linux-centos7-x86_64/gcc-8.2.0/ecflow-4.11.1-*
# ll lib/python2.7/site-packages/ecflow/ecflow.so
-rwxr-xr-x 1 root root 159M Nov  7 18:12 lib/python2.7/site-packages/ecflow/ecflow.so*

# nm lib/python2.7/site-packages/ecflow/ecflow.so |grep -i boost
000000000082c740 W boost_asio_detail_posix_thread_function
0000000000f97e40 d DW.ref._ZTIN5boost10filesystem16filesystem_errorE
...

If I find a way to dinamically link boost in ecFlow I think it would be the best solution.

I'm fine setting it to false, even though leaving it to True shouldn't harm (right?), and we could avoid re-compiling boost if a library like ecFlow (which I'm going to submit soon) depends on boost and wants the pic variant: when setting the pic variant to False, a user that compiles boost for other packages would compile the non-pic version, and when packages like ecFlow are compiled, the pic version is required, and a new boost should be built.

Your build faild because you're trying to link with the boost static

Looking at https://travis-ci.org/spack/spack/jobs/451977005, it seems more of a connection problem to codecov (and I've already seen some of these in other builds):

==> Uploading
    .url https://codecov.io
    .query pr=9750&service=travis&package=py2.0.15&yaml=.codecov.yml&job=451977005&flags=unitlinux&build=24937.3&branch=develop&commit=445034c6741ed51d88940aa19b165390454b4796&slug=spack%2Fspack
    Pinging Codecov...
Error: ('Connection aborted.', BadStatusLine("''",))

Either way, the description is misleading and should instead say that the variant adds fPIC to the static libraries.

I copied it from the variant description of many packages:
https://github.com/spack/spack/search?q=Produce+position-independent+code+(for+shared+libs)

I can open another issue substituting all those description (should be an easy job with a sed line), but I think we should stick to the "wrong", but commonly used, variant description inside this issue.

@mwkrentel
Copy link
Copy Markdown
Member

PIC can slightly slow down the code. Not a lot, but there are plenty
of people who want to link boost into a full executable (not another
library) and don't want this. So, I suggest default False.

For the description, I suggest "Build static libraries with
position-independent code."

The problem with saying "for shared libs" is that it suggests that
this option is necessary for building shared libs. Shared libs always
need PIC, so the risk is that someone would read this as, "I want the
boost shared libs, so I better use +pic or else it won't build shared
libraries."

That's why I consider it misleading.

@bebosudo
Copy link
Copy Markdown
Contributor Author

bebosudo commented Nov 9, 2018

PIC can slightly slow down the code. Not a lot, but there are plenty
of people who want to link boost into a full executable (not another
library) and don't want this. So, I suggest default False.

Didn't know this. I'm not a great compiler expert. Thanks!

I'll change the description text according with your suggestions.

@adamjstewart adamjstewart merged commit 7938ff9 into spack:develop Nov 13, 2018
@bebosudo bebosudo deleted the exactlab_boost_add_pic_flag branch November 14, 2018 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants