Skip to content

Add package for libbsd, add variant to expat for libbsd#4945

Merged
adamjstewart merged 3 commits intospack:developfrom
hartzell:fix-expat
Aug 3, 2017
Merged

Add package for libbsd, add variant to expat for libbsd#4945
adamjstewart merged 3 commits intospack:developfrom
hartzell:fix-expat

Conversation

@hartzell
Copy link
Copy Markdown
Contributor

@hartzell hartzell commented Aug 1, 2017

The recent expat release requires a high quality source of randomness.

CentOS 7 does not seem to have one, but one is available in libbsd.

This commit adds a package for libbsd and adds a variant to expat to
use it (defaults to False).

I'd appreciate feedback on the use of a variant for this (as opposed to just making it the default, perhaps) and whether the variant should default to True or False.

Fixes #4943

The recent expat release requires a high quality source of randomness.

CentOS 7 does not seem to have one, but one is available in libbsd.

This commit adds a package for libbsd and adds a variant to expat to
use it (defaults to False).
@adamjstewart
Copy link
Copy Markdown
Member

Hmm, CentOS 7 isn't exactly an old kernel, and many HPC centers will be using it in the future. I think it should be enabled by default. In general, variants that add dependencies should default to False, but in this case, the default doesn't build.

The only thing that needs to change is that this flag (and dependency) don't apply to older releases. We could add a conflicts(), or we could check the variant AND version before adding the dependency/flag.

Make the libbsd variant default to true.

Conflict if you're asking for libbsd and an older version of expat.

This means that in order to install an older version of expat you'll
need to specify `~libbsd`.
@hartzell
Copy link
Copy Markdown
Contributor Author

hartzell commented Aug 1, 2017

Here's an example of handling it with a conflict.

What I'd really like to be able to do is change the variant's default value based on the version. Is that possible?

homepage = "http://expat.sourceforge.net/"
url = "https://sourceforge.net/projects/expat/files/expat/2.2.2/expat-2.2.2.tar.bz2"

# Version 2.2.2 introduced a requirement for a high quality
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like it was actually 2.2.1 that introduced this flag.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that there were some get_random knobs in 2.2.1 (here, if I've navigated the release right) but the code that makes the build fail came in for 2.2.2 (here, ditto) and here(changelog).

@adamjstewart
Copy link
Copy Markdown
Member

What I'd really like to be able to do is change the variant's default value based on the version. Is that possible?

Not at the moment.

@hartzell
Copy link
Copy Markdown
Contributor Author

hartzell commented Aug 3, 2017

@adamjstewart -- is this good to go?

@adamjstewart
Copy link
Copy Markdown
Member

It still worries me that spack install [email protected] will fail by default. What if we remove the conflicts() and instead only add the libbsd dependency and --with-libbsd flag [email protected]:+libbsd? The downside of this is that [email protected]+libbsd and [email protected]~libbsd are the same thing, but I think it's worth it if [email protected] builds by default.

Get rid of the conflicts and use better constraints/tests in the
`depends_on` and the `configure_args` bits.
@hartzell
Copy link
Copy Markdown
Contributor Author

hartzell commented Aug 3, 2017

Halllmark needs to market a line of greeting cards with the theme "Thanks for making me a better contributor!" I'm failing to think what the graphics would look like, but....

I think that the most recent commit does what you're suggesting:

[hartzelg@lb097hmdev spack-previous-expat]$ spack spec [email protected]
[...]
Concretized
--------------------------------
[email protected]%[email protected]+libbsd arch=linux-centos7-x86_64
    ^[email protected]%[email protected] arch=linux-centos7-x86_64

[hartzelg@lb097hmdev spack-previous-expat]$ spack spec [email protected]+libbsd
[...]
Concretized
--------------------------------
[email protected]%[email protected]+libbsd arch=linux-centos7-x86_64
    ^[email protected]%[email protected] arch=linux-centos7-x86_64

[hartzelg@lb097hmdev spack-previous-expat]$ spack spec [email protected]~libbsd
[...]
Concretized
--------------------------------
[email protected]%[email protected]~libbsd arch=linux-centos7-x86_64

[hartzelg@lb097hmdev spack-previous-expat]$ spack spec [email protected]
[...]
Concretized
--------------------------------
[email protected]%[email protected]+libbsd arch=linux-centos7-x86_64
    ^[email protected]%[email protected] arch=linux-centos7-x86_64

[hartzelg@lb097hmdev spack-previous-expat]$ spack spec [email protected]~libbsd
[...]
Concretized
--------------------------------
[email protected]%[email protected]~libbsd arch=linux-centos7-x86_64

[hartzelg@lb097hmdev spack-previous-expat]$ spack spec [email protected]
[...]
Concretized
--------------------------------
[email protected]%[email protected]+libbsd arch=linux-centos7-x86_64
[hartzelg@lb097hmdev spack-previous-expat]$ spack install [email protected]
==> Installing expat
[...]
==> Successfully installed expat
  Fetch: 0.01s.  Build: 8.39s.  Total: 8.40s.
[+] /home/hartzelg/tmp/spack-previous-expat/opt/spack/linux-centos7-x86_64/gcc-5.4.0/expat-2.2.0-7ovzkb5szpllhxu45sqy7i7uwnlf4eho
[hartzelg@lb097hmdev spack-previous-expat]$ spack install [email protected]
==> Installing libbsd
[...]
==> Successfully installed libbsd
  Fetch: 0.02s.  Build: 7.24s.  Total: 7.26s.
[+] /home/hartzelg/tmp/spack-previous-expat/opt/spack/linux-centos7-x86_64/gcc-5.4.0/libbsd-0.8.6-rex3ft7n34x2hcoeye4na6ebywvxpvyi
==> Installing expat
[...]
==> Successfully installed expat
  Fetch: 0.02s.  Build: 6.65s.  Total: 6.67s.
[+] /home/hartzelg/tmp/spack-previous-expat/opt/spack/linux-centos7-x86_64/gcc-5.4.0/expat-2.2.2-tgzd26hp7pv6w4434blyzyhiibnomnpk

@jgalarowicz
Copy link
Copy Markdown
Contributor

I just ran into this issue too. Looking forward to the changes! Thanks! Jim G

@adamjstewart adamjstewart merged commit f24398c into spack:develop Aug 3, 2017
@jgalarowicz
Copy link
Copy Markdown
Contributor

jgalarowicz commented Aug 3, 2017

@hartzell @adamjstewart I'm seeing a failure in the libbsd build. expat builds ok now.

[jeg@localhost spack]$ spack install libbsd
==> Installing libbsd
==> Using cached archive: /home/jeg/spack_updates/spack/var/spack/cache/libbsd/libbsd-0.8.6.tar.xz
==> Already staged libbsd-0.8.6-pinxuc2xn7dgyatilr27cr5wkt3qnxcn in /home/jeg/spack_updates/spack/var/spack/stage/libbsd-0.8.6-pinxuc2xn7dgyatilr27cr5wkt3qnxcn
==> No patches needed for libbsd
==> Building libbsd [AutotoolsPackage]
==> Executing phase : 'autoreconf'
==> Executing phase : 'configure'
==> Error: ProcessError: Command exited with status 1:
    '/home/jeg/spack_updates/spack/var/spack/stage/libbsd-0.8.6-pinxuc2xn7dgyatilr27cr5wkt3qnxcn/libbsd-0.8.6/configure' '--prefix=/home/jeg/spack_updates/spack/opt/spack/linux-fedora19-x86_64/gcc-4.8.3/libbsd-0.8.6-pinxuc2xn7dgyatilr27cr5wkt3qnxcn'
/home/jeg/spack_updates/spack/lib/spack/spack/build_systems/autotools.py:268, in configure:
     260      def configure(self, spec, prefix):
     261          """Runs configure with the arguments specified in
     262          :py:meth:`~.AutotoolsPackage.configure_args`
     263          and an appropriately set prefix.
     264          """
     265          options = ['--prefix={0}'.format(prefix)] + self.configure_args()
     266  
     267          with working_dir(self.build_directory, create=True):
  >> 268              inspect.getmodule(self).configure(*options)

@adamjstewart
Copy link
Copy Markdown
Member

@jgalarowicz All that tells me is that configure crashed for some reason. Can you post the spack-build.out and config.log?

@adamjstewart
Copy link
Copy Markdown
Member

@hartzell I'm also seeing build errors on macOS:

In file included from /Users/Adam/spack/var/spack/stage/libbsd-0.8.6-y6pciy4czpfghl622mv2wu26achcbtxx/libbsd-0.8.6/include/bsd/stdlib.h:37:
/Users/Adam/spack/var/spack/stage/libbsd-0.8.6-y6pciy4czpfghl622mv2wu26achcbtxx/libbsd-0.8.6/include/bsd/libutil.h:42:10: fatal error: 'features.h' file not found
#include <features.h>
         ^

@jgalarowicz
Copy link
Copy Markdown
Contributor

jgalarowicz commented Aug 3, 2017

SPACK-BUILD.out:
See build log for details:
/tmp/jeg/spack-stage/spack-stage-KH0Olh/libbsd-0.8.6/spack-build.out
[jeg@localhost spack]$ cat /tmp/jeg/spack-stage/spack-stage-KH0Olh/libbsd-0.8.6/spack-build.out

==> '/home/jeg/spack_updates/spack/var/spack/stage/libbsd-0.8.6-pinxuc2xn7dgyatilr27cr5wkt3qnxcn/libbsd-0.8.6/configure' '--prefix=/home/jeg/spack_updates/spack/opt/spack/linux-fedora19-x86_64/gcc-4.8.3/libbsd-0.8.6-pinxuc2xn7dgyatilr27cr5wkt3qnxcn'
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
configure: error: source directory already configured; run "make distclean" there first
[jeg@localhost spack]$ 

CONFIG.LOG
[jeg@localhost libbsd-0.8.6]$ cat config.log

This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by libbsd configure 0.8.6, which was
generated by GNU Autoconf 2.69.  Invocation command line was

  $ /home/jeg/spack_updates/spack/var/spack/stage/libbsd-0.8.6-pinxuc2xn7dgyatilr27cr5wkt3qnxcn/libbsd-0.8.6/configure --prefix=/home/jeg/spack_updates/spack/opt/spack/linux-fedora19-x86_64/gcc-4.8.3/libbsd-0.8.6-pinxuc2xn7dgyatilr27cr5wkt3qnxcn

## --------- ##
## Platform. ##
## --------- ##

hostname = localhost
uname -m = x86_64
uname -r = 3.14.27-100.fc19.x86_64
uname -s = Linux
uname -v = #1 SMP Wed Dec 17 19:36:34 UTC 2014

/usr/bin/uname -p = x86_64
/bin/uname -X     = unknown

/bin/arch              = x86_64
/usr/bin/arch -k       = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo      = unknown
/bin/machine           = unknown
/usr/bin/oslevel       = unknown
/bin/universe          = unknown

PATH: /home/jeg/spack_updates/spack/lib/spack/env
PATH: /home/jeg/spack_updates/spack/lib/spack/env/case-insensitive
PATH: /home/jeg/spack_updates/spack/lib/spack/env/gcc
PATH: /home/jeg/spack_updates/spack/bin
PATH: /usr/local/krb5/bin
PATH: /usr/kerberos/bin
PATH: /opt/cisco/anyconnect/bin
PATH: /usr/local/krb5/bin
PATH: /usr/kerberos/bin
PATH: /opt/cisco/anyconnect/bin
PATH: /usr/lib64/qt-3.3/bin
PATH: /usr/kerberos/sbin
PATH: /usr/kerberos/bin
PATH: /usr/local/bin
PATH: /usr/bin
PATH: /bin
PATH: /usr/local/sbin
PATH: /usr/sbin
PATH: /usr/openwin/bin
PATH: /home/jeg/.local/bin
PATH: /home/jeg/bin


## ----------- ##
## Core tests. ##
## ----------- ##

configure:2460: checking for a BSD-compatible install
configure:2528: result: /usr/bin/install -c
configure:2539: checking whether build environment is sane
configure:2594: result: yes
configure:2745: checking for a thread-safe mkdir -p
configure:2784: result: /usr/bin/mkdir -p
configure:2791: checking for gawk
configure:2807: found /usr/bin/gawk
configure:2818: result: gawk
configure:2829: checking whether make sets $(MAKE)
configure:2851: result: yes
configure:2880: checking whether make supports nested variables
configure:2897: result: yes
configure:2914: error: source directory already configured; run "make distclean" there first

## ---------------- ##
## Cache variables. ##
## ---------------- ##

ac_cv_env_CC_set=set
ac_cv_env_CC_value=/home/jeg/spack_updates/spack/lib/spack/env/gcc/gcc
ac_cv_env_CFLAGS_set=set
ac_cv_env_CFLAGS_value=
ac_cv_env_CPPFLAGS_set=set
ac_cv_env_CPPFLAGS_value=
ac_cv_env_CPP_set=
ac_cv_env_CPP_value=
ac_cv_env_LDFLAGS_set=set
ac_cv_env_LDFLAGS_value=
ac_cv_env_LIBS_set=
ac_cv_env_LIBS_value=
ac_cv_env_LT_SYS_LIBRARY_PATH_set=
ac_cv_env_LT_SYS_LIBRARY_PATH_value=
ac_cv_env_build_alias_set=
ac_cv_env_build_alias_value=
ac_cv_env_host_alias_set=
ac_cv_env_host_alias_value=
ac_cv_env_target_alias_set=
ac_cv_env_target_alias_value=
ac_cv_path_install='/usr/bin/install -c'
ac_cv_path_mkdir=/usr/bin/mkdir
ac_cv_prog_AWK=gawk
ac_cv_prog_make_make_set=yes
am_cv_make_support_nested_variables=yes

## ----------------- ##
## Output variables. ##
## ----------------- ##

ACLOCAL=''
AMDEPBACKSLASH=''
AMDEP_FALSE=''
AMDEP_TRUE=''
AMTAR=''
AM_BACKSLASH='\'
AM_DEFAULT_V='$(AM_DEFAULT_VERBOSITY)'
AM_DEFAULT_VERBOSITY='1'
AM_V='$(V)'
AR=''
AUTOCONF=''
AUTOHEADER=''
AUTOMAKE=''
AWK='gawk'
BUILD_LIBBSD_CTOR_FALSE=''
BUILD_LIBBSD_CTOR_TRUE=''
CC='/home/jeg/spack_updates/spack/lib/spack/env/gcc/gcc'
CCDEPMODE=''
CFLAGS=''
CLOCK_GETTIME_LIBS=''
CPP=''
CPPFLAGS=''
CYGPATH_W=''
DEFS=''
DEPDIR=''
DLLTOOL=''
DSYMUTIL=''
DUMPBIN=''
ECHO_C=''
ECHO_N='-n'
ECHO_T=''
EGREP=''
EXEEXT=''
FGREP=''
GREP=''
HAVE_GETENTROPY_FALSE=''
HAVE_GETENTROPY_TRUE=''
HAVE_LIBTESTU01_FALSE=''
HAVE_LIBTESTU01_TRUE=''
INSTALL_DATA='${INSTALL} -m 644'
INSTALL_PROGRAM='${INSTALL}'
INSTALL_SCRIPT='${INSTALL}'
INSTALL_STRIP_PROGRAM='$(install_sh) -c -s'
LD=''
LDFLAGS=''
LIBBSD_ABI=''
LIBOBJS=''
LIBS=''
LIBTOOL=''
LIPO=''
LN_S=''
LTLIBOBJS=''
LT_SYS_LIBRARY_PATH=''
MAKEINFO=''
MANIFEST_TOOL=''
MKDIR_P='/usr/bin/mkdir -p'
NM=''
NMEDIT=''
OBJDUMP=''
OBJEXT=''
OTOOL64=''
OTOOL=''
PACKAGE=''
PACKAGE_BUGREPORT='[email protected]'
PACKAGE_NAME='libbsd'
PACKAGE_STRING='libbsd 0.8.6'
PACKAGE_TARNAME='libbsd'
PACKAGE_URL=''
PACKAGE_VERSION='0.8.6'
PATH_SEPARATOR=':'
RANLIB=''
SED=''
SET_MAKE=''
SHELL='/bin/sh'
STRIP=''
TESTU01_LIBS=''
VERSION=''
ac_ct_AR=''
ac_ct_CC=''
ac_ct_DUMPBIN=''
am__EXEEXT_FALSE=''
am__EXEEXT_TRUE=''
am__fastdepCC_FALSE=''
am__fastdepCC_TRUE=''
am__include=''
am__isrc=' -I$(srcdir)'
am__leading_dot='.'
am__nodep=''
am__quote=''
am__tar=''
am__untar=''
bindir='${exec_prefix}/bin'
build=''
build_alias=''
build_cpu=''
build_os=''
build_vendor=''
datadir='${datarootdir}'
datarootdir='${prefix}/share'
docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
dvidir='${docdir}'
exec_prefix='NONE'
host=''
host_alias=''
host_cpu=''
host_os=''
host_vendor=''
htmldir='${docdir}'
includedir='${prefix}/include'
infodir='${datarootdir}/info'
install_sh='${SHELL} /tmp/jeg/spack-stage/spack-stage-KH0Olh/libbsd-0.8.6/build-aux/install-sh'
libdir='${exec_prefix}/lib'
libexecdir='${exec_prefix}/libexec'
localedir='${datarootdir}/locale'
localstatedir='${prefix}/var'
mandir='${datarootdir}/man'
mkdir_p=''
oldincludedir='/usr/include'
pdfdir='${docdir}'
prefix='/home/jeg/spack_updates/spack/opt/spack/linux-fedora19-x86_64/gcc-4.8.3/libbsd-0.8.6-pinxuc2xn7dgyatilr27cr5wkt3qnxcn'
program_transform_name='s,x,x,'
psdir='${docdir}'
runstatedir='${localstatedir}/run'
sbindir='${exec_prefix}/sbin'
sharedstatedir='${prefix}/com'
sysconfdir='${prefix}/etc'
target_alias=''

## ----------- ##
## confdefs.h. ##
## ----------- ##

/* confdefs.h */
#define PACKAGE_NAME "libbsd"
#define PACKAGE_TARNAME "libbsd"
#define PACKAGE_VERSION "0.8.6"
#define PACKAGE_STRING "libbsd 0.8.6"
#define PACKAGE_BUGREPORT "[email protected]"
#define PACKAGE_URL ""

configure: exit 1

[jeg@localhost libbsd-0.8.6]$

@adamjstewart
Copy link
Copy Markdown
Member

@jgalarowicz Run spack purge and then re-run spack install libbsd.

@jgalarowicz
Copy link
Copy Markdown
Contributor

jgalarowicz commented Aug 3, 2017

@adamjstewart @hartzell
Now I get this in spack-build.out for the compile of libbsd:

make[2]: Entering directory `/tmp/jeg/spack-stage/spack-stage-ATCin7/libbsd-0.8.6/man'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/tmp/jeg/spack-stage/spack-stage-ATCin7/libbsd-0.8.6/man'
Making all in src
make[2]: Entering directory `/tmp/jeg/spack-stage/spack-stage-ATCin7/libbsd-0.8.6/src'
  CC       setproctitle_ctor.o
  CC       arc4random.lo
  CC       arc4random_uniform.lo
  CC       bsd_getopt.lo
  CC       closefrom.lo
  CC       dehumanize_number.lo
  CC       expand_number.lo
  CC       err.lo
In file included from /usr/include/features.h:375:0,
                 from /usr/include/unistd.h:25,
                 from /tmp/jeg/spack-stage/spack-stage-ATCin7/libbsd-0.8.6/include/bsd/unistd.h:29,
                 from setproctitle_ctor.c:27:
/tmp/jeg/spack-stage/spack-stage-ATCin7/libbsd-0.8.6/include/bsd/sys/cdefs.h:39:23: error: missing binary operator before token "("
 #if __has_include_next(<sys/cdefs.h>)
                       ^
In file included from /tmp/jeg/spack-stage/spack-stage-ATCin7/libbsd-0.8.6/include/bsd/unistd.h:29:0,
                 from setproctitle_ctor.c:27:
/usr/include/unistd.h: In function ‘access’:
/usr/include/unistd.h:287:52: error: expected declaration specifiers before ‘__THROW’
 extern int access (const char *__name, int __type) __THROW __nonnull ((1));
                                                    ^
/usr/include/unistd.h:293:6: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__THROW’
      __THROW __nonnull ((1));
      ^
/usr/include/unistd.h:297:6: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__THROW’
      __THROW __nonnull ((1));
      ^
/usr/include/unistd.h:305:6: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__THROW’
      __THROW __nonnull ((2)) __wur;
      ^
/usr/include/unistd.h:334:65: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__THROW’
 extern __off_t lseek (int __fd, __off_t __offset, int __whence) __THROW;
                                                                 ^
/usr/include/unistd.h:346:6: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__THROW’
      __THROW;

@hartzell
Copy link
Copy Markdown
Contributor Author

hartzell commented Aug 4, 2017

Nuts.

@jgalarowicz -- Can you build [email protected]~libbsd on your Mac? I think that a current Mac might already have an adequate source of randomness that's provided by the OS and "someone" (george...) should conditional-ize the dependency on the platform (can we do that?).

But, Mac OS looks like it's not simple. See here, and here and here.

@adamjstewart
Copy link
Copy Markdown
Member

I'm able to build [email protected]~libbsd on macOS Sierra (in fact, that's what got us into this mess). But libbsd definitely doesn't build on macOS.

@hartzell
Copy link
Copy Markdown
Contributor Author

hartzell commented Aug 4, 2017

How about if we make libbsd conflict with mac os and further conditionalize the expat dependency on the platform (assuming there are sufficient hooks to make that happen)?

@adamjstewart
Copy link
Copy Markdown
Member

That works for me but not for @jgalarowicz who is on Fedora 19.

@hartzell
Copy link
Copy Markdown
Contributor Author

hartzell commented Aug 4, 2017

@jgalarowicz -- What compiler are you using?

The build is dying at the last line of this bit of libbsd's cdefs.h:

#ifndef __has_include
#define __has_include 1
#endif
#ifndef __has_include_next
#define __has_include_next 1
#endif

#ifdef LIBBSD_OVERLAY
/*
 * Some libc implementations do not have a <sys/cdefs.h>, in particular
 * musl, try to handle this gracefully.
 */
#if __has_include_next(<sys/cdefs.h>)

__has_include _next is a clang extension; the preceding bits of cdef.h replace it with a 1 for compilers that do not define the extension.

A bit of poking around with gcc -E proves to me that lines 27 -> 46 of libbsd's cdefs.h disappear when processed by my spack-build [email protected] on CentOS 7.

[hartzelg@lb097hmdev tmp]$ cat blah.c
a = 1;
#ifndef __has_include
#define __has_include 1
#endif
#ifndef __has_include_next
#define __has_include_next 1
#endif

#ifdef LIBBSD_OVERLAY
/*
 * Some libc implementations do not have a <sys/cdefs.h>, in particular
 * musl, try to handle this gracefully.
 */
#if __has_include_next(<sys/cdefs.h>)
#include_next <sys/cdefs.h>
#endif
#else
#if __has_include(<sys/cdefs.h>)
#include <sys/cdefs.h>
#endif
#endif
b=2;

[hartzelg@lb097hmdev tmp]$ gcc -E  blah.c
# 1 "blah.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 1 "<command-line>" 2
# 1 "blah.c"
a = 1;
# 19 "blah.c"
# 1 "/usr/include/sys/cdefs.h" 1 3 4
# 24 "/usr/include/sys/cdefs.h" 3 4
# 1 "/usr/include/features.h" 1 3 4
# 399 "/usr/include/features.h" 3 4
# 1 "/usr/include/gnu/stubs.h" 1 3 4
# 10 "/usr/include/gnu/stubs.h" 3 4
# 1 "/usr/include/gnu/stubs-64.h" 1 3 4
# 11 "/usr/include/gnu/stubs.h" 2 3 4
# 400 "/usr/include/features.h" 2 3 4
# 25 "/usr/include/sys/cdefs.h" 2 3 4
# 392 "/usr/include/sys/cdefs.h" 3 4
# 1 "/usr/include/bits/wordsize.h" 1 3 4
# 393 "/usr/include/sys/cdefs.h" 2 3 4
# 20 "blah.c" 2


b=2;

@hartzell
Copy link
Copy Markdown
Contributor Author

hartzell commented Aug 4, 2017

@jgalarowicz
Copy link
Copy Markdown
Contributor

@hartzell Yes, 4.8.3. I'm planning on upgrading soon to Fedora 26.

@jgalarowicz
Copy link
Copy Markdown
Contributor

@hartzell @adamjstewart I can build expat 2.2.0 to get around this temporarily, if this is an older compiler issue.

@hartzell
Copy link
Copy Markdown
Contributor Author

hartzell commented Aug 4, 2017

I didn't think carefully about my gcc -E output above. As the comments show it is processing bits of those includes, the entire chunk does not just disappear.

@hartzell
Copy link
Copy Markdown
Contributor Author

hartzell commented Aug 4, 2017

[edited to fix the fenced code block]

It turns out that I can not build it on CentOS 7 with the system's [email protected].

Seems to have the same problem:

  CC       reallocarray.lo
In file included from /usr/include/features.h:375:0,
                 from /usr/include/unistd.h:25,
                 from /tmp/hartzelg/spack-stage/spack-stage-GBnaQd/libbsd-0.8.6/include/bsd/unistd.h:29,
                 from setproctitle_ctor.c:27:
/tmp/hartzelg/spack-stage/spack-stage-GBnaQd/libbsd-0.8.6/include/bsd/sys/cdefs.h:39:23: error: missing binary operator before token "("
 #if __has_include_next(<sys/cdefs.h>)
                       ^
In file included from /tmp/hartzelg/spack-stage/spack-stage-GBnaQd/libbsd-0.8.6/include/bsd/unistd.h:29:0,
                 from setproctitle_ctor.c:27:
/usr/include/unistd.h: In function ‘access’:
/usr/include/unistd.h:287:52: error: expected declaration specifiers before ‘__THROW’

@hartzell
Copy link
Copy Markdown
Contributor Author

hartzell commented Aug 4, 2017

Sadly, the libbsd Bugzilla has Zarro Boogs for 'missing binary operator`....

@hartzell
Copy link
Copy Markdown
Contributor Author

hartzell commented Aug 4, 2017

TL;DR

This seems to let it build for me:

--- include/bsd/sys/cdefs.h.orig        2017-08-04 16:34:56.404995800 -0700
+++ include/bsd/sys/cdefs.h     2017-08-04 16:35:19.345043883 -0700
@@ -25,10 +25,10 @@
  */

 #ifndef __has_include
-#define __has_include 1
+#define __has_include(X) 1
 #endif
 #ifndef __has_include_next
-#define __has_include_next 1
+#define __has_include_next(X) 1
 #endif

 #ifdef LIBBSD_OVERLAY

I'll add something to the package, see if it builds for me, and PR it for discussion.

Details

In the spirit of showing my work (aka showing where I went off the rails...), here's how I ended up there, starting by isolating the gcc invocation that was failing and removing everything I could.

Given a blah.c that contains:

#include "blah.h"

and a blah.h that contains this trimmed down copy of libbsd's include/bsd/sys/cdefs.h:

#ifndef __has_include
#define __has_include 1
#endif
#ifndef __has_include_next
#define __has_include_next 1
#endif

#ifdef LIBBSD_OVERLAY
/*
 * Some libc implementations do not have a <sys/cdefs.h>, in particular
 * musl, try to handle this gracefully.
 */
#if __has_include_next(<sys/cdefs.h>)
#include_next <sys/cdefs.h>
#endif
#else
#if __has_include(<sys/cdefs.h>)
#include <sys/cdefs.h>
#endif
#endif

The running this command (I started with the failing make command and trimmed until I could trim no more) with CentOS's [email protected] from the src/ subdir of a staged libbsd demonstrates the same symptom:

[hartzelg@lb097hmdev src]$ gcc -E -isystem ../include/bsd/ -DLIBBSD_OVERLAY -c blah.c
# 1 "blah.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 1 "<command-line>" 2
# 1 "blah.c"
# 1 "blah.h" 1
In file included from blah.c:1:0:
blah.h:13:23: error: missing binary operator before token "("
 #if __has_include_next(<sys/cdefs.h>)
                       ^
# 1 "blah.c" 2

If I change blah.h to this (to see what value the __has_include_next has):

#ifndef __has_include
#define __has_include 1
#endif
#ifndef __has_include_next
#define __has_include_next 1
#endif

__has_include_next

I get this output:

[hartzelg@lb097hmdev src]$ gcc -E -isystem ../include/bsd/ -DLIBBSD_OVERLAY -c blah.c
# 1 "blah.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 1 "<command-line>" 2
# 1 "blah.c"
# 1 "blah.h" 1







1
# 1 "blah.c" 2

which suggests that the compiler is tripping over a construct that looks like #if 1(<sys/cdefs.h>). Sure enough, replacing the contents of blah.h with this:

#if 1(<sys/cdefs.h>)
a=1;
#endif

recapitulates the error:

[hartzelg@lb097hmdev src]$ gcc -E -isystem ../include/bsd/ -DLIBBSD_OVERLAY -c blah.c
# 1 "blah.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 1 "<command-line>" 2
# 1 "blah.c"
# 1 "blah.h" 1
In file included from blah.c:1:0:
blah.h:1:6: error: missing binary operator before token "("
 #if 1(<sys/cdefs.h>)
      ^
# 1 "blah.c" 2

hartzell pushed a commit to hartzell/spack that referenced this pull request Aug 4, 2017
See the discussion in spack#4945 (after the merge) for additional
background.

libbsd builds with [email protected] on CentOS 7, but not with the system's
[email protected].  Others have reported problems with [email protected] on Fedora 19.

The problem boils down to the lack of support for the clang extension
`__has_include_next`.  The immediate symptom seems to be the
pre-processor using defining macro like this

```
```

then then tripping over an expansion of it like this:

```
blah.h:13:23: error: missing binary operator before token "("
```

This patch changes the macro definition to:

```
```

which swallows the arguments with which the macro is invoked.

The end result is that libbsd builds for me on CentOS 7 using the
system compiler.
adamjstewart pushed a commit that referenced this pull request Aug 5, 2017
* Fix cdefs macro to be compatible with gcc 4.8.x

See the discussion in #4945 (after the merge) for additional
background.

libbsd builds with [email protected] on CentOS 7, but not with the system's
[email protected].  Others have reported problems with [email protected] on Fedora 19.

The problem boils down to the lack of support for the clang extension
`__has_include_next`.  The immediate symptom seems to be the
pre-processor using defining macro like this

```
```

then then tripping over an expansion of it like this:

```
blah.h:13:23: error: missing binary operator before token "("
```

This patch changes the macro definition to:

```
```

which swallows the arguments with which the macro is invoked.

The end result is that libbsd builds for me on CentOS 7 using the
system compiler.

* Apply this patch for any compiler version before 5

This includes subversions of 4, like 4.8.5.
citibeth pushed a commit to citibeth/spack that referenced this pull request May 15, 2018
…rsion of libexpat did. Also update for Expat's move to GitHub.
citibeth pushed a commit to citibeth/spack that referenced this pull request May 15, 2018
…rsion of libexpat did. Also update for Expat's move to GitHub.
adamjstewart pushed a commit that referenced this pull request May 18, 2018
* PR #4945 did not make this work on SuSE 11.  Adding the latest version of libexpat did.  Also update for Expat's move to GitHub.

* Update package.py

* Update package.py

Move to url_for_version()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants