Skip to content

Conversation

@fanquake
Copy link
Member

@fanquake fanquake commented Nov 19, 2021

Currently, ./build.sh doesn't seem to respect CXX and CXXFLAGS. This
becomes more obvious when you build depends with CC and CXX set,
without another compiler installed. i.e when using g++-8 on Ubuntu
Bionic, but not having g++ (GGC-7) installed. The native_b2 build will
fail with:

make -C depends/ -j9 CC=gcc-8 CXX=g++-8
...
/bitcoin/depends/sources/boost_1_71_0.tar.bz2: OK
Preprocessing native_b2...
Configuring native_b2...
Building native_b2...
B2_TOOLSET is gcc, but the 'gcc' command cannot be executed.
Make sure 'gcc' is in PATH, or use a different toolset.

Instead, don't set CXX or CXXFLAGS before calling build.sh.
In this case, we either get the default CXX (g++), or, we'll get the CXX set by the user. i.e:

make -C depends/ -j9 CC=gcc-8 CXX=g++-8
...
/bitcoin/depends/sources/boost_1_71_0.tar.bz2: OK
Preprocessing native_b2...
Configuring native_b2...
Building native_b2...
g++-8 --version
g++-8 (Ubuntu 8.4.0-1ubuntu1~18.04) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

g++-8 -x c++ -std=c++11 -O2 -s .... <snip>

@hebasto
Copy link
Member

hebasto commented Nov 19, 2021

Concept ACK.

@maflcko
Copy link
Member

maflcko commented Nov 19, 2021

Does this allow to simplify the msan ci with boost_cxxflags?

@dongcarl
Copy link
Contributor

Actually, boost 1.71.0 (the version we use)'s ./build.sh does respect CXX env vars, see:

https://github.com/boostorg/build/blob/boost-1.71.0/src/engine/build.sh#L157

However, this was changed in boostorg/build@7497f61, which was first included in boost 1.76.0

The reason why you're seeing the current breakage in the first case is because our well-known var propagation in depends is limited to non-native packages, native packages do not yet get them: #18820 (comment).

However, I have not figured out why it works in the second case. My reading of boost's build.sh as of 1.71.0 seems to indicate that the --cxx and --cxxflags would be ignored: https://github.com/boostorg/build/blob/04efaac148671479b514db882b1c520dcaafe89e/src/engine/build.sh#L149-L155

So perhaps there's a CXX env var that's set somewhere in depends that we don't know about? Curious.

@DrahtBot
Copy link
Contributor

Guix builds

File commit fe03f7a
(master)
commit 32f340ba7ad12008cb5bbde535e5eec9dd128a8f
(master and this pull)
SHA256SUMS.part 044bb745547def7a... a540a026627d762c...
*-aarch64-linux-gnu-debug.tar.gz 09bbff302ad3331e... cfa7d315f8340a9a...
*-aarch64-linux-gnu.tar.gz 430b9f97f423b06a... 260f643ece176933...
*-arm-linux-gnueabihf-debug.tar.gz 50a27653d6994b6a... e78afb97501fdc1d...
*-arm-linux-gnueabihf.tar.gz 81a07a9da6f858a0... 6f1046d349c70172...
*-osx-unsigned.dmg 3e2009f5309e90c5... 3d76472b83cb69c5...
*-osx-unsigned.tar.gz 4b62c341efb32478... f56fa9436b77af0f...
*-osx64.tar.gz 15db1c07fa4e9d30... 4ac9ee302dc308d2...
*-powerpc64-linux-gnu-debug.tar.gz a1e42ccb04825279... ac47d7d19548ee18...
*-powerpc64-linux-gnu.tar.gz 378f31682cfccc5f... 179d75cdfdef7b55...
*-powerpc64le-linux-gnu-debug.tar.gz 7a93b3ab8e90cf7e... 9e6f4edbb304f765...
*-powerpc64le-linux-gnu.tar.gz a1277ecf841344cf... 730b64a10a2b07de...
*-riscv64-linux-gnu-debug.tar.gz b43821a269135bd8... b093538d744e2e23...
*-riscv64-linux-gnu.tar.gz 7e274b4336478453... b85922e22e83b9e2...
*-win-unsigned.tar.gz 99e7f5561fe50fff... 2a506a2bb4ccd875...
*-win64-debug.zip 41e203dfc0daedec... c09814d390c57bf3...
*-win64-setup-unsigned.exe 63dc48e24fa4c39a... 82057dfe19025a8e...
*-win64.zip 5535e45ef7494fba... fc22d0050ea949c0...
*-x86_64-linux-gnu-debug.tar.gz 38eaad4f9ea9d1c6... 44fc15627569b7ee...
*-x86_64-linux-gnu.tar.gz a33a93e0a8ac54a5... 94fae578176d774c...
*.tar.gz f5fcc5155e947e72... 25d65c8d33f850ac...
guix_build.log fb12e25d5ee0f080... 35ccd4e8cfcc4b8e...
guix_build.log.diff 57e93b4c9abcb153...

Currently, ./build.sh doesn't seem to respect `CXX` and `CXXFLAGS`.
This becomes more obvious when you build depends with `CC` and `CXX` set,
without another compiler installed. i.e when using `g++-8` on Ubuntu
Bionic, but not having `g++` (GGC-7) installed. The native_b2 build will
fail with:

```bash
make -C depends/ -j9 CC=gcc-8 CXX=g++-8
...
/bitcoin/depends/sources/boost_1_71_0.tar.bz2: OK
Preprocessing native_b2...
Configuring native_b2...
Building native_b2...
B2_TOOLSET is gcc, but the 'gcc' command cannot be executed.
Make sure 'gcc' is in PATH, or use a different toolset.
```

Instead, don't set CXX or CXXFLAGS before calling `build.sh`.
In this case, we either get the default CXX (`g++`), or, we'll
get the CXX set by the user. i.e:
```bash
make -C depends/ -j9 CC=gcc-8 CXX=g++-8
...
/bitcoin/depends/sources/boost_1_71_0.tar.bz2: OK
Preprocessing native_b2...
Configuring native_b2...
Building native_b2...
g++-8 --version
g++-8 (Ubuntu 8.4.0-1ubuntu1~18.04) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

g++-8 -x c++ -std=c++11 -O2 -s .... <snip>
```
@fanquake fanquake force-pushed the actually_set_native_b2_cxx branch from 8b7f561 to eaa42d0 Compare November 22, 2021 03:54
@fanquake
Copy link
Member Author

@dongcarl Thanks for having a look.

The reason why you're seeing the current breakage in the first case is because our well-known var propagation in depends is limited to non-native packages, native packages do not yet get them: #18820 (comment).

I'm not sure about this. If I just remove the setting of CXX and CXXFLAGS before calling ./build.sh, the package ends up being built with the CXX set on the command line. I was fooled into thinking that our current Boost supported the --cxx options, as when I started using them, things were compiled as expected. However I think the actual fix here, was removing the the setting of CXX and CXXFLAGS, which was undermining the CXX and CXXFLAGS being set by the user.

I've updated this change to just drop CXX="$($(package)_cxx)" CXXFLAGS="$($(package)_cxxflags)", and everything now works as expected. If CXX is not set, you end up with CXX=g++, which is the current behaviour. However if CXX is set, that compiler will be used, as opposed to the build failure which happens now.

We can look at using --cxx if/when we move to a newer Boost, i.e: #23340.

@hebasto
Copy link
Member

hebasto commented Nov 22, 2021

This is still broken:

$ make --no-print-directory -C depends native_b2 CC=clang CXX=clang++
Extracting native_b2...
/home/hebasto/GitHub/bitcoin/depends/sources/boost_1_71_0.tar.bz2: OK
Preprocessing native_b2...
Configuring native_b2...
Building native_b2...
###
###
### Using 'gcc' toolset.
###
###
clang++ --version
clang version 10.0.0-4ubuntu1 
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
###
###
clang++ -x c++ -std=c++11 -O2 -s -DNDEBUG builtins.cpp class.cpp command.cpp compile.cpp constants.cpp cwd.cpp debug.cpp debugger.cpp execcmd.cpp filesys.cpp frames.cpp function.cpp glob.cpp hash.cpp hcache.cpp hdrmacro.cpp headers.cpp jam.cpp jambase.cpp jamgram.cpp lists.cpp make.cpp make1.cpp md5.cpp mem.cpp modules.cpp native.cpp object.cpp option.cpp output.cpp parse.cpp pathsys.cpp regexp.cpp rules.cpp scan.cpp search.cpp strings.cpp subst.cpp sysinfo.cpp timestamp.cpp variable.cpp w32_getreg.cpp modules/order.cpp modules/path.cpp modules/property-set.cpp modules/regex.cpp modules/sequence.cpp modules/set.cpp execunix.cpp fileunix.cpp pathunix.cpp -o b2
cp b2 bjam
Staging native_b2...
Postprocessing native_b2...
Caching native_b2...

Note Using 'gcc' toolset.

Please consider an alternative #23571.

@fanquake
Copy link
Member Author

Please consider an alternative #23571.

Lets look at that for now.

@fanquake fanquake closed this Nov 23, 2021
@bitcoin bitcoin locked and limited conversation to collaborators Nov 23, 2022
@fanquake fanquake deleted the actually_set_native_b2_cxx branch November 29, 2022 15:49
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants