linux/bootstrap-tools: move libstdc++ out of default library search path#209054
Merged
trofi merged 1 commit intoNixOS:stagingfrom Jan 12, 2023
Merged
linux/bootstrap-tools: move libstdc++ out of default library search path#209054trofi merged 1 commit intoNixOS:stagingfrom
trofi merged 1 commit intoNixOS:stagingfrom
Conversation
This change allows building new gcc during bootstrap without fear of
pulling in outdated libstdc++.so after g++ switched from bootstrapTools
to freshly built g++.
Noticed when tried to add early bootstrap stage to rebuild `gcc` before
`glibc` is fully untangled from `bootstrapTools` as a failure to built
`binutils`:
ld: dwp.o: in function `__gnu_cxx::new_allocator<gold::Dwp_output_file::Contribution>::allocate(unsigned long, void const*)':
/nix/store/...-gcc-11.3.0/include/c++/11.3.0/ext/new_allocator.h:116: undefined reference to `std::__throw_bad_array_new_length()'
The change survives existing bootstrap and unblockes early `gcc` bootstrap.
ghost
approved these changes
Jan 9, 2023
ghost
left a comment
There was a problem hiding this comment.
+2 Thank you for figuring this out! We should definitely merge this.
| fi | ||
|
|
||
| # path to version-specific libraries, like libstdc++.so | ||
| LIBSTDCXX_SO_DIR=$(echo $out/lib/gcc/*/*) |
There was a problem hiding this comment.
I checked that for all 16 of our existing bootstrap-files tarballs there is only one subdirectory under lib/gcc, and only one subdirectory beneath that (unpack-bootstrap-tools.sh is a huge pain because every time we touch it we have to rebuild the whole universe).
Contributor
Author
|
This regressed tests for bootstrap tarball building: I think it's a pure test issue. Test does not use any wrappers and assumes certain directory layout. Looking. |
trofi
added a commit
to trofi/nixpkgs
that referenced
this pull request
Jan 14, 2023
After NixOS#209054 we started moving libstdc++.so out of default glibc's paths. This exposed bootstrap tools build failure as: $ nix build --no-link -f ./pkgs/stdenv/linux/make-bootstrap-tools.nix ... > .../bin/bar: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory Note that bootstrap itself did not break. The change only expands handcrafted `-rpath` entries.
13 tasks
Contributor
Author
|
Proposed the fix as #210752 |
ConnorBaker
pushed a commit
to nixos-cuda/cuda-legacy
that referenced
this pull request
Apr 14, 2025
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like NixOS/nixpkgs#132343
#### Incorporates
- NixOS/nixpkgs#210004
- NixOS/nixpkgs#36948 (unreverted)
- NixOS/nixpkgs#210325
- NixOS/nixpkgs#210118
- NixOS/nixpkgs#210132
- NixOS/nixpkgs#210109
- NixOS/nixpkgs#213909
- NixOS/nixpkgs#216136
- NixOS/nixpkgs#216237
- NixOS/nixpkgs#210019
- NixOS/nixpkgs#216232
- NixOS/nixpkgs#216016
- NixOS/nixpkgs#217977
- NixOS/nixpkgs#217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @Ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: NixOS/nixpkgs#210004
[versioned directory]: NixOS/nixpkgs#209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: NixOS/nixpkgs#210112 (comment)
[crisis]: NixOS/nixpkgs#108305
[foreign]: NixOS/nixpkgs#170857 (comment)
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change allows building new gcc during bootstrap without fear of pulling in outdated libstdc++.so after g++ switched from bootstrapTools to freshly built g++.
Noticed when tried to add early bootstrap stage to rebuild
gccbeforeglibcis fully untangled frombootstrapToolsas a failure to builtbinutils:The change survives existing bootstrap and unblockes early
gccbootstrap.Description of changes
Things done
sandbox = trueset innix.conf? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)nixos/doc/manual/md-to-db.shto update generated release notes