Skip to content

llvmPackages_*.stdenv: use top-level libc++ on Darwin#264091

Closed
reckenrode wants to merge 18 commits intoNixOS:stagingfrom
reckenrode:stdenv-libcxx
Closed

llvmPackages_*.stdenv: use top-level libc++ on Darwin#264091
reckenrode wants to merge 18 commits intoNixOS:stagingfrom
reckenrode:stdenv-libcxx

Conversation

@reckenrode
Copy link
Contributor

@reckenrode reckenrode commented Oct 29, 2023 •
edited
Loading

Description of changes

This PR is an alternative to #263847 to resolve the nodejs_18 crashes in staging-next #263535.

Instead of overriding the version of libc++ used by clang in each derivation, this PR moves that logic to llvmPackages_*.clang. This allows llvmPackages_*.stdenv to build with one version of clang while linking dependencies built with different versions of clang (because they all use the same libc++), which helps avoid crashes at runtime.

This approach is similar to how the llvmPackages_*.stdenv works with Linux, which uses the same libstdc++ regardless of clang version. While libc++ maintains ABI compatibility, newer headers are not compatible with older compilers. This is addressed by using the original headers with the new libc++ (due to libc++’s ABI stability).

This PR was tested with nodejs_14, nodejs_16, nodejs_18, and Swift (which also performed this override). All built successfully. paperless-ngx.frontend was also built successfully to test node. Other packages built successfully include darktable (clang 13), cone (clang 7), and dar (clang 12).

Users who wish to use a specific version of libc++ with clang can continue using llvmPackages_*.libcxxClang and llvmPackages_*.libcxxStdenv.

Closes #263847.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@reckenrode
llvmPackages_5.clang: use the stdenv libc++ on Darwin
7b0d16c 
This change treats the top-level libc++ as the “system” libc++ on
Darwin, matching the way libstdc++ is used on Linux. The approach it
takes is to use the requested version’s libc++ headers while linking
against the top-level library. This is safe to do because libc++ is
built with the stable ABI on Darwin by default.

This change is necessary to support building a package with one version
of clang while linking against libraries that are built with different
versions. Otherwise, if they both use C++, they will link against
different versions of libc++ and (probably) crash at runtime. Examples of
where this is an issue include older versions of nodejs building with
clang 15 and darktable (which is currently built with clang 13).
@reckenrode
llvmPackages_6.clang: use the stdenv libc++ on Darwin
cb9a7e2 
This change treats the top-level libc++ as the “system” libc++ on
Darwin, matching the way libstdc++ is used on Linux. The approach it
takes is to use the requested version’s libc++ headers while linking
against the top-level library. This is safe to do because libc++ is
built with the stable ABI on Darwin by default.

This change is necessary to support building a package with one version
of clang while linking against libraries that are built with different
versions. Otherwise, if they both use C++, they will link against
different versions of libc++ and (probably) crash at runtime. Examples of
where this is an issue include older versions of nodejs building with
clang 15 and darktable (which is currently built with clang 13).
@reckenrode
llvmPackages_7.clang: use the stdenv libc++ on Darwin
fe66a29 
This change treats the top-level libc++ as the “system” libc++ on
Darwin, matching the way libstdc++ is used on Linux. The approach it
takes is to use the requested version’s libc++ headers while linking
against the top-level library. This is safe to do because libc++ is
built with the stable ABI on Darwin by default.

This change is necessary to support building a package with one version
of clang while linking against libraries that are built with different
versions. Otherwise, if they both use C++, they will link against
different versions of libc++ and (probably) crash at runtime. Examples of
where this is an issue include older versions of nodejs building with
clang 15 and darktable (which is currently built with clang 13).
@reckenrode
llvmPackages_8.clang: use the stdenv libc++ on Darwin
4d837bf 
This change treats the top-level libc++ as the “system” libc++ on
Darwin, matching the way libstdc++ is used on Linux. The approach it
takes is to use the requested version’s libc++ headers while linking
against the top-level library. This is safe to do because libc++ is
built with the stable ABI on Darwin by default.

This change is necessary to support building a package with one version
of clang while linking against libraries that are built with different
versions. Otherwise, if they both use C++, they will link against
different versions of libc++ and (probably) crash at runtime. Examples of
where this is an issue include older versions of nodejs building with
clang 15 and darktable (which is currently built with clang 13).
@reckenrode
llvmPackages_9.clang: use the stdenv libc++ on Darwin
58ccbaf 
This change treats the top-level libc++ as the “system” libc++ on
Darwin, matching the way libstdc++ is used on Linux. The approach it
takes is to use the requested version’s libc++ headers while linking
against the top-level library. This is safe to do because libc++ is
built with the stable ABI on Darwin by default.

This change is necessary to support building a package with one version
of clang while linking against libraries that are built with different
versions. Otherwise, if they both use C++, they will link against
different versions of libc++ and (probably) crash at runtime. Examples of
where this is an issue include older versions of nodejs building with
clang 15 and darktable (which is currently built with clang 13).
@reckenrode
llvmPackages_10.clang: use the stdenv libc++ on Darwin
a1d08cb 
This change treats the top-level libc++ as the “system” libc++ on
Darwin, matching the way libstdc++ is used on Linux. The approach it
takes is to use the requested version’s libc++ headers while linking
against the top-level library. This is safe to do because libc++ is
built with the stable ABI on Darwin by default.

This change is necessary to support building a package with one version
of clang while linking against libraries that are built with different
versions. Otherwise, if they both use C++, they will link against
different versions of libc++ and (probably) crash at runtime. Examples of
where this is an issue include older versions of nodejs building with
clang 15 and darktable (which is currently built with clang 13).
@reckenrode
llvmPackages_11.clang: use the stdenv libc++ on Darwin
0ab9ddd 
This change treats the top-level libc++ as the “system” libc++ on
Darwin, matching the way libstdc++ is used on Linux. The approach it
takes is to use the requested version’s libc++ headers while linking
against the top-level library. This is safe to do because libc++ is
built with the stable ABI on Darwin by default.

This change is necessary to support building a package with one version
of clang while linking against libraries that are built with different
versions. Otherwise, if they both use C++, they will link against
different versions of libc++ and (probably) crash at runtime. Examples of
where this is an issue include older versions of nodejs building with
clang 15 and darktable (which is currently built with clang 13).
@reckenrode
llvmPackages_12.clang: use the stdenv libc++ on Darwin
e6a8275 
This change treats the top-level libc++ as the “system” libc++ on
Darwin, matching the way libstdc++ is used on Linux. The approach it
takes is to use the requested version’s libc++ headers while linking
against the top-level library. This is safe to do because libc++ is
built with the stable ABI on Darwin by default.

This change is necessary to support building a package with one version
of clang while linking against libraries that are built with different
versions. Otherwise, if they both use C++, they will link against
different versions of libc++ and (probably) crash at runtime. Examples of
where this is an issue include older versions of nodejs building with
clang 15 and darktable (which is currently built with clang 13).
@reckenrode
llvmPackages_13.clang: use the stdenv libc++ on Darwin
b93182e 
This change treats the top-level libc++ as the “system” libc++ on
Darwin, matching the way libstdc++ is used on Linux. The approach it
takes is to use the requested version’s libc++ headers while linking
against the top-level library. This is safe to do because libc++ is
built with the stable ABI on Darwin by default.

This change is necessary to support building a package with one version
of clang while linking against libraries that are built with different
versions. Otherwise, if they both use C++, they will link against
different versions of libc++ and (probably) crash at runtime. Examples of
where this is an issue include older versions of nodejs building with
clang 15 and darktable (which is currently built with clang 13).
@reckenrode
llvmPackages_14.clang: use the stdenv libc++ on Darwin
685e3cf 
This change treats the top-level libc++ as the “system” libc++ on
Darwin, matching the way libstdc++ is used on Linux. The approach it
takes is to use the requested version’s libc++ headers while linking
against the top-level library. This is safe to do because libc++ is
built with the stable ABI on Darwin by default.

This change is necessary to support building a package with one version
of clang while linking against libraries that are built with different
versions. Otherwise, if they both use C++, they will link against
different versions of libc++ and (probably) crash at runtime. Examples of
where this is an issue include older versions of nodejs building with
clang 15 and darktable (which is currently built with clang 13).
@reckenrode
llvmPackages_15.clang: use the stdenv libc++ on Darwin
5cdf21d 
This change treats the top-level libc++ as the “system” libc++ on
Darwin, matching the way libstdc++ is used on Linux. The approach it
takes is to use the requested version’s libc++ headers while linking
against the top-level library. This is safe to do because libc++ is
built with the stable ABI on Darwin by default.

This change is necessary to support building a package with one version
of clang while linking against libraries that are built with different
versions. Otherwise, if they both use C++, they will link against
different versions of libc++ and (probably) crash at runtime. Examples of
where this is an issue include older versions of nodejs building with
clang 15 and darktable (which is currently built with clang 13).
@reckenrode
llvmPackages_16.clang: use the stdenv libc++ on Darwin
da8ba14 
This change treats the top-level libc++ as the “system” libc++ on
Darwin, matching the way libstdc++ is used on Linux. The approach it
takes is to use the requested version’s libc++ headers while linking
against the top-level library. This is safe to do because libc++ is
built with the stable ABI on Darwin by default.

This change is necessary to support building a package with one version
of clang while linking against libraries that are built with different
versions. Otherwise, if they both use C++, they will link against
different versions of libc++ and (probably) crash at runtime. Examples of
where this is an issue include older versions of nodejs building with
clang 15 and darktable (which is currently built with clang 13).
@reckenrode
Revert "swift: use stdenv libc++ on Darwin"
2712c86 
This reverts commit efdf4e9.
@reckenrode
nodejs_14: drop manual override of libc++
d1cfea5
@reckenrode
nodejs_14: use Python 3.10 to build
c41ddc4
@reckenrode
nodejs_16: drop manual override of libc++
1ae5372
@reckenrode
Revert "nodejs_18: fix build with clang 16"
4bcd49c 
This reverts commit 81e4ca5.
@reckenrode
nodejs_18: fix build with clang 16 mk2
f33a858 
Trying to backport the fixes from v8 caused crashes with npm when
building other packages, so just build it with clang 15
@github-actions github-actions bot added the 6.topic: nodejs Node.js is a free, open-source, cross-platform JavaScript runtime environment label Oct 29, 2023
@ofborg ofborg bot added 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 501-1000 This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. labels Oct 29, 2023
@ofborg ofborg bot added the 10.rebuild-linux: 1001-2500 This PR causes many rebuilds on Linux and should target the staging branches. label Oct 29, 2023
@reckenrode reckenrode changed the title llvmPackages_*\.stdenv llvmPackages_*.stdenv Oct 29, 2023
@reckenrode reckenrode mentioned this pull request Oct 29, 2023
Merged
13 tasks
@wegank
Copy link
Member

wegank commented Oct 30, 2023

Can confirm that this PR fixes #244319.

@reckenrode reckenrode marked this pull request as draft October 30, 2023 15:55
@reckenrode
Copy link
Contributor Author

reckenrode commented Oct 30, 2023

Converting this to draft since it missed the deadline for breaking LLVM changes. I’ll pick it back up post-23.11.

@reckenrode reckenrode changed the title llvmPackages_*.stdenv llvmPackages_*.stdenv: use top-level libc++ on Darwin Oct 30, 2023
@vcunat vcunat changed the base branch from staging-next to staging October 30, 2023 16:35
@ofborg ofborg bot added the 6.topic: darwin Running or building packages on Darwin label Oct 30, 2023
@wegank wegank mentioned this pull request Nov 4, 2023
Merged
13 tasks
reckenrode added a commit to reckenrode/nixpkgs that referenced this pull request Nov 4, 2023
@reckenrode
stdenvAdapters: add overrideLibcxx
cc4fcc1 
This was taken from NixOS#264091 to use in the interim before that PR lands
(sometime after the release of 23.11). It allows different versions of
clang to link the same libc++, allowing dependencies to be linked when
they are built with a different version of clang than the stdenv.
@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 20, 2024
@rrbutani rrbutani added the 6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related label May 27, 2024
@wegank wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 4, 2024
@reckenrode reckenrode mentioned this pull request Oct 3, 2024
Merged
52 tasks
@aviallon
Copy link
Contributor

aviallon commented Oct 17, 2024

Closed by #346043

@aviallon aviallon closed this Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@RaitoBezarius RaitoBezarius Awaiting requested review from RaitoBezarius

@lovek323 lovek323 Awaiting requested review from lovek323

@Ericson2314 Ericson2314 Awaiting requested review from Ericson2314

@alyssais alyssais Awaiting requested review from alyssais

@rrbutani rrbutani Awaiting requested review from rrbutani

@primeos primeos Awaiting requested review from primeos

@sternenseemann sternenseemann Awaiting requested review from sternenseemann

@dtzWill dtzWill Awaiting requested review from dtzWill

Assignees

No one assigned

Labels

2.status: merge conflict This PR has merge conflicts with the target branch 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: darwin Running or building packages on Darwin 6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related 6.topic: nodejs Node.js is a free, open-source, cross-platform JavaScript runtime environment 10.rebuild-darwin: 501-1000 This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 1001-2500 This PR causes many rebuilds on Linux and should target the staging branches.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@reckenrode @wegank @aviallon @rrbutani