llvmPackages_15: init at `15.0.7` by rrbutani · Pull Request #194634 · NixOS/nixpkgs

rrbutani · 2022-10-05T18:33:31Z

Note
Further work is being tracked in #213033

Description of changes

The commit messages in this PR have more details but at a high level this PR contains a few different kinds of changes:

introduces llvmPackages_15
switching to using clang from this package set to bootstrap libcxx, sidestepping some of the issues we've seen with older LLVM package sets
switches to ninja where possible
re-enables Polly by default
enables the LLVM tests on macOS (x86_64 and aarch64)
miscellaneous fixes (lldb, manpages) and updates (LLVM license, dropping some flags we don't need anymore)
mostly mechanical changes to make it easier to override the monorepo source used in the package set (the last two commits)

TODOs:

Checks:

llvmPackages_15.*
passthru tests
test cross / use as a cross compiler with this script
- only focusing on this passing eval successfully for now
  - i.e.:
    - nix path-info --file ./test.nix --derivation
    - nix path-info --file ./test.nix --derivation --arg systems '[ "aarch64-darwin" "x86_64-darwin" ]' --arg testStdenvPackages '[]'
- and checking that the following don't regress:
  - pkgsCross.x86_64-freebsd.boost (with llvmPackages = llvmPackages_15)
    - i.e. nix build --expr '(import ./. { system = "x86_64-linux"; overlays = [(f: p: { llvmPackages = f.llvmPackages_15; })]; }).pkgsCross.x86_64-freebsd.boost'
    - this is currently broken in libcxx
rdeps with nixpkgs-review
- not applicable since we're not bumping llvmPackages_latest

Closes #191132.

Things done

pkgs/development/compilers/llvm/15/libcxx/default.nix

pkgs/development/compilers/llvm/15/llvm/default.nix

pkgs/development/compilers/llvm/15/default.nix

rrbutani · 2022-10-07T20:49:20Z

With ab9d3df, libcxx-abi (and the rest of llvmPackages_15) builds fine on aarch64-linux as well as x86_64-linux. Darwin is now hitting an unrelated issue in compiler-rt; I'll tackle that next.

After that:

lldb-manpages llvmPackages_15: init at 15.0.7 #194634 (comment)
the skipped LLVM lit tests llvmPackages_15: init at 15.0.7 #194634 (comment)

rrbutani · 2022-10-07T21:57:18Z

Fixed one compiler-rt build issue, hit another one.

compiler-rt's build now attempts to detect whether the ld used automatically does codesigning, based on version. Our cctools ld is too old for this but postLinkSignHook handles codesigning for us, regardless.

Unfortunately there doesn't seem to be a way to communicate this to the compiler-rt build machinery. In the interest of avoiding making another patch, I tried to add sigtool to the derivation and just let the compiler-rt build sign some dylibs twice but unfortunately compiler-rt's build uses --sign instead of -s which the sigtool in nixpkgs has not been updated to support.

I opened a PR for this (#194994) but given that sigtool has many transitive deps and that that PR is likely to trigger many rebuilds I think it's probably faster to just add a compiler-rt patch.

rrbutani · 2022-10-07T22:19:13Z

With 974ea1c this now builds on aarch64-darwin.

Testing x86_64-darwin now; hopefully it Just Works 🤞.

Update: lldb fails to build on x86_64-darwin; it's failing on #include <bsm/audit_session.h> and CPU_SUBTYPE_ARM64E not being present in mach/machine.h 😕

`<bsm/audit_session.h>`

This isn't an issue on aarch64-darwin because we use a newer SDK there:

nixpkgs/pkgs/top-level/darwin-packages.nix

Line 45 in 4a1fb3a

    
           apple_sdk = if stdenv.hostPlatform.isAarch64 then apple_sdk_11_0 else apple_sdk_10_12;

This is the change in upstream causing the issue: llvm/llvm-project@b945b62

lldb appears to only use AU_SESSION_FLAG_HAS_GRAPHIC_ACCESS from audit_session.h which, worst case, would be easy to stub or patch out.

Fortunately, this header is available in the macOS 10.12 SDK (i.e. nixpkgs.darwin.apple_sdk.sdk.dev). Is it okay for me to depend on this header? This was the only precedent in nixpkgs I could find:

nixpkgs/pkgs/development/libraries/webkitgtk/default.nix

Line 168 in 299ec2c

    
                 install -Dm444 "${lib.getDev apple_sdk.sdk}"/include/libproc.h "$out"/include/libproc.h

`CPU_SUBTYPE_ARM64E`

For CPU_SUBTYPE_ARM64E I am not sure why we're just hitting this issue now; it appears to have been added in LLVM 13? llvm/llvm-project@479b672

Edit: oh, of course; lldb is marked as broken for LLVM 13 and 14:

nixpkgs/pkgs/development/compilers/llvm/13/lldb/default.nix

Line 105 in 88a2353

broken = stdenv.isDarwin;

nixpkgs/pkgs/development/compilers/llvm/14/lldb/default.nix

Line 119 in 299ec2c

    
           broken = stdenv.isDarwin; # error: use of undeclared identifier 'CPU_SUBTYPE_ARM64E'

It's not marked as broken in llvmPackages_git but I'm willing to bet that has the same issue.

Fixed in 040efed and opened #195013 to fix lldb 14 and lldb 13 on macOS x86_64.

@lovesegfault

None of the patches required any touch-up; the only change of note is: - due to changes in the libc++/libc++abi build (https://reviews.llvm.org/D120719 and https://reviews.llvm.org/D131037) we have to add an extra build option to the libc++ header only build that sidesteps bits of the libc++ build config that assume libc++-abi is present in the build: https://github.com/llvm/llvm-project/blob/4f827318e3e8ccab4ff131e06234caa827e91e4e/libcxx/src/CMakeLists.txt#L255-L256 Rather than maintaining a precise set of build options that let us dodge referencing libc++-abi variables in the libc++ header only build, we set `LIBCXX_CXX_ABI` to `none`, as suggested by @lovesegfault. More discussion about this here: #194634 (comment) Co-authored-by: Bernardo Meurer <[email protected]>

See the comments here for context: #194634 (comment) Co-authored-by: Weijia Wang <[email protected]>

see: #194634 (comment)

this is a backport of 8afa321 from NixOS#194634 (llvmPackages_15)

…as overridable args backport of d231d18 from NixOS#194634 (llvmPackages_15)

@lovesegfault

Port of bc4dbee ("llvmPackages_15: updates for LLVM 15"). None of the patches required any touch-up; the only change of note is: - due to changes in the libc++/libc++abi build (https://reviews.llvm.org/D120719 and https://reviews.llvm.org/D131037) we have to add an extra build option to the libc++ header only build that sidesteps bits of the libc++ build config that assume libc++-abi is present in the build: https://github.com/llvm/llvm-project/blob/4f827318e3e8ccab4ff131e06234caa827e91e4e/libcxx/src/CMakeLists.txt#L255-L256 Rather than maintaining a precise set of build options that let us dodge referencing libc++-abi variables in the libc++ header only build, we set `LIBCXX_CXX_ABI` to `none`, as suggested by @lovesegfault. More discussion about this here: #194634 (comment) Co-authored-by: Bernardo Meurer <[email protected]>

Port of 4fabcf4 ("llvmPackages_15: apply #211230 to llvmPackages_15"). See the comments here for context: #194634 (comment) Co-authored-by: Weijia Wang <[email protected]>

Partial port of 00839fe ("llvmPackages_15.compiler-rt: fixes for Darwin") see: #194634 (comment)

Partial port of 00839fe ("llvmPackages_15.compiler-rt: fixes for Darwin") see: NixOS#194634 (comment)

@lovesegfault

Port of bc4dbee ("llvmPackages_15: updates for LLVM 15"). None of the patches required any touch-up; the only change of note is: - due to changes in the libc++/libc++abi build (https://reviews.llvm.org/D120719 and https://reviews.llvm.org/D131037) we have to add an extra build option to the libc++ header only build that sidesteps bits of the libc++ build config that assume libc++-abi is present in the build: https://github.com/llvm/llvm-project/blob/4f827318e3e8ccab4ff131e06234caa827e91e4e/libcxx/src/CMakeLists.txt#L255-L256 Rather than maintaining a precise set of build options that let us dodge referencing libc++-abi variables in the libc++ header only build, we set `LIBCXX_CXX_ABI` to `none`, as suggested by @lovesegfault. More discussion about this here: NixOS#194634 (comment) Co-authored-by: Bernardo Meurer <[email protected]>

Port of 4fabcf4 ("llvmPackages_15: apply NixOS#211230 to llvmPackages_15"). See the comments here for context: NixOS#194634 (comment) Co-authored-by: Weijia Wang <[email protected]>

this is a backport of 8afa321 from NixOS#194634 (llvmPackages_15)

…as overridable args backport of d231d18 from NixOS#194634 (llvmPackages_15)

this is a backport of 8afa321 from #194634 (llvmPackages_15)

…as overridable args backport of d231d18 from #194634 (llvmPackages_15)

ofborg bot added the 8.has: package (new) This PR adds a new package label Oct 5, 2022

ofborg bot requested review from 7c6f434c, dtzWill, lovek323 and primeos October 5, 2022 18:45

rrbutani force-pushed the feature/llvm-15 branch from c11356f to 30373c3 Compare October 5, 2022 18:55

zowoq mentioned this pull request Oct 6, 2022

llvmPackages_15: init at 15.0.2 #194781

Closed

13 tasks

lovesegfault reviewed Oct 7, 2022

View reviewed changes

pkgs/development/compilers/llvm/15/libcxx/default.nix Outdated Show resolved Hide resolved

lovesegfault reviewed Oct 7, 2022

View reviewed changes

pkgs/development/compilers/llvm/15/llvm/default.nix Outdated Show resolved Hide resolved

lovesegfault reviewed Oct 7, 2022

View reviewed changes

pkgs/development/compilers/llvm/15/default.nix Outdated Show resolved Hide resolved

lovesegfault reviewed Oct 7, 2022

View reviewed changes

pkgs/development/compilers/llvm/15/default.nix Outdated Show resolved Hide resolved

rrbutani force-pushed the feature/llvm-15 branch 2 times, most recently from 79aa175 to 8484b4d Compare October 7, 2022 20:39

rrbutani force-pushed the feature/llvm-15 branch from 8484b4d to 123aa5e Compare October 7, 2022 20:43

rrbutani mentioned this pull request Oct 7, 2022

sigtool: 0.1.2 → 0.1.3 #194994

Closed

13 tasks

rrbutani force-pushed the feature/llvm-15 branch from 123aa5e to d0c37e4 Compare October 7, 2022 22:17

RaitoBezarius pushed a commit that referenced this pull request Mar 26, 2023

llvmPackages_git: apply #211230 to llvmPackages_git

2f0ae7f

See the comments here for context: #194634 (comment) Co-authored-by: Weijia Wang <[email protected]>

RaitoBezarius pushed a commit that referenced this pull request Mar 26, 2023

llvmPackages_git.compiler-rt: fixes for Darwin

e3e564e

see: #194634 (comment)

rrbutani added a commit to rrbutani/nixpkgs that referenced this pull request Apr 3, 2023

llvmPackages_git.llvm: add checks for the LLVM version

f74a6ce

this is a backport of 8afa321 from NixOS#194634 (llvmPackages_15)

rrbutani added a commit to rrbutani/nixpkgs that referenced this pull request Apr 3, 2023

llvmPackages_git: expose the release information and monorepo source …

20674e4

…as overridable args backport of d231d18 from NixOS#194634 (llvmPackages_15)

RaitoBezarius pushed a commit that referenced this pull request Apr 5, 2023

llvmPackages_git.compiler-rt: fixes for Darwin

6707fa4

Partial port of 00839fe ("llvmPackages_15.compiler-rt: fixes for Darwin") see: #194634 (comment)

rrbutani added a commit to rrbutani/nixpkgs that referenced this pull request Apr 5, 2023

llvmPackages_git.compiler-rt: fixes for Darwin

e6a57c7

Partial port of 00839fe ("llvmPackages_15.compiler-rt: fixes for Darwin") see: NixOS#194634 (comment)

rrbutani added a commit to rrbutani/nixpkgs that referenced this pull request Apr 5, 2023

llvmPackages_git.llvm: add checks for the LLVM version

d0770bc

this is a backport of 8afa321 from NixOS#194634 (llvmPackages_15)

rrbutani added a commit to rrbutani/nixpkgs that referenced this pull request Apr 5, 2023

llvmPackages_git: expose the release information and monorepo source …

bd864b5

…as overridable args backport of d231d18 from NixOS#194634 (llvmPackages_15)

ghost mentioned this pull request Apr 17, 2023

clang_15: fix build!=(host==target) cross compilation #226551

Merged

12 tasks

alyssais pushed a commit that referenced this pull request Apr 17, 2023

llvmPackages_git.llvm: add checks for the LLVM version

4917dc7

this is a backport of 8afa321 from #194634 (llvmPackages_15)

alyssais pushed a commit that referenced this pull request Apr 17, 2023

llvmPackages_git: expose the release information and monorepo source …

a77eef2

…as overridable args backport of d231d18 from #194634 (llvmPackages_15)

Artturin mentioned this pull request Nov 25, 2023

llvmPackages_git: remove #269994

Closed

13 tasks

ghost mentioned this pull request Jan 5, 2024

llvmPackages_{15,16,17,git}.libcxx: use LIBCXX_CXX_ABI=libcxxabi link flags on darwin #278945

Merged

13 tasks

rrbutani mentioned this pull request May 6, 2024

Request for a Zulip account for governance discussion NixOS/foundation#143

Closed

rrbutani added the 6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related label May 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

llvmPackages_15: init at `15.0.7`#194634

llvmPackages_15: init at `15.0.7`#194634
wegank merged 30 commits intoNixOS:masterfrom
rrbutani:feature/llvm-15

rrbutani commented Oct 5, 2022 •

edited by alyssais

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rrbutani commented Oct 7, 2022 •

edited

Loading

Uh oh!

rrbutani commented Oct 7, 2022

Uh oh!

rrbutani commented Oct 7, 2022 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Uh oh!

Conversation

rrbutani commented Oct 5, 2022 • edited by alyssais Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Things done

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rrbutani commented Oct 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rrbutani commented Oct 7, 2022

Uh oh!

rrbutani commented Oct 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

<bsm/audit_session.h>

CPU_SUBTYPE_ARM64E

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

rrbutani commented Oct 5, 2022 •

edited by alyssais

Loading

rrbutani commented Oct 7, 2022 •

edited

Loading

rrbutani commented Oct 7, 2022 •

edited

Loading

`<bsm/audit_session.h>`

`CPU_SUBTYPE_ARM64E`