Skip to content

llvmPackages.llvm: Drop dependency on target through libbfd#355532

Merged
emilazy merged 1 commit intoNixOS:stagingfrom
pwaller:drop-bfd-dependency
Nov 15, 2024
Merged

llvmPackages.llvm: Drop dependency on target through libbfd#355532
emilazy merged 1 commit intoNixOS:stagingfrom
pwaller:drop-bfd-dependency

Conversation

@pwaller
Copy link
Contributor

@pwaller pwaller commented Nov 12, 2024

Currently the target triple leaks into the clang build via llvm using
libbfd, whose build varies according to the target triple.

LLVM only uses libbfd to enable LTO via the linker plugin (called
LLVMgold.so, though multiple linkers can use the same plugin).

Drop the dependency on the libbfd build, and consume the only needed
source instead. (This would be a good use of CA-derivations FWIW).

The result of this commit is that these match:

  • nix eval --raw nixpkgs#clang.cc
  • nix eval --raw nixpkgs#pkgsStatic.pkgsLLVM.stdenv.cc.cc
  • nix eval --raw nixpkgs#pkgsCross.aarch64-multiplatform.pkgsLLVM.stdenv.cc.cc

This means fewer clang builds will be needed to support cross
configurations, and users wanting to target an exotic cross
configuration should be able to do so without a rebuild of clang.

Also drops libbfd.hasPluginAPI which no longer has any users.

Signed-off-by: Peter Waller [email protected]

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added the 6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related label Nov 12, 2024
@pwaller
Copy link
Contributor Author

pwaller commented Nov 12, 2024

Ideas on where to put a test for this welcome. I'm thinking it would be nice to have an assertion somewhere that checks clang.cc == pkgsStatic.pkgsLLVM.stdenv.cc.cc == pkgsCross.aarch64-multiplatform.pkgsLLVM.stdenv.cc.cc but I'm not sure of an appropriate place to put it.

@pwaller pwaller force-pushed the drop-bfd-dependency branch from a318410 to 5d94d6f Compare November 12, 2024 21:56
@pwaller pwaller force-pushed the drop-bfd-dependency branch from 5d94d6f to 54d9479 Compare November 13, 2024 17:41
@ofborg ofborg bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Nov 14, 2024
@sternenseemann
Copy link
Member

I wonder if the header has any target specific CPP? If not, we could actually build LLVMgold.so unconditonally. Currently we have some silly non-sharing of LLVM depending on whether platforms are supported by bfd or not. Another worthwhile investigation could be to figure out if we could split out the plugin from the LLVM derivation altogether which would arguably be the cleanest solution.

Copy link
Member

@emilazy emilazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default value of enableGoldPlugin will need adjusting (to true, presumably) to accomplish the intended result of this PR.

To achieve the final goal of a target‐independent Clang we will also have to decide what to do about the nostdlibinc patch. We could make it conditional on the target inside the patch rather than outside, but currently it just hacks into command‐line parsing where I don’t believe that information is yet reliably available, so it would likely need moving deeper into LLVM internals. Pinging @reckenrode, since AIUI he was of the opinion that the patch is working around the wrong thing.

Frankly I think we could just move that flag back to the wrapper: I believe the problem it fixes is that an unwrapped libcxxStdenv Clang picks up libstdc++ headers, but (a) it is expected that unwrapped compilers may not have all the necessary flags to integrate with Nix libraries and (b) it is normal on other distros to have to specify a flag to Clang when you want a C++ library other than the system’s default. Ideally, perhaps, we could make it so that Clang does not default to either of libstdc++ or libc++ regardless of platform, and requires a C++ library to be explicitly specified when compiling C++. I am not sure if that is a configuration knob they expose or not.

Though with that said, 1e26d33 says that there was a problem with the wrapped Clang without the patch, so I don’t know.

@pwaller pwaller force-pushed the drop-bfd-dependency branch from 54d9479 to fcff31e Compare November 14, 2024 19:30
@pwaller pwaller changed the title llvmPackages.llvm: Enable single clang build for multiple targets llvmPackages.llvm: Drop dependency on target through libbfd Nov 14, 2024
@pwaller
Copy link
Contributor Author

pwaller commented Nov 14, 2024

To simplify this PR I've reduced the scope by changing the PR title and body -- now it's not quite as complete, it's just dropping the dependency on target creeping in via libbfd. Other dependency creep can be solved in other PRs.

I've dropped the enableGoldPlugin and hasPluginAPI since they don't have users in nixpkgs anymore, if someone wants to disable this feature they can do so by passing llvmPackages.override { libbfd = null; }.

@pwaller pwaller force-pushed the drop-bfd-dependency branch from fcff31e to 6c94a6f Compare November 14, 2024 19:34
Copy link
Member

@emilazy emilazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One required change, and this needs to target staging. LGTM otherwise.

@pwaller pwaller force-pushed the drop-bfd-dependency branch from 6c94a6f to 0c284c8 Compare November 14, 2024 21:42
@pwaller pwaller changed the base branch from master to staging November 14, 2024 21:43
Currently the target triple leaks into the clang build via llvm using
libbfd, whose build varies according to the target triple.

LLVM only uses libbfd to enable LTO via the linker plugin (called
LLVMgold.so, though multiple linkers can use the same plugin).

Drop the dependency on the libbfd build, and consume the only needed
source instead.  (This would be a good use of CA-derivations FWIW).

The result of this commit is that these match:

* nix eval --raw nixpkgs#clang.cc
* nix eval --raw nixpkgs#pkgsStatic.pkgsLLVM.stdenv.cc.cc
* nix eval --raw nixpkgs#pkgsCross.aarch64-multiplatform.pkgsLLVM.stdenv.cc.cc

This means fewer clang builds will be needed to support cross
configurations, and users wanting to target an exotic cross
configuration should be able to do so without a rebuild of clang.

Also drops libbfd.hasPluginAPI which no longer has any users.

Signed-off-by: Peter Waller <[email protected]>
@pwaller pwaller force-pushed the drop-bfd-dependency branch from 0c284c8 to 7903d0b Compare November 14, 2024 21:43
@pwaller
Copy link
Contributor Author

pwaller commented Nov 14, 2024

Rebased to staging.

@pwaller pwaller marked this pull request as ready for review November 14, 2024 21:50
@emilazy emilazy merged commit bee6382 into NixOS:staging Nov 15, 2024
emilazy pushed a commit that referenced this pull request Nov 21, 2024
In theory, Clang is a cross-compiler by default. After the following
PRs, this is true on platforms other than Darwin.

- #355532
- #356162

The reason Darwin needs additional changes is it only overlays Clang and
LLVM in the native target case. When cross-compiling, it builds Clang
and LLVM again. This is unnecessary as long as the Darwin stdenv avoids
propagating the Clang wrappers, which are target-dependent.

In theory, ld64 is also a cross-linker by default. Realizing that for
nixpkgs requires additional work that will be done separately.
@pwaller pwaller deleted the drop-bfd-dependency branch November 22, 2024 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants