Skip to content

llvmPackages: Reinstate overriding across whole package set#341855

Merged
RossComputerGuy merged 1 commit intoNixOS:masterfrom
pwaller:reinstate-llvmPackages-overrides
Sep 22, 2024
Merged

llvmPackages: Reinstate overriding across whole package set#341855
RossComputerGuy merged 1 commit intoNixOS:masterfrom
pwaller:reinstate-llvmPackages-overrides

Conversation

@pwaller
Copy link
Contributor

@pwaller pwaller commented Sep 14, 2024

Description of changes

Pull #320261 introduced the possibility to consistently override
dependencies within an llvm package set. I think #325175 accidentally
dropped this, so reinstate it.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added the 6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related label Sep 14, 2024
@pwaller
Copy link
Contributor Author

pwaller commented Sep 14, 2024

I'm going to give this a bit of testing and report back if it works for me. When I'm happy I'll mark it ready for review.

@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Sep 14, 2024
@pwaller pwaller marked this pull request as ready for review September 15, 2024 09:07
Pull NixOS#320261 introduced the possibility to consistently override
dependencies within an llvm package set. I think NixOS#325175 accidentally
dropped this, so reinstate it.

Signed-off-by: Peter Waller <[email protected]>
@pwaller pwaller force-pushed the reinstate-llvmPackages-overrides branch from e152016 to ddafd4b Compare September 15, 2024 09:10
@pwaller
Copy link
Contributor Author

pwaller commented Sep 15, 2024

I ran nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD" which successfully dropped me into a shell, and I've tested this works for my purposes.

/cc other reviewers of #325175 who may me interested: @paparodeo @reckenrode @alyssais

pwaller added a commit to pwaller/nixpkgs that referenced this pull request Sep 15, 2024
cmake flags have a 'last flag wins' logic, so by appending to the end of
the flags it is possible to override any cmake flag.

It also ignores (and warns) if a flag is unused, so passing flags across
all packages should be safe if you want to target one package.

In combination with NixOS#320261, this PR allows consistently overriding all
packages within LLVM with additional cmake arguments. Consistency here
means for example 'if you override LLVM, then all dependencies on it are
also see the overridden LLVM in their input'. Consistency is hard to
achieve with the other obvious way of implementing this as a user: if
you use overrideAttrs then you have to write a big mess of override code
in order to override all dependents, and this can be very difficult in a
cross-compilation scenario using crossSystem and useLLVM, for example.

With this PR it is possible to write an overlay which overlays
`llvmPackages` with `llvmPackage.override { devExtraCmakeFlags = [ ... ]; }`,
and then the toolchain used with useLLVM in effect should respect
these flags.

This is useful in development for experimenting with the effect of
various flags, hence the chosen name `devCmakeFlags`.

This won't work out of the box without NixOS#341855 applied, which fixes
override passthrough.

See-Also: NixOS#320261, NixOS#341855
Signed-off-by: Peter Waller <[email protected]>
Copy link
Member

@RossComputerGuy RossComputerGuy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i approb

Copy link
Contributor

@paparodeo paparodeo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

verified same tests from #320261 (review) ran as expected.

looks good. minor comment about overriding stdenv inline.

version
;
}
// packageSetArgs # Allow overrides.
Copy link
Contributor

@paparodeo paparodeo Sep 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this go after the stdenv so it (stdenv) can get overridden too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking was that this involves conditional logic to pull in a different stdenv; that conditional logic still needs to apply. If the user wants to override gcc12Stdenv or stdenv they still can, through the original argument mechanism. What we don't want is to override the stdenv which has been conditionally set to gcc12Stdenv with the original stdenv.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What we don't want is to override the stdenv which has been conditionally set to gcc12Stdenv with the original stdenv.

i think that is exactly what we want -- an override that does an override. anyway, i don't think this needs needs to hold up merging given it is pretty minor and the llvm which pins gcc12 will eventually get dropped.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW i don't think this comment should've been marked as resolved but i am unable to change to unresolved.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marked unresolved for you. I think we need someone to break the tie. My point is that the recipe has conditional logic which pulls in the correct stdenv required to build a given LLVM version. If I splat the args in after the conditional logic, the original stdenv would override the conditional logic. This is clearly not what we want. So, I reason that if we put the conditional logic after the args, it's still possible for someone to apply overrides; they just need to override gcc12Stdenv on the input if the conditional logic is in effect.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. i am just being dense. moving packageSetArgs below the conditional breaks llvm13 as clobbers stdenv . so that won't work.

tho -- i am getting build failures just trying to build llvmPackages_13 from this PR with no modifications.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for testing - can you confirm that the commit on which this is based will build llvmPackages_13? (If not just say and I'll do it when I get a chance)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but maybe it was marked broken on master.

error: Package ‘lldb-13.0.1’ in  is marked as broken, refusing to evaluate.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for testing - can you confirm that the commit on which this is based will build llvmPackages_13? (If not just say and I'll do it when I get a chance)

4f807e8 appears to have the same issue

error: build of '/nix/store/4813qnnz6ymba266v0yjas8b8j6faggl-llef-1.2.0.drv', '/nix/store/s3n5dm0s2x57yr18mqrzy8r0j1nvhzi3-lldb-13.0.1.drv' failed

@wegank wegank added the 12.approvals: 1 This PR was reviewed and approved by one person. label Sep 17, 2024
@RossComputerGuy RossComputerGuy added 12.approvals: 2 This PR was reviewed and approved by two persons. and removed 12.approvals: 1 This PR was reviewed and approved by one person. labels Sep 22, 2024
@RossComputerGuy
Copy link
Member

Shall we merge?

@RossComputerGuy RossComputerGuy merged commit 25e07e7 into NixOS:master Sep 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. 12.approvals: 2 This PR was reviewed and approved by two persons.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants