Full build failure log is here:
build-failure.log.gz

I'm not quite sure what is failing. Will continue looking into it.

Looks like that should be using the binutils wrapper and not the clang wrapper.

Ah, strip lives in stdenv.cc.bintools.bintools. Weird. Is there a reason, we don't create a link to it in pkgs/development/compilers/llvm/common/bintools.nix as well?

There should be a strip in bintools:

(I'm assuming llvm-objcopy must be multi-call or something for it to be the target of the link.)

There should be a strip in bintools:

nixpkgs/pkgs/development/compilers/llvm/common/bintools.nix

Line 32 in d78d093

ln -s $llvmBin/llvm-objcopy $out/bin/${targetPrefix}strip

(I'm assuming llvm-objcopy must be multi-call or something for it to be the target of the link.)

Only stdenv.cc.bintools.bintools has strip:

❯ nix-build -A pkgsLLVM.stdenv.cc.bintools 
/nix/store/rqlaazha773dz6xidv8nl2qbrn2j3fjw-x86_64-unknown-linux-gnu-llvm-binutils-wrapper-18.1.8

nixpkgs on  kernel-lld [$?] 
❯ ls result/bin | grep strip

❯ nix-build -A pkgsLLVM.stdenv.cc.bintools.bintools
/nix/store/kamnb5s922xg1hj0mkyjvds0bbaygy6w-llvm-binutils-18.1.8
 
❯ ls result/bin | grep strip                       
x86_64-unknown-linux-gnu-llvm-bitcode-strip
x86_64-unknown-linux-gnu-llvm-strip
x86_64-unknown-linux-gnu-strip

There is also a llvm-strip binary so the code in bintools.nix looks like a copy'n'paste issue that works by accident. :)

Want to fix it then?

Want to fix it then?

Yes, will open a different PR for this given that it's not a requirement for this one.

The build goes further now:

# LD      arch/x86/boot/setup.elf
  /nix/store/zb769plpdp2cj3yj43zp666qsyqyw4r2-x86_64-unknown-linux-gnu-clang-wrapper-18.1.8/bin/x86_64-unknown-linux-gnu-ld -m elf_x86_64 -z noexecstack  -m e
lf_i386 -z noexecstack -T ../arch/x86/boot/setup.ld arch/x86/boot/a20.o arch/x86/boot/bioscall.o arch/x86/boot/cmdline.o arch/x86/boot/copy.o arch/x86/boot/cp
u.o arch/x86/boot/cpuflags.o arch/x86/boot/cpucheck.o arch/x86/boot/early_serial_console.o arch/x86/boot/edd.o arch/x86/boot/header.o arch/x86/boot/main.o arc
h/x86/boot/memory.o arch/x86/boot/pm.o arch/x86/boot/pmjump.o arch/x86/boot/printf.o arch/x86/boot/regs.o arch/x86/boot/string.o arch/x86/boot/tty.o arch/x86/
boot/video.o arch/x86/boot/video-mode.o arch/x86/boot/version.o arch/x86/boot/video-vga.o arch/x86/boot/video-vesa.o arch/x86/boot/video-bios.o -o arch/x86/bo
ot/setup.elf
x86_64-unknown-linux-gnu-ld: error: ../arch/x86/boot/setup.ld:15: unable to move location counter (0x204) backward to 0x1ef for section '.bstext'
make[2]: *** [../arch/x86/boot/Makefile:93: arch/x86/boot/setup.elf] Error 1
make[1]: *** [../arch/x86/Makefile:302: bzImage] Error 2
make[1]: *** Waiting for unfinished jobs....

We're getting closer.

I've opened #345691 for the strip issue.

So I've narrowed down the build issue to the compiler/bintools wrappers. If I use the unwrapped toolchain for building (the last XXX commit), the build succeeds. I'm currently debugging what the wrappers do that causes this problem.

I wish there were a flag to turn the wrapper magic off for cases like these...

You have ran into #321667 if you're on x86_64-linux.

You have ran into #321667 if you're on x86_64-linux.

Thanks for the pointer! But why would the kernel need patching, if it builds fine with the unwrapped toolchain? It seems like we do something weird in our wrapping logic.

Thanks for the pointer! But why would the kernel need patching, if it builds fine with the unwrapped toolchain? It seems like we do something weird in our wrapping logic.

Our wrapper does invoke a behavior of LLVM which GNU ld doesn't have. GNU ld has some logic to not link with dynamic libraries when necessary. LLVM ld doesn't have this behavior. I did start some work in the wrapper to implement the same GNU ld behavior but I just haven't allocated enough time into this problem. I did however open an issue in LLVM itself to see about getting this behavior implemented.

I'm also wondering whether LD etc would be better set as environment variables in stdenv. Would you be interested in exploring that as followup? It would likely fix other packages with similar problems.

I'm testing this locally now, but I don't understand why we need absolute paths for anything apart from the linker on x86_64. A comment mentions PATH clobbering issues, but what path clobbering issues? Have we ever seen any?

The problem it refers to seems to be that buildPackages are in PATH, which messes up the autodetection of what tools to use in the Linux build system.

I'm also wondering whether LD etc would be better set as environment variables in stdenv. Would you be interested in exploring that as followup? It would likely fix other packages with similar problems.

I can look into this. I see a couple of packages that use "CC=${stdenv.cc.targetPrefix}cc". The problem is that packages also use export CC=$CC_FOR_BUILD. So with the cross-compilation issues this might not be a clear win.

The problem it refers to seems to be that buildPackages are in PATH, which messes up the autodetection of what tools to use in the Linux build system.

But they're not going to be autodetected if we're explicitly setting them to the prefixed versions?

I'm also wondering whether LD etc would be better set as environment variables in stdenv. Would you be interested in exploring that as followup? It would likely fix other packages with similar problems.

I can look into this. I see a couple of packages that use "CC=${stdenv.cc.targetPrefix}cc". The problem is that packages also use export CC=$CC_FOR_BUILD. So with the cross-compilation issues this might not be a clear win.

For CC this is already done by stdenv. I was just talking about e.g. LD, for consistency.

The problem it refers to seems to be that buildPackages are in PATH, which messes up the autodetection of what tools to use in the Linux build system.

But they're not going to be autodetected if we're explicitly setting them to the prefixed versions?

I looked into this and we don't have to set them at all! We only need to set CROSS_COMPILE to the right target prefix and everything works out correctly. This simplifies everything a lot. See the last patch.

I'm currently compiling some kernels, but so far it looks good. Testing is welcome!

@alyssais If this approach works out, I'll squash the commits.

@RossComputerGuy Can you also check how this version looks for you? If this works, it would remove quite a bit of clutter from the derivations.

@alyssais @RossComputerGuy Do you have any input how to move this forward?

Do you think it's ready? It looks fine to me with a squash.

I think it's fine.

Do you think it's ready? It looks fine to me with a squash.

I've rebased this and squashed the ld wrapper fixup. The other commits are reasonably self-sufficient. In case there are issues with 7ccb461e6f4243688418a573b4182516203339a2, it could also be reverted individually without breaking functionality.

I'm going to give this another nixpkgs-review round.

Result of nixpkgs-review pr 344665 run on x86_64-linux 1

The failed kernel modules seem unrelated. But maybe has a better overview of what's supposed to fail? What's our policy here? Does it make sense to mark them broken in another PR?

Yeah. It's just that it needs investigation of what conditions cause them to fail to build, so having them marked broken properly requires somebody to put in the time to do that investigation.

I'm going to look into the kernel modules:

• ch9344: 2.0 -> 0-unstable-2024-11-15 #354536
• linuxPackages.cryptodev: 1.13 -> 1.14 #359871
• linuxPackages.drbd: 9.2.8 -> 9.2.12 #360238 (or linuxPackages.drbd: 9.2.8 -> 9.2.9 #312011)
• linux_latest-libre: fix build #359464
• lkrg: 0.9.5 -> 0.9.9 #361465
• linuxPackages.xone: 0.3-unstable-2024-03-16 -> 0.3-unstable-2024-04-25 #361474
• shufflecake: 0.4.4 -> 0.5.1 #343906
• linuxPackages.vmm_clock: 0.2.0 -> 0.2.1 #361631
• linuxPackages.virtio_vmmci: 0.6.0 -> 0.6.1 #361728

(I'll update this comment as I find out more.)

Result of nixpkgs-review pr 344665 --checkout commit run on x86_64-linux 1