Skip to content

ghc cross-compilation and ghc-musl: notes, issues encountered, progress #37522

@dtzWill

Description

@dtzWill

Notes

Some of this, I just discovered, repeats items from #36200 and its discussion, but including for now while brain-dump things :).

Apologies that at this hour I'm mostly full of issues and not notes.

For now, checkmarks are issues tackled in my branch/efforts, not neccesarily upstream yet.

Issues

  • AFAICT specifying BuildFlavour in mk/build.mk does nothing by default, in mk/build.mk.sample (for versions I inspected, and I suspect all) there's code that goes "if buildFlavour != "" then include ./mk/flavours/${buildFlavour}.mk`. I'm not sure what "flavour" we want, this seems like something worth checking out--cross and non-cross-- to ensure we're doing what we want.
  • CXX is wrong--there is nothing with the -cxx suffix, it should be c++
  • dangling symlinks: gentoo has some patches regarding this, particularly this one. This behavior can be seen by inspecting a path such as /nix/store/4yjvzsnk906bm61sk2037bvwbcnlsydg-aarch64-unknown-linux-musl-ghc-8.2.2 (available via cache.nixos.org).
  • bundled terminfo is very special (1, 2, 3) and as-is needs ncurses (and host's gcc??)-- which we aren't currently providing AFAICT. This is hopefully easy, although I'm a bit unclear what inputs are from (ghc's "confused" terminology vs our own certainly doesn't help, heh) where :) so it's taking a number of iterations to sort this out.

Minor?

  • paxmark in postInstall assumes installation will be in $out/lib/${targetPrefix}-ghc-${version} (or so), this is not the case unless ghc "thinks" it is cross-compiling. Which it doesn't when cross-compiling to musl O:). D'oh to hit this after building all the rest, hehe.
  • Related to the above: At least in 8.2.2 (version I'm looking at the most since it's currently our default) terminfo dropped the --with-curses-includes flag, because of reasons. Anyway only use of setting this is to add ncurses.dev as a build dep of some sort-- which if that is all we want should probably be accomplished more directly.
  • GHC seems to prefer if we specified tools with --with-as=/path/to/as instead of setting AS, but I'm not sure.
  • Perhaps cross-GHC should be built statically? This seems to be what the "flavour" files for cross specify.
  • The resulting ghc is ... well, host is glibc :). This seems to be expected, but means this is mostly useful for cross-compiling things to musl not being used on musl. Is there a nixpkgs way to cross-compile GHC instead of creating a GHC cross-compiler? (shift a layer?)
  • libffi is parameter to expression but unused-- might be best to use system libffi?

Good news / Progress

Aarch64-musl works?!

As linked above.. apparently GHC for aarch64-musl has been working for a while now?!
Forgive the lack of styling, but here's cross-trunk filtered on "musl": https://hydra.nixos.org/jobset/nixpkgs/cross-trunk/jobs-tab?filter=musl

Where you can see 8.2.2 and HEAD are building!

A copy of "hello" built against musl for aarch64 can be obtained and inspected (it's a cross-trunk job) right now:

$  nix-store -r /nix/store/lcw90icbaakys5c1vi4ixvjsyriycxk1-hello-1.0.0.2-aarch64-unknown-linux-musl/bin/hello

You can even run it with qemu-aarch64! 😁

armv6l

Similarly: /nix/store/83w221rd46wa08ik5gx7g8nw739rmflg-hello-1.0.0.2-armv6l-unknown-linux-musleabihf/bin/hello is in the nix cache and seems to work via qemu-arm!

x86_64

I believe I have a somewhat working version, currently iterating on cleanup and investigating/resolving some of the issues mentioned above. Not entirely sure I didn't manage to just build a broken glibc-based GHC though, haha, will report as I work on this further.

EDIT: Yep, works! Largest outstanding problem is that GHC doesn't treat this as a cross-compile, and so doesn't prefix tools. This is a problem because our plumbing expects tools to be prefixed if host != target. For now just punting and manually creating links to prefixed versions to get things going :).

My dev branch (forgive any rebase/force-pushes please) is here: https://github.com/dtzWill/nixpkgs/tree/fix/ghc-cross-musl

EDIT2: https://github.com/dtzWill/nixpkgs/releases/tag/ghc-cross-musl-works works for x86_64? 😁

Metadata

Metadata

Assignees

No one assigned

    Labels

    6.topic: muslRunning or building packages with musl libc

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions