[WIP] mingw-w64 for haskell cross compilation with ghc. by angerman · Pull Request #37254 · NixOS/nixpkgs

angerman · 2018-03-17T15:41:39Z

Motivation for this change

This includes all the necessary changes to nixpkgs, that allow me to cross compile haskell to
windows via mingw-w64. It is not yet sufficient to allow cross compilation of Template Haskell.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

angerman · 2018-03-17T15:42:55Z

/cc @Ericson2314

angerman · 2018-03-17T15:44:13Z

pkgs/development/compilers/ghc/8.2.1-binary.nix

While disabling this check, allows most of the cross compilation to succeed on macOS, I believe this is also the reason why we end up with mixed (build and target) packages in the stdenv eventually.

angerman · 2018-03-17T15:45:57Z

pkgs/development/compilers/ghc/head.nix

This change is motivated by the fact that we do not necessarily build haddock with quick flavours. As such we fall over trying to paxmark a file that does not exist.

Sounds good-- probably can collapse this to just {ghc,haddock} if we check for existence anyway.

angerman · 2018-03-17T15:46:37Z

pkgs/development/haskell-modules/generic-builder.nix

--via-asm is a rather recent feature in hsc2hs that is part of ghc > 8.4

angerman · 2018-03-17T15:47:28Z

pkgs/os-specific/darwin/cctools/port.nix

This could potentially also be the reason for the incorrect package set on darwin.

Ericson2314 · 2018-03-20T18:46:46Z

pkgs/development/compilers/ghc/head.nix

Better to do ghcFlavour ? if targetPlatform != hostPlatform then "perf-cros" else ""

dtzWill · 2018-03-22T11:33:33Z

pkgs/development/compilers/ghc/head.nix

Does terminfo/ncurses not work on windows?

If it was failing to find ncurses headers/libs see the fix on my musl PR (remove attempt to specify where to find them).

In the case of cross compiling to windows, we don't want terminfo, the GHC build system disables terminfoin that case anyway.

It only disables it in some cross build "flavour"s, which we don't have to use as-is. Works for me building cross-aarch64-musl and such. Anyway sounds like you're saying windows in particular shouldn't use this, which I'll take your word for.

dtzWill · 2018-03-22T11:35:34Z

pkgs/development/haskell-modules/generic-builder.nix

dtzWill · 2018-03-22T14:48:18Z

If you're touching cctools, #37606 might have some useful bits.

peti · 2018-03-23T08:46:41Z

pkgs/development/compilers/ghc/head.nix

I believe that the description of this option is actually quite incorrect and misleading. Setting this to true has no effect on whether ghc builds shared libraries or not. Rather, it affects the code that's used by ghci to load compiled objects for interpretation. There's a good chance that this setting is completely ignored on Windows anyway. Could you please check whether the change you've made here is actually necessary?

enable* is about how the current package is linked. I think this does just that. That this also affects how GHCi works is just a side affect due to GHCi's fragile interpretation.

dtzWill · 2018-04-21T13:01:42Z

Beep boop, how's this going?

angerman · 2018-04-21T14:06:09Z

@dtzWill this does work as-is (I've used this to build dhall-json among others...). I did however get severely frustrated with the conditionals missing in cabal2nix generated cabal expressions. I did look into making cabal2nix generate better expressions, but learned that what I wanted was a bad fit for what cabal2nix does. And I don't want to disrupt any working solution in nix.

So I ended up writing Cabal2Nix, which really needs a better name. I now have hackage.nix, stackage.nix as well as some glue code in haskell.nix. This is pretty much work-in-progress. While it does allow me to build some trivial hs-hello package, it is nowhere near polished yet, and will likely see some additional changes.

I believe it will provide a much better basis for cross compilation haskell packages.

Ericson2314 · 2018-04-22T00:43:31Z

@angerman that sounds fascinating but I think your links are broken?

angerman · 2018-04-22T06:54:00Z

@Ericson2314 I see... Here are the full links:

... thanks GitHub.

kirelagin · 2018-04-29T21:51:04Z

pkgs/development/haskell-modules/setup.nix

Here propagated setup dependencies are not taken into account. That is, your pkgdb will contain only immediate setup dependencies but their transitive dependencies will not be in it, so it won’t compile.

kirelagin · 2018-04-29T21:54:23Z

pkgs/development/haskell-modules/generic-builder.nix

inherit setupHaskellDepends, I guess.

kirelagin · 2018-04-29T22:03:28Z

pkgs/development/haskell-modules/setup.nix

I think ghc should go to nativeBuildInputs.

For cross compilation, we don't need it, it's deferred to the Setup derivation. When ghc is th enativeGhc, we don't need to add it to the buildInputs either.

nixpkgs#37012 and nixpkgs#37707 introduces the setup-hooks for libiconv, which inject `-liconv` into the `NIX_LDFLAGS`. This breaks horribly on windows where the linker end up having no idea how to linke `-liconv`. The configure.ac file specifically ignores libiconv on windows.

This reverts commit 7f86470.

Nix cc logic relies on passing /lib for each dependency as a library search path. This makes little sense for haskell packages, as their libs are nested deeply in the /lib tree. Also GHC will generate the right library serach paths on its own from the package database entries. The larger issue though is that with one search path entry for each library, and a similar one generated by GHC for each dependency will eventually overflow the command line argument limits. As such we do *not* put haskell libraries into $out anymore.

angerman requested a review from peti as a code owner March 17, 2018 15:41

angerman commented Mar 17, 2018

View reviewed changes

bjornfor added the 6.topic: cross-compilation Building packages on a different platform than they will be used on label Mar 19, 2018

Ericson2314 reviewed Mar 20, 2018

View reviewed changes

angerman force-pushed the feature/ghc-cross-mingw branch from c8f682e to de9b33e Compare March 21, 2018 23:36

angerman requested a review from FRidh as a code owner March 21, 2018 23:36

GrahamcOfBorg added 6.topic: python Python is a high-level, general-purpose programming language. 8.has: package (new) This PR adds a new package labels Mar 21, 2018

angerman mentioned this pull request Mar 21, 2018

ghc-{8.2.2,8.4.2}: fixups mostly for 2-stage cross, musl #37598

Merged

8 tasks

dtzWill reviewed Mar 22, 2018

View reviewed changes

matthewbauer added the 2.status: work-in-progress label Mar 22, 2018

peti requested changes Mar 23, 2018

View reviewed changes

bgamari mentioned this pull request Apr 24, 2018

Cross-compilation of Haskell packages with custom setup program #39417

Closed

kirelagin mentioned this pull request Apr 28, 2018

haskell: Setup.hs should not be compiled with the same pkgdb as the package itself #39646

Closed

kirelagin suggested changes Apr 29, 2018

View reviewed changes

kirelagin reviewed Apr 29, 2018

View reviewed changes

pkgs/development/haskell-modules/generic-builder.nix Outdated

Copy link

Member

kirelagin Apr 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inherit setupHaskellDepends, I guess.

kirelagin reviewed Apr 29, 2018

View reviewed changes

angerman mentioned this pull request May 1, 2018

haskell: build Setup.hs with a separate pkgdb #39735

Closed

angerman and others added 26 commits May 16, 2018 16:42

Adds custom Setup derivation.

6c7fb30

use/provide setup in package sets.

2a8ded3

use Setup derivation when cross compiling.

56d28ed

do not add nativeGhc to the buildInputs

dcb0cb8

For cross compilation, we don't need it, it's deferred to the Setup derivation. When ghc is th enativeGhc, we don't need to add it to the buildInputs either.

no -staticlib for windows.

fcb8be5

Adds more asserts

cd79349

more logging.

2276ff8

Revert incorrect naming.

55f320f

Use buildpackages for nix and nix-prefetch-scripts

5666ec8

used stdenv.targetPlatform.isDarwin and not stdenv.isDarwin.

a302505

fix python27

093afd8

WIP

ca6cf4a

fix lz4 cross-build

a7f9517

make rocksdb use cmake (doesnt work)

3443dd3

fix libuv

45e4258

rocksdb (don't fully work yet...)

f4f044f

proper mingw names

3c032a1

revert rocksdb

7f86470

cleanup mingw

2cce00a

Adds preliminary gcc-ng infra.

728ada5

Adds wine (darwin)

cd3b3de

make bulding libgcc work.

1353a6e

Revert "revert rocksdb"

b1adfd1

This reverts commit 7f86470.

use gcc7-ng-msvcrt by default.

3c044d2

bump

68d2d15

angerman force-pushed the feature/ghc-cross-mingw branch from 98cff92 to 68d2d15 Compare June 8, 2018 15:46

angerman requested a review from nbp as a code owner June 8, 2018 15:46

angerman closed this Jul 27, 2018

Uh oh!

Comments

Conversation

angerman commented Mar 17, 2018

Motivation for this change

Things done

Uh oh!

angerman commented Mar 17, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dtzWill Mar 22, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dtzWill commented Mar 22, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dtzWill commented Apr 21, 2018

Uh oh!

angerman commented Apr 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Ericson2314 commented Apr 22, 2018

Uh oh!

angerman commented Apr 22, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

dtzWill Mar 22, 2018 •

edited

Loading

angerman commented Apr 21, 2018 •

edited

Loading