Skip to content

add build target support#120249

Merged
cdepillabout merged 3 commits intoNixOS:haskell-updatesfrom
Fresheyeball:haskell-target
May 2, 2021
Merged

add build target support#120249
cdepillabout merged 3 commits intoNixOS:haskell-updatesfrom
Fresheyeball:haskell-target

Conversation

@Fresheyeball
Copy link
Contributor

Motivation for this change

We may wish to build a single executable from a haskell package, instead of all executables in the package.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@github-actions github-actions bot added the 6.topic: haskell General-purpose, statically typed, purely functional programming language label Apr 22, 2021
@ofborg ofborg bot added 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Apr 22, 2021
@cdepillabout
Copy link
Member

/rebase haskell-updates

(we generally like to have Haskell related changes based on the haskell-updates branch. In particularly, this PR will rebuild all Haskell packages, so the haskell-updates branch is generally a safer choice.)

@github-actions
Copy link
Contributor

Failed to rebase

@cdepillabout
Copy link
Member

/rebase haskell-updates

@cdepillabout
Copy link
Member

Okay well I have no idea why this rebase action is not working.

@Fresheyeball could you rebase this onto the haskell-updates branch?

@github-actions github-actions bot changed the base branch from master to haskell-updates April 23, 2021 04:11
@github-actions github-actions bot closed this Apr 23, 2021
@github-actions
Copy link
Contributor

Rebased, please reopen the pull request to restart CI

@cdepillabout cdepillabout reopened this Apr 23, 2021
Copy link
Member

@cdepillabout cdepillabout left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, uhh, I guess the rebase action ended up working!

Just some other comments / questions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do install and copy do the exact same thing here? Is there no difference in the output derivation?

Would you be able to provide some proof that this doesn't actually break anything?

I'm imagining taking a bunch of popular Haskell packages (where isLibrary is set to false), compiling them both with this PR and without, and showing that the output derivations are identical.

I'm somewhat surprised that this would cause so many rebuilds, especially since it should only affect non-library Haskell executables. I thought we just didn't have too many of those in nixpkgs.

Copy link
Contributor Author

@Fresheyeball Fresheyeball Apr 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do install and copy do the exact same thing here? Is there no difference in the output derivation?

If there is a difference, I didn't find it.

Would you be able to provide some proof that this doesn't actually break anything?

I did this:

nix-build ./. -A haskellPackages.pandoc
./result/bin/pandoc --help

and it worked. Not sure what more to do here.

I'm imagining taking a bunch of popular Haskell packages (where isLibrary is set to false), compiling them both with this PR and without, and showing that the output derivations are identical.

With the new version I pushed, these libraries should be uneffected. In fact, I think that there are no packages currently that use the buildTarget parameter and have isLibrary == false because they would not build today anyway.

@sternenseemann
Copy link
Member

@ofborg build haskellPackages.git-annex haskellPackages.wstunnel haskellPackages.spacecookie

@sternenseemann
Copy link
Member

@ofborg eval

@Fresheyeball
Copy link
Contributor Author

@cdepillabout @sternenseemann I completely re-did the code. Now has comments and the requested tests as well. I also made a change to prevent so much of the ecosystem from needing recompilation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cdepillabout This line is what should preserve existing packages without rebuilds.

@cdepillabout
Copy link
Member

I rebased this on the current haskell-updates branch.

Originally, ofborg had run this against a branch that did not evaluate. This PR should now evaluate, so hopefully ofborg will update this with how many changes this will cause. Above, Fresheyeball was careful to write this in a way that should cause no changes.

@GrahamcOfBorg build tests.haskell-setBuildTarget

Copy link
Member

@cdepillabout cdepillabout Apr 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, sorry I didn't catch this earlier. callCabal2nix actually won't work here because it does IFD. This won't work on Hydra.

I think ofborg catches this, since evaluation still fails (https://gist.github.com/GrahamcOfBorg/4c5ca9d702aead273a0b8cedf6cb9fe1), but from the error message it is not clear what is going on.

@Fresheyeball could you rewrite this so it doesn't use IFD? Inlining the output from cabal2nix would probably be the easiest way to get this working.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message is an import failure of a store path, so that's precisely the ifd failure :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cdepillabout I admit I am dependant on callCabal2nix and don't know how to do what you ask.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

callCabal2nix just runs cabal2nix and uses imports the resulting .nix source file. You can just generate that file once by running the program manually.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Fresheyeball I added commit a77475b that shows what I was trying to say.

If you take a look at this and agree this looks alright, I think this should be okay to be merged in.

@cdepillabout
Copy link
Member

@GrahamcOfBorg eval

@GrahamcOfBorg build tests.haskell-setBuildTarget

@ofborg ofborg bot added 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. labels Apr 30, 2021
@ofborg ofborg bot removed 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Apr 30, 2021
@cdepillabout
Copy link
Member

Oh good, ofborg re-did the labels. Now it is clear this won't cause any rebuilds.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replacing buildTarget with a list parameter called buildTargets would be nice, but I'm not sure to what extent buildTarget may be considered public API?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yeah, that is a good idea. Although I'd argue that it shouldn't block this PR getting merged in as it currently as.

As for whether it would be considered a public API, I'm not sure. I've wondered about that as well. We have the functions in pkgs/development/haskell-modules/lib.nix that allow us to set the arguments in generic-builder.nix, but we don't have a function in pkgs/development/haskell-modules/lib.nix for every argument in generic-builder.nix, so sometimes you have to set them directly. It is also more convenient to set the arguments directly sometimes rather than going through the functions in pkgs/development/haskell-modules/lib.nix.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this hasn't been working properly until now, we probably can change stuff about it without to much problems, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see what you're saying. Since buildTarget hasn't actually been working correctly, then we don't really need to consider it part of the public API.

I think that makes sense, and it sounds like it makes sense to make that change.

Although I don't think it should necessarily stop this PR from being merged in.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has been working correctly for libraries

@cdepillabout
Copy link
Member

There was a merge conflict due to the tests slightly changing on the haskell-updates branch. I fixed this locally and rebased this PR on the most recent haskell-updates branch.

I'm going to go ahead and merge this in when ofborg finishes.

@sternenseemann if we'd like to change the buildtarget argument on the generic builder, lets do that in a future PR.

@Fresheyeball Thanks for getting this started and doing most of the work here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: haskell General-purpose, statically typed, purely functional programming language 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants