Skip to content

buildRustPackage: factor out setting up cargoBuildHook, add maturinBuildHook#112804

Merged
Mic92 merged 8 commits intoNixOS:stagingfrom
danieldk:cargo-build-hook
Feb 14, 2021
Merged

buildRustPackage: factor out setting up cargoBuildHook, add maturinBuildHook#112804
Mic92 merged 8 commits intoNixOS:stagingfrom
danieldk:cargo-build-hook

Conversation

@danieldk
Copy link
Contributor

@danieldk danieldk commented Feb 11, 2021 •
edited
Loading

Motivation for this change

Convert the build phase of buildRustPackage into the cargoBuildHook hook. Also add a maturinBuildHook and convert the derivations that use maturin to buildPythonPackage + cargoSetupHook + maturinBuildHook.

Additional comments:

  • buildRustPackage has an API change: remove the target argument of buildRustPackage, the target should always be in sync with the C/C++ compiler that is used.
  • Gathering of binaries has moved from buildPhase to installPhase, this simplifies the hook and orders this functionality logically with the installation logic.

Todo:

  • Add documentation (will keep this a draft request until this is done).
  • Read the changes once more to see if I missed something.

Related issues:

CC @FRidh @Mic92 @adisbladis @DavHau

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@danieldk
buildRustPackage: factor out build phase to cargoBuildHook
a8efb20 
- API change: remove the `target` argument of `buildRustPackage`, the
  target should always be in sync with the C/C++ compiler that is used.

- Gathering of binaries has moved from `buildPhase` to `installPhase`,
  this simplifies the hook and orders this functionality logically
  with the installation logic.
@danieldk
fetchCargoTarball: set default sourceRoot to the empty string
7876d1c 
This avoids that non-buildRustPackage derivations need to specify
sourceRoot when the fetcher performs root stripping.
@ofborg ofborg bot added 6.topic: python Python is a high-level, general-purpose programming language. 6.topic: rust General-purpose programming language emphasizing performance, type safety, and concurrency. labels Feb 11, 2021
@FRidh FRidh mentioned this pull request Feb 11, 2021
Merged
10 tasks
FRidh
FRidh reviewed Feb 11, 2021
View reviewed changes
Copy link
Member

@FRidh FRidh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

@ofborg ofborg bot added 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Feb 11, 2021
@danieldk danieldk marked this pull request as ready for review February 12, 2021 07:37
@danieldk
Copy link
Contributor Author

danieldk commented Feb 12, 2021 •
edited
Loading

Added documentation and lifted the Draft PR toggle.

@ofborg ofborg bot added the 8.has: documentation This PR adds or changes documentation label Feb 12, 2021
jonringer
jonringer approved these changes Feb 12, 2021
View reviewed changes
Copy link
Contributor

@jonringer jonringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

diff LGTM

@danieldk
Copy link
Contributor Author

danieldk commented Feb 13, 2021

@Mic92 is this ok for you to merge?

@Mic92
Copy link
Member

Mic92 commented Feb 13, 2021

Result of nixpkgs-review pr 112804 run on x86_64-linux 1

3 packages built:
  • python3Packages.johnnycanencrypt
  • python3Packages.retworkx
  • python3Packages.wasmer

@Mic92 Mic92 merged commit b5b47d6 into NixOS:staging Feb 14, 2021
@danieldk danieldk deleted the cargo-build-hook branch February 14, 2021 19:50
@danieldk
Copy link
Contributor Author

danieldk commented Feb 14, 2021

I just noticed that I made an error -- finding libraries and executables must be done before checkPhase. But I'll do a PR for cargoInstallHook tomorrow, which also fixes this issue.

@FRidh FRidh mentioned this pull request Feb 15, 2021
Merged
10 tasks
@collares
Copy link
Member

collares commented Feb 28, 2021 •
edited
Loading

Is there a chance that this broke gnome-tour on master? When built via meson/ninja, it compiles resources (using gnome.compile_resources) before starting rust compilation (here's a link to the relevant part of src/meson.build), which then includes the generated file into a rust source using a macro (include_bytes!). On master, cargoBuildHook runs and builds the project before the resources are generated (i.e. ignoring the dependency), leading to a compilation failure. I believe this failure is currently a channel blocker.

Useful links:

cc @worldofpeace

@collares collares mentioned this pull request Feb 28, 2021
Closed
@vcunat
Copy link
Member

vcunat commented Feb 28, 2021

In the meantime I really bisected the regression to the first commit of this PR (without understanding it). A catch is that you need a later change

--- a/pkgs/development/libraries/webkitgtk/default.nix
+++ b/pkgs/development/libraries/webkitgtk/default.nix
@@ -155,6 +155,9 @@ stdenv.mkDerivation rec {
     "-DPORT=GTK"
     "-DUSE_LIBHYPHEN=OFF"
     "-DUSE_WPE_RENDERER=OFF"
+    # ensure backward compatibility with the latest version of icu:
+    # http://linuxfromscratch.org/blfs/view/svn/x/webkitgtk.html
+    "-DCMAKE_CXX_FLAGS=-DU_DEFINE_FALSE_AND_TRUE=1"
   ] ++ optionals stdenv.isDarwin [
     "-DENABLE_GRAPHICS_CONTEXT_3D=OFF"
     "-DENABLE_GTKDOC=OFF"

to fix the build of the webkit dependency (and some of these take a long time to build, too).

@vcunat
Copy link
Member

vcunat commented Feb 28, 2021

Yes, it landed at once with the breakage during the last staging-next merge. And now gnome-tour is fixed as well 🎉

@minijackson minijackson mentioned this pull request Mar 6, 2021
Merged
10 tasks
@dotlambda dotlambda mentioned this pull request Mar 9, 2021
Merged
10 tasks
baloo added a commit to baloo/nixpkgs that referenced this pull request Oct 19, 2021
@baloo
doc: rust: target escape hatch has been removed
c1a440b 
As far as I can tell, a8efb20 removed
the `target =` escape hatch.
See NixOS#112804

This commit removes it from the documentation.

Signed-off-by: Arthur Gautier <[email protected]>
@baloo baloo mentioned this pull request Oct 19, 2021
Merged
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@0x4A6F 0x4A6F 0x4A6F approved these changes

@drewrisinger drewrisinger Awaiting requested review from drewrisinger

@SuperSandro2000 SuperSandro2000 Awaiting requested review from SuperSandro2000

@andir andir Awaiting requested review from andir

@LnL7 LnL7 Awaiting requested review from LnL7

@Mic92 Mic92 Awaiting requested review from Mic92

@zowoq zowoq Awaiting requested review from zowoq

+2 more reviewers

@FRidh FRidh FRidh left review comments

@jonringer jonringer jonringer approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

6.topic: python Python is a high-level, general-purpose programming language. 6.topic: rust General-purpose programming language emphasizing performance, type safety, and concurrency. 8.has: documentation This PR adds or changes documentation 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@danieldk @Mic92 @collares @vcunat @FRidh @jonringer @0x4A6F @nixos-discourse