Skip to content

perlPackages.ModuleBuild and .FileBaseDir: fix cross-compilation WIP#222427

Closed
ck3d wants to merge 2 commits intoNixOS:stagingfrom
ck3d:fix-cross-perl-module-build
Closed

perlPackages.ModuleBuild and .FileBaseDir: fix cross-compilation WIP#222427
ck3d wants to merge 2 commits intoNixOS:stagingfrom
ck3d:fix-cross-perl-module-build

Conversation

@ck3d
Copy link
Contributor

@ck3d ck3d commented Mar 21, 2023 •
edited
Loading

Description of changes

This PR fixes #66741, the cross compilation for Perl module ModuleBuild and FileBaseDir. Following command will succeed:

nix-build -A pkgsCross.aarch64-multiplatform.perlPackages.ModuleBuild
nix-build -A pkgsCross.aarch64-multiplatform.perlPackages.FileBaseDir

A cross compiled Perl module gets pkgs.perl.mini passed instead of pkgs.perl. The mini do not allow dynamic loading, which causes errors during build.

definition of perl.mini:
https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/interpreters/perl/default.nix#L34

usage of perl.mini:
https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/perl-modules/generic/default.nix#L46

Following issues suffer from the same problem and can hopefully fixed in a similar way:

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.05 Release Notes (or backporting 22.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@ck3d ck3d requested review from dasJ, stigtsp and zakame as code owners March 21, 2023 22:16
@ofborg ofborg bot added 6.topic: cross-compilation Building packages on a different platform than they will be used on 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Mar 21, 2023
@Mindavi Mindavi requested a review from Artturin March 26, 2023 09:39
NickCao
NickCao reviewed Mar 30, 2023
View reviewed changes
pkgs/top-level/perl-packages.nix Outdated
Comment on lines 15237 to 15238
Copy link
Member

@NickCao NickCao Mar 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use addToSearchPath PATH "$fullperl/bin"

Copy link
Contributor Author

@ck3d ck3d Mar 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your proposal doesn't work. The function addToSearchPath adds dir at the end off and not at the beginning.

@NickCao NickCao mentioned this pull request Apr 7, 2023
Merged
@ghost
Copy link

ghost commented Apr 7, 2023

@ofborg build pkgsCross.aarch64-multiplatform.xdg-utils

@ghost
Copy link

ghost commented Apr 7, 2023

No luck

@ck3d ck3d changed the title perlPackages.ModuleBuild: fix cross-compilation perlPackages.ModuleBuild and .FileBaseDir: fix cross-compilation Apr 8, 2023
@ck3d
Copy link
Contributor Author

ck3d commented Apr 8, 2023

I added the same patch to FileBaseDir. You can now cross-compile xdg-utils

@NickCao
Copy link
Member

NickCao commented Apr 8, 2023

Basically the same fix, but in another approach, is being carried out in #225086, may you also take a look?

@ck3d
Copy link
Contributor Author

ck3d commented Apr 8, 2023

Thanks, looks better than my approach. I will update this PR.

@ck3d ck3d force-pushed the fix-cross-perl-module-build branch from 5d96bb2 to 8795962 Compare April 8, 2023 08:12
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. and removed 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Apr 8, 2023
@apfelkuchen6 apfelkuchen6 mentioned this pull request Apr 8, 2023
Closed
12 tasks
apfelkuchen6
apfelkuchen6 approved these changes Apr 8, 2023
View reviewed changes
Copy link
Contributor

@apfelkuchen6 apfelkuchen6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Since this causes 0 rebuilds, this can go to master directly.

@Artturin
Copy link
Member

Artturin commented Apr 8, 2023

Isn't the reason why we use miniperl is that the normal perl doesn't cross compile stuff?

@apfelkuchen6
Copy link
Contributor

apfelkuchen6 commented Apr 8, 2023 •
edited
Loading

Oh, shit:

➜ find /nix/store/l5dn2khpgsf74ai5kjr0qmc3xwdx3n8f-perl5.36.0-Module-Build-0.4231-aarch64-unknown-linux-gnu      
[...]
/nix/store/l5dn2khpgsf74ai5kjr0qmc3xwdx3n8f-perl5.36.0-Module-Build-0.4231-aarch64-unknown-linux-gnu/lib/perl5/site_perl/5.36.0/Module/Build/PodParser.pm
/nix/store/l5dn2khpgsf74ai5kjr0qmc3xwdx3n8f-perl5.36.0-Module-Build-0.4231-aarch64-unknown-linux-gnu/lib/perl5/site_perl/5.36.0/x86_64-linux-thread-multi
/nix/store/l5dn2khpgsf74ai5kjr0qmc3xwdx3n8f-perl5.36.0-Module-Build-0.4231-aarch64-unknown-linux-gnu/lib/perl5/site_perl/5.36.0/x86_64-linux-thread-multi/auto
/nix/store/l5dn2khpgsf74ai5kjr0qmc3xwdx3n8f-perl5.36.0-Module-Build-0.4231-aarch64-unknown-linux-gnu/lib/perl5/site_perl/5.36.0/x86_64-linux-thread-multi/auto/Module
/nix/store/l5dn2khpgsf74ai5kjr0qmc3xwdx3n8f-perl5.36.0-Module-Build-0.4231-aarch64-unknown-linux-gnu/lib/perl5/site_perl/5.36.0/x86_64-linux-thread-multi/auto/Module/Build
/nix/store/l5dn2khpgsf74ai5kjr0qmc3xwdx3n8f-perl5.36.0-Module-Build-0.4231-aarch64-unknown-linux-gnu/lib/perl5/site_perl/5.36.0/x86_64-linux-thread-multi/auto/Module/Build/.packlist

In this case, there are no binaries in there and the result probably runs on aarch64 (I'll test), but this approach will totally break for things that build perl modules.

Edit: the "cross-compiled" File::BaseDir (non-native) perl module successfully loads on aarch64.

@ck3d
Copy link
Contributor Author

ck3d commented Apr 8, 2023

The differences between perl and perl.mini in case of cross compilation are:

  1. Scrips in bin/ point to the wrong perl interpreter. These scripts will not run on the target.
  2. Target specific module directory is wrong. The directory name should be aarch64-linux/, but it is named x86_64-linux-thread-multi/.

I will mark this PR as WIP as long we did not fix these issues.

@ck3d ck3d changed the title perlPackages.ModuleBuild and .FileBaseDir: fix cross-compilation perlPackages.ModuleBuild and .FileBaseDir: fix cross-compilation WIP Apr 8, 2023
@ck3d ck3d marked this pull request as draft April 8, 2023 19:53
@ck3d
Copy link
Contributor Author

ck3d commented Apr 10, 2023

I close this PR in favor of #225640

@ck3d ck3d closed this Apr 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@NickCao NickCao NickCao left review comments

@stigtsp stigtsp Awaiting requested review from stigtsp

@zakame zakame Awaiting requested review from zakame

@dasJ dasJ Awaiting requested review from dasJ

@Artturin Artturin Awaiting requested review from Artturin

+1 more reviewer

@apfelkuchen6 apfelkuchen6 apfelkuchen6 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

6.topic: cross-compilation Building packages on a different platform than they will be used on 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ck3d @NickCao @Artturin @apfelkuchen6