chromium: cross compilation support · Pull Request #229265 · NixOS/nixpkgs

ghost · 2023-05-01T10:05:13Z

Description of changes

This commit successfully cross-compiles chromium from buildPlatform=x86_64-linux to hostPlatform=aarch64-linux.

Other PRs included in this one (can be reviewed/merged independently):

Things done

Built on platform(s)
- x86_64-linux
- aarch64-linux (cross from x86_64-linux)
Tested basic functionality of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

pkgs/applications/audio/espeak-ng/default.nix

pkgs/applications/audio/espeak/default.nix

Artturin · 2023-05-01T21:13:04Z

separated chromium: set TERM=dumb for ninja & chromium: do not skip {pre,post}Build #229367

ghost · 2023-05-01T21:16:38Z

$ nix-instantiate . -A pkgsCross.aarch64-multiplatform.ungoogled-chromium
/nix/store/va4rvfk8i5q7pgl9g7l0wph9s5aj3lfc-ungoogled-chromium-aarch64-unknown-linux-gnu-112.0.5615.165.drv
$ nix-build /nix/store/va4rvfk8i5q7pgl9g7l0wph9s5aj3lfc-ungoogled-chromium-aarch64-unknown-linux-gnu-112.0.5615.165.drv
/nix/store/w2waiscdk40d07cd3f5x0hjca32j65gz-ungoogled-chromium-aarch64-unknown-linux-gnu-112.0.5615.165

@ofborg build pkgsCross.aarch64-multiplatform.ungoogled-chromium

xdg-utils does not cross-compile. Let's late-bind (using the runtime $PATH) in this scenario so a native-built xdg-utils can be used instead.

`llvmPackages.clang` is used in the `gnFlags` attrset, so we need to indicate explicitly which "on" platform (the build) and which "for" platform (the host) we want. This commit does that.

ungoogled-chromium is, contrary to its name, not a build of chromium. It is a patched copy of chromium's *source code*. Therefore, it needs to come from pkgsBuildBuild, because it contains python scripts which get /nix/store/.../bin/python3 patched into their shebangs.

Sometimes we access `llvmPackages` via `pkgs`, and other times via `pkgsFooBar`, so unfortunately a string (attrname) is the only way to have a single point of control over the LLVM version used for both buildPlatform and hostPlatform.

This commit successfully cross-compiles chromium from buildPlatform=x86_64-linux to hostPlatform=aarch64-linux. Other PRs in this project (can be reviewed/merged independently of this one): - #227707 - #227708 - #227710 - #227719 - #227720 - #227722 - #227723

ghost · 2023-07-12T19:38:45Z

Rebased.

michaeladler

Looks like quite the ordeal. Nice job! If it builds, then it's fine with me :)

pkgs/applications/networking/browsers/chromium/common.nix

Artturin · 2023-07-13T22:09:57Z

pkgs/applications/networking/browsers/chromium/common.nix

That's weird, Why is it necessary

After investigating more closely I think the comment is incorrect.

I think pkgsBuildBuild.libpng is needed for something Chromium is doing at build time, not because of anything related to splicing. I'm verifying this and will remove the comment (or the entire line) once I do.

I'm saying ++ buildInputs is weird

Chromium compiles a ton of stuff for both the hostPlatform and the buildPlatform (which, confusingly, it calls "host"... argh). For example, in order to dump out a "quick-start image" for the javascript VM it builds an entire buildPlatform version of it!

I could slog through every last dependency and figure out exactly which ones are/aren't needed on the host, but then that list would need to be maintained separately. It's definitely more than half of them.

See ea7c3166548919ba63c204f47598e8522de1c2d3

ghost · 2023-07-18T20:28:11Z

Rebased.

Artturin · 2023-08-03T22:29:35Z

Very messy but I guess that's just how browsers are...

Artturin · 2023-08-03T22:33:36Z

Merge after someone confirms that there's no build time deps included in the cross closure
nix path-info -rsh ./result | rg --invert-match "aarch"

Artturin · 2023-08-04T03:04:16Z

There's no for build deps in the output

It took 3 hours to build chromium...

ghost · 2023-08-05T05:33:06Z

There's no for build deps in the output

Should I add a disallowedReferences to ensure that none leak into the output in the future?

Artturin · 2023-08-05T05:42:21Z

There's no for build deps in the output

Should I add a disallowedReferences to ensure that none leak into the output in the future?

Sure, check how I did it in systemd.

emilylange · 2023-10-19T00:59:58Z

This increased the build steps and build time for non cross-compilation (e.g. chromium.browser) by a lot.
We are talking ~54000 vs ~85000 (and lately ~95000, showing a trend that suggests it gets even worse with each release).

It almost feels like we are building certain things twice.

Would you mind looking into this again @amjoseph-nixpkgs?

Edit: Wait. Are we cross-compiling to the same architecture if not invoked by pkgsCross?

Artturin · 2023-10-29T00:27:23Z

pkgs/applications/networking/browsers/chromium/common.nix

+      target_cpu    = chromiumRosettaStone.cpu stdenv.hostPlatform;
+      v8_target_cpu = chromiumRosettaStone.cpu stdenv.hostPlatform;
+      target_os     = chromiumRosettaStone.os  stdenv.hostPlatform;


Setting targets may be the cause of always cross-compiling

https://github.com/void-linux/void-packages/blob/7b23d90a5ea77b8d91f12c818d620e90eb710b3c/srcpkgs/chromium/template#L250-L281

Possible improvement:

chromium: do not set snapshot toolchain for native builds #264326

ghost · 2023-10-30T10:02:31Z

Edit: Wait. Are we cross-compiling to the same architecture if not invoked by pkgsCross?

Chromium has its own totally custom build system, and it's a gigantic pain in the ass. That's why chromium (and &(^&% QT) were among the very last parts of nixpkgs to support cross-compiling.

ofborg bot added the 6.topic: cross-compilation Building packages on a different platform than they will be used on label May 1, 2023

ghost requested a review from Artturin May 1, 2023 11:36

ofborg bot added the 8.has: clean-up This PR removes packages or removes other cruft label May 1, 2023

ofborg bot requested review from michaeladler, primeos, squalus and thefloweringash May 1, 2023 12:17

ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels May 1, 2023

ghost marked this pull request as ready for review May 1, 2023 12:52

Artturin reviewed May 1, 2023

View reviewed changes

pkgs/applications/audio/espeak-ng/default.nix Outdated Show resolved Hide resolved

Artturin reviewed May 1, 2023

View reviewed changes

pkgs/applications/audio/espeak/default.nix Outdated Show resolved Hide resolved

ofborg bot requested a review from networkException May 31, 2023 05:34

ofborg bot added 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. and removed 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels May 31, 2023

Adam Joseph added 6 commits July 5, 2023 03:32

xdg-utils: mark broken if cross

84ada07

chromium: late-bind xdg-utils if broken

758bf4c

xdg-utils does not cross-compile. Let's late-bind (using the runtime $PATH) in this scenario so a native-built xdg-utils can be used instead.

chromium: take llvmPackages from pkgsBuildTarget

c25897c

`llvmPackages.clang` is used in the `gnFlags` attrset, so we need to indicate explicitly which "on" platform (the build) and which "for" platform (the host) we want. This commit does that.

michaeladler approved these changes Jul 13, 2023

View reviewed changes

Artturin reviewed Jul 13, 2023

View reviewed changes

pkgs/applications/networking/browsers/chromium/common.nix Outdated Show resolved Hide resolved

Artturin reviewed Jul 13, 2023

View reviewed changes

ofborg bot requested a review from michaeladler July 14, 2023 06:04

ghost requested a review from lorenz July 18, 2023 20:28

ghost mentioned this pull request Jul 18, 2023

Init: chromium darwin #244152

Closed

12 tasks

Artturin merged commit 4055d18 into NixOS:master Aug 4, 2023

ghost deleted the pr/chromium/cross/all branch August 5, 2023 05:32

This was referenced Aug 5, 2023

chromium: (cross) invoke ungoogled-chromium via buildPackages #227710

Closed

chromium: (cross) take llvmPackages from pkgsBuildTarget #227719

Closed

Artturin reviewed Oct 29, 2023

View reviewed changes

ghost mentioned this pull request Oct 30, 2023

chromium: do not set snapshot toolchain for native builds #264326

Closed

13 tasks

emilylange mentioned this pull request Dec 10, 2023

chromium: fix increased build time for non-cross-compilation builds, add ungoogled- prefix to chromium-unwrapped, move stray patches to right location #273435

Merged

13 tasks

Artturin mentioned this pull request Jan 29, 2024

List of @amjoseph-nixpkgs Pull requests before account removal #284796

Open

emilylange mentioned this pull request Jun 10, 2024

[staging-next] electron: fix build #318857

Merged

13 tasks

Uh oh!

Conversation

ghost commented May 1, 2023 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Things done

Uh oh!

Uh oh!

Uh oh!

Artturin commented May 1, 2023

Uh oh!

ghost commented May 1, 2023

Uh oh!

ghost commented Jul 12, 2023

Uh oh!

michaeladler left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Artturin Jul 13, 2023

Choose a reason for hiding this comment

Uh oh!

ghost Jul 14, 2023

Choose a reason for hiding this comment

Uh oh!

Artturin Jul 14, 2023

Choose a reason for hiding this comment

Uh oh!

ghost Jul 14, 2023

Choose a reason for hiding this comment

Uh oh!

ghost Jul 14, 2023

Choose a reason for hiding this comment

Uh oh!

ghost commented Jul 18, 2023

Uh oh!

Artturin commented Aug 3, 2023

Uh oh!

Artturin commented Aug 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Artturin commented Aug 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Aug 5, 2023

Uh oh!

Artturin commented Aug 5, 2023

Uh oh!

emilylange commented Oct 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Artturin Oct 29, 2023

Choose a reason for hiding this comment

Uh oh!

ghost Oct 30, 2023

Choose a reason for hiding this comment

Uh oh!

ghost commented Oct 30, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ghost commented May 1, 2023 •

edited by ghost

Loading

Artturin commented Aug 3, 2023 •

edited

Loading

Artturin commented Aug 4, 2023 •

edited

Loading

emilylange commented Oct 19, 2023 •

edited

Loading