Skip to content

qt5: cross compile with a clean separation of build and host binaries#267311

Draft
eryngion wants to merge 26 commits intoNixOS:stagingfrom
eryngion:qt5-cross-with-a-clean-native-split
Draft

qt5: cross compile with a clean separation of build and host binaries#267311
eryngion wants to merge 26 commits intoNixOS:stagingfrom
eryngion:qt5-cross-with-a-clean-native-split

Conversation

@eryngion
Copy link

@eryngion eryngion commented Nov 13, 2023

Description of changes

By default when cross compiling QT builds all binaries that are marked in their .pro files with option(host_build) for the build platform instead of the host platform, and those binaries end up in the dev outputs (maybe not only there) of packages intended for the host platform. While there is some expectation of this situation in Nix [1], it prevents us from easily swapping natively and cross built dependencies, i.e. "lazy cross-compiling" and the host platform doesn't get the full set of binaries.

Here I'm attempting to have only host-targeted binaries in the cross-compiled packages and somehow live with this new situation. Even if this approach fails, I have here a couple of fixes that can helpful elsewhere.

I'll direct this PR to staging when we will be done discussing it.

[1] patchShebangs expects that binaries in dev outputs will be run on the build platform, and propagatedNativeBuildInputs, being implemented as an in-store dependency, essentially adds the same hard dependency on the platform where the package was built.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux (cross)
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@github-actions github-actions bot added 6.topic: python Python is a high-level, general-purpose programming language. 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: qt/kde Object-oriented framework for GUI creation 6.topic: kernel The Linux kernel 8.has: documentation This PR adds or changes documentation 8.has: module (update) This PR changes an existing module in `nixos/` 6.topic: printing Drivers, CUPS & Co. 6.topic: policy discussion Discuss policies to work in and around Nixpkgs 6.topic: vim Advanced text editor 6.topic: ocaml OCaml is a general-purpose, high-level, multi-paradigm programming language. 6.topic: steam Steam game store/launcher (store.steampowered.com) 6.topic: nodejs Node.js is a free, open-source, cross-platform JavaScript runtime environment labels Nov 13, 2023
@eryngion eryngion changed the base branch from staging to master November 13, 2023 22:02
@github-actions github-actions bot removed 6.topic: python Python is a high-level, general-purpose programming language. 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: kernel The Linux kernel 8.has: documentation This PR adds or changes documentation 8.has: module (update) This PR changes an existing module in `nixos/` 6.topic: printing Drivers, CUPS & Co. 6.topic: policy discussion Discuss policies to work in and around Nixpkgs 6.topic: vim Advanced text editor 6.topic: ocaml OCaml is a general-purpose, high-level, multi-paradigm programming language. 6.topic: steam Steam game store/launcher (store.steampowered.com) 6.topic: nodejs Node.js is a free, open-source, cross-platform JavaScript runtime environment labels Nov 13, 2023
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe qmakePathHook should be run for both types of packages? Not sure yet.

@NixOSInfra NixOSInfra added the 12.first-time contribution This PR is the author's first one; please be gentle! label Nov 13, 2023
@eryngion
Copy link
Author

@amjoseph-nixpkgs @Artturin

@FliegendeWurst FliegendeWurst added the 6.topic: cross-compilation Building packages on a different platform than they will be used on label Nov 13, 2023
@ofborg ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 13, 2023
eryngion and others added 24 commits February 5, 2026 15:29
It never worked and it's just an attempt to duplicate the logic
from qmake (i.e. QMAKEPATH -> QMAKEMODULES translation) that qmake
does all by itself.
hyphen's sources in tarbals are identical, and the version from
pkgs properly supports cross compilation.
…qmakeFlags

but don't use __structuredAttrs
Add to qtbase support for building a small bootstrap package.

Move qmake and other tools from qtbase.dev to qtbase.qmake
to avoid pulling in a second version of qtbase.dev in cross builds.

Use hidden bootstrap packages for qtbase and qttools instead of
overrideScope-based staging process to solve some spurious builds
and infinite recursions.

This commit by itself will allow to cross compile only qttranslations
and qtbase.
Also teach the setup hook, qmake and cmake scripts to search various
build tools in PATH and use them for cross compilation.

This will allow to cross compile packages that require only qtbase and
qmake.
… a native-built one

For this we need to dynamically detect cross compilation in qtbase-setup-hook.sh
and in mkspecs, and stop unconditionally using binary lndir in propagatedNativeBuildInputs.

For now qtbase substitution will work only in one (most used) direction, and in many cases
will additionally require to manually add a buildHost version of qtbase.qmake to
nativeBuildInputs of the package that depends on qtbase.
Without it qtwebchannel will fail to find some library includes,
qtquickcontrols will refuse to build without properly failing the
build:
```
Some of the required modules (qtHaveModule(quick)) are not available.
Skipped.
```
And there are probably more problems, not always obvious.
It needs only qtdeclarative to build, not qtquickcontrols.
If you're a nativeBuildInput somewhere, you probably shouldn't propagate
qtbase.
…s with qttools as a buildInput

And don't propagate qtbase to avoid pulling in a second qtbase.dev in
cross builds.
Not sure why it was done.

This reverts commit 49e8d9d.
I expect here that qtwebengine will propagate qtwebchannel and qtdeclarative and that
qtdeclarative will propagate itself as a nativeBuildInput. But maybe we
shouldn't rely on those propagations?
@eryngion eryngion force-pushed the qt5-cross-with-a-clean-native-split branch from 15f1639 to 3bff735 Compare February 5, 2026 18:49
@eryngion
Copy link
Author

eryngion commented Feb 5, 2026

Okay, there's now a second track with simpler changes in #487372. But since they're required for everything else and GitHub doesn't seem to have stacked PRs, for now I'm keeping them here too.

Also the checks are exploding on me for the second day: yesterday it was formatting that I can't get out of nixfmt 1.2.0 locally and today it's some non-existing changes to the files I didn't touch. Could the latter be related to python3Packages in commit messages?

@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Feb 8, 2026
@doronbehar
Copy link
Contributor

Also the checks are exploding on me for the second day: yesterday it was formatting that I can't get out of nixfmt 1.2.0 locally and today it's some non-existing changes to the files I didn't touch. Could the latter be related to python3Packages in commit messages?

Hopefully these issues will be fixed by the time we reach to this PR again. Let's focus on #487372 in the meantime (I'm marking this one as a draft 🙏).

@doronbehar doronbehar marked this pull request as draft February 10, 2026 08:23
@doronbehar doronbehar mentioned this pull request Feb 21, 2026
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2.status: merge conflict This PR has merge conflicts with the target branch 6.topic: cross-compilation Building packages on a different platform than they will be used on 6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 501-1000 This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 1001-2500 This PR causes many rebuilds on Linux and should target the staging branches. 12.first-time contribution This PR is the author's first one; please be gentle!

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants