Skip to content

pkgsStatic: make pkgsMusl a no-op#136549

Open
r-burns wants to merge 1 commit intoNixOS:masterfrom
r-burns:pkgs-static-musl
Open

pkgsStatic: make pkgsMusl a no-op#136549
r-burns wants to merge 1 commit intoNixOS:masterfrom
r-burns:pkgs-static-musl

Conversation

@r-burns
Copy link
Contributor

@r-burns r-burns commented Sep 3, 2021

When pkgsStatic is already musl, we can let pkgsMusl simply be a no-op.

As the interaction of pkgsStatic and pkgsMusl machinery is currently broken
(see discussion in #114510), this fixes pkgsStatic.pkgsMusl.stdenv.

Motivation for this change
Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • 21.11 Release Notes (or backporting 21.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@r-burns r-burns added 6.topic: static Static builds (e.g. pkgsStatic) 6.topic: stdenv Standard environment labels Sep 3, 2021
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Sep 3, 2021
@github-actions github-actions bot removed the 6.topic: stdenv Standard environment label Oct 7, 2021
@r-burns r-burns requested a review from alyssais October 9, 2021 19:04
@sternenseemann
Copy link
Member

sternenseemann commented Oct 22, 2021

pkgsStatic.pkgsMusl seems to work fine for me, e. g. I can build pkgsStatic.pkgsMusl.hello. I don't think there's a need for this change here?

@r-burns
Copy link
Contributor Author

r-burns commented Oct 22, 2021

It appears to work but is incorrect:

nix eval -f . pkgsStatic.pkgsMusl.stdenv.hostPlatform.isStatic
false

pkgsMusl is "resetting" the isStatic back to false

@stale
Copy link

stale bot commented Apr 25, 2022

I marked this as stale due to inactivity. → More info

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Apr 25, 2022
@r-burns
Copy link
Contributor Author

r-burns commented Apr 25, 2022

Still relevant. Previous command gives same result on current master.

@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Apr 25, 2022
@rnhmjoj
Copy link
Contributor

rnhmjoj commented May 1, 2022 •
edited
Loading

The fact pkgsStatic is using musl is an accident: glibc static build was broken at the time and musl was used instead. Ideally pkgsStatic should differ from pkgs in just having -static kind of flags.

Nowadays glibc.static is working and I was actually thinking about changing pkgsStatic back to using it.

@r-burns
Copy link
Contributor Author

r-burns commented May 1, 2022 •
edited
Loading

Well, this whole contraption is conditional on useMusl anyway, so if we stop setting useMusl for glibc linux, it should gracefully use static glibc instead.

@superherointj
Copy link
Contributor

superherointj commented Sep 9, 2022 •
edited
Loading

@rnhmjoj Do you still plan on moving static back to glibc?

@rnhmjoj
Copy link
Contributor

rnhmjoj commented Sep 10, 2022

That's the idea, but I don't plan on working on this soon.

@uri-canva uri-canva mentioned this pull request Oct 16, 2022
Open
@uri-canva
Copy link
Contributor

uri-canva commented Oct 16, 2022

Opened a new issue for pkgsStatic using musl, so it's easier to find and people can refer to it: #196329.

@wegank wegank added 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 2.status: merge conflict This PR has merge conflicts with the target branch labels Mar 19, 2024
@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 20, 2024
@wolfgangwalther wolfgangwalther mentioned this pull request Apr 1, 2024
Open
@wolfgangwalther wolfgangwalther mentioned this pull request Apr 13, 2024
Merged
13 tasks
wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this pull request Dec 21, 2024
@wolfgangwalther
pkgs/top-level: make package sets composable
ed0cfe3 
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.gnu64 is a cross compilation from musl to glibc
  - pkgsCross.gnu64.pkgsMusl is a cross compilation to musl

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead of
    musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this pull request Jan 25, 2025
@wolfgangwalther
pkgs/top-level: make package sets composable
01a02e4 
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
  - pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
MattSturgeon pushed a commit to MattSturgeon/nixpkgs that referenced this pull request Feb 8, 2025
@wolfgangwalther @MattSturgeon
pkgs/top-level: make package sets composable
3c80385 
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
  - pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Resolves NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
nadiaholmquist pushed a commit to nadiaholmquist/nixpkgs that referenced this pull request Feb 9, 2025
@wolfgangwalther @nadiaholmquist
pkgs/top-level: make package sets composable
943507a 
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
  - pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Supersedes NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this pull request Apr 1, 2025
@wolfgangwalther
pkgs/top-level: make package sets composable
acbabb9 
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.

There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:

- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.

- Once pkgsCross is used any subsequent package sets should affect the
  **host platform** and not the build platform. Examples:
  - pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
  - pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64

- Modifications from an earlier layer should not be lost, unless
  explicitly overwritten. Examples:
  - pkgsStatic.pkgsMusl should still be static.
  - pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of musl.

Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
  same flags over and over again.

Resolves NixOS#136549
Resolves NixOS#114510
Resolves NixOS#212494
Resolves NixOS#281596
@nixpkgs-ci nixpkgs-ci bot removed 2.status: merge conflict This PR has merge conflicts with the target branch 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md labels Jan 11, 2026
@r-burns
pkgsStatic: make pkgsMusl a no-op
69c3a51 
When pkgsStatic is already musl, we can let pkgsMusl simply be a no-op.
As the interaction of pkgsStatic and pkgsMusl machinery is currently
broken, this fixes pkgsStatic.pkgsMusl.stdenv.
@r-burns
Copy link
Contributor Author

r-burns commented Jan 11, 2026

Was looking through my old open PRs and discovered that this is still (!) an issue.

On master:

nix eval -f . pkgsStatic.pkgsMusl.stdenv.hostPlatform.isStatic                                                                                                                             
false

Rebased to fix conflicts, and simplified the patch a bit. There are no longer any assumptions about pkgsStatic using musl - the no-op overlay for pkgsMusl is only applied when super'.stdenv.hostPlatform is already Musl.

Would appreciate a review to fix this long-standing issue.

@r-burns r-burns requested a review from ConnorBaker January 21, 2026 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Ericson2314 Ericson2314 Awaiting requested review from Ericson2314

@matthewbauer matthewbauer Awaiting requested review from matthewbauer

@nbp nbp Awaiting requested review from nbp

@alyssais alyssais Awaiting requested review from alyssais

@sternenseemann sternenseemann Awaiting requested review from sternenseemann

@ConnorBaker ConnorBaker Awaiting requested review from ConnorBaker

Assignees

No one assigned

Labels

6.topic: static Static builds (e.g. pkgsStatic) 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@r-burns @sternenseemann @rnhmjoj @superherointj @uri-canva @wegank