Skip to content

pkgsStatic: fix in combination with __structuredAttrs#428430

Merged
symphorien merged 1 commit intoNixOS:masterfrom
symphorien:pkstaticstructuredattrs
Jul 27, 2025
Merged

pkgsStatic: fix in combination with __structuredAttrs#428430
symphorien merged 1 commit intoNixOS:masterfrom
symphorien:pkstaticstructuredattrs

Conversation

@symphorien
Copy link
Member

pkgsStatic injects { NIX_CFLAGS_COMPILE = " -static"; }. When __structuredAttrs is true, it must inject { env.NIX_CFLAGS_COMPILE = " -static"; } instead, as attributes outside env are ignored.

Things done

  • Built on platform:
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • Tested, as applicable:
  • Ran nixpkgs-review on this PR. See nixpkgs-review usage.
  • Tested basic functionality of all binary files, usually in ./result/bin/.
  • Nixpkgs Release Notes
    • Package update: when the change is major or breaking.
  • NixOS Release Notes
    • Module addition: when adding a new NixOS module.
    • Module update: when the change is significant.
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

Add a 👍 reaction to pull requests you find important.

@emilazy
Copy link
Member

emilazy commented Jul 25, 2025

Can we just do this unconditionally regardless of __structuredAttrs?

@symphorien
Copy link
Member Author

the current formulation avoids a mass rebuild

@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. 6.topic: stdenv Standard environment labels Jul 25, 2025
@emilazy
Copy link
Member

emilazy commented Jul 25, 2025

Did this regress recently? Otherwise I think it’s okay to wait for staging, since we’d want to clean this up on staging anyway. I’m also not sure how many rebuilds pkgsStatic is; we don’t build a full jobset on it, so maybe it’s fine to send to master?

@RaitoBezarius
Copy link
Member

Did this regress recently? Otherwise I think it’s okay to wait for staging, since we’d want to clean this up on staging anyway. I’m also not sure how many rebuilds pkgsStatic is; we don’t build a full jobset on it, so maybe it’s fine to send to master?

this regressed Lix static builds :'( — why not send this and clean it up on staging (by reverting this) as well?

@symphorien
Copy link
Member Author

also I don't want to know what happens if a derivation with { NIX_CFLAGS_LINK = "foo"; } is overridden to { NIX_CFLAGS_LINK = "foo"; env.NIX_CFLAGS_LINK = "foo -static"; }

Copy link
Member

@emilazy emilazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I’m okay landing this if it’s a recent regression. I just didn’t know if it was or not.

@nixpkgs-ci nixpkgs-ci bot added the 12.approvals: 1 This PR was reviewed and approved by one person. label Jul 25, 2025
@philiptaron
Copy link
Contributor

Could someone link the commit / PR that's being referenced as the source of this issue?

@lf-
Copy link
Member

lf- commented Jul 26, 2025

Could someone link the commit / PR that's being referenced as the source of this issue?

#394674

@alois31
Copy link
Contributor

alois31 commented Jul 26, 2025

also I don't want to know what happens if a derivation with { NIX_CFLAGS_LINK = "foo"; } is overridden to { NIX_CFLAGS_LINK = "foo"; env.NIX_CFLAGS_LINK = "foo -static"; }

There will be an evaluation error. However quite a few packages already override NIX_CFLAGS_LINK manually so that I think it's still enough of a footgun that it should better be avoided.

@alois31
Copy link
Contributor

alois31 commented Jul 26, 2025

However what could work is basically inverting the current test and putting it in env by default, and only in the attrs themselves when it's already there.

@symphorien
Copy link
Member Author

symphorien commented Jul 26, 2025

that would have the effect that (stdenv.mkDerivation { NIX_CFLAGS_LINK = "foo"; }).overrideAttrs { NIX_CFLAGS_LINK = "bar"; }) would work in top level pkgs and fail to evaluate in pkgsStatic.

pkgsStatic injects { NIX_CFLAGS_COMPILE = " -static"; }. When
__structuredAttrs is true, it must inject { env.NIX_CFLAGS_COMPILE = "
-static"; } instead, as attributes outside env are ignored.
@alois31
Copy link
Contributor

alois31 commented Jul 26, 2025

That would actually still work because NIX_CFLAGS_COMPILE is already in the original. I assume you mean that (stdenv.mkDerivation { }).overrideAttrs { NIX_CFLAGS_LINK = "bar"; } would break, which is indeed the case?

@symphorien
Copy link
Member Author

wouldn't the adapter move the first NIX_CFLAGS_LINK to env and the the override add it to top-level ? anyway your example also illustrates it well.

@alois31
Copy link
Contributor

alois31 commented Jul 26, 2025

My proposal was not that the adapter would move it. It would append to NIX_CFLAGS_COMPILE if it's already there and create or append to env.NIX_CFLAGS_COMPILE otherwise. However I think your approach of checking for __structuredAttrs has even less potential for breakage and hence might be preferable.

Copy link
Contributor

@alois31 alois31 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beside Jade's concern, this also breaks the configureFlags and cmakeFlags just below because the conditional has lower precedence than the attrset merge. (The bug was already there before, but only exposed with env.NIX_CFLAGS_LINK set while now __structuredAttrs is enough.)

@github-project-automation github-project-automation bot moved this to In Progress in Stdenv Jul 26, 2025
@symphorien symphorien force-pushed the pkstaticstructuredattrs branch from 7ce532c to 6b621d1 Compare July 26, 2025 14:03
@symphorien
Copy link
Member Author

this also breaks the configureFlags and cmakeFlags just below because the conditional has lower precedence than the attrset merge

well spotted, thanks

@symphorien symphorien force-pushed the pkstaticstructuredattrs branch from 6b621d1 to d0b787a Compare July 26, 2025 14:15
@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. and removed 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. labels Jul 26, 2025
Copy link
Contributor

@alois31 alois31 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, does not break my system, and fixes the static lix build. Thanks!

@nixpkgs-ci nixpkgs-ci bot added 12.approvals: 2 This PR was reviewed and approved by two persons. and removed 12.approvals: 1 This PR was reviewed and approved by one person. labels Jul 27, 2025
@symphorien symphorien merged commit 128cdc5 into NixOS:master Jul 27, 2025
28 of 30 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Stdenv Jul 27, 2025
@RaitoBezarius
Copy link
Member

On behalf of Lix, thank you so much @symphorien!

@wolfgangwalther
Copy link
Contributor

Bisect leads me here: This broke pkgsStatic.libpq.

According to #429535 it also broke pkgsStatic.postgresql.

It would have been great to mention this PR in the structuredAttrs tracking issue #205690.

@alois31
Copy link
Contributor

alois31 commented Aug 7, 2025

Note that the issue does not appear to be related to __structuredAttrs itself, but due the pre-existing precedence issue that was fixed here because the __structuredAttrs change would have made it worse. I am investigating it but it could possibly take some time because postgresql builds a full LLVM.

@wolfgangwalther
Copy link
Contributor

wolfgangwalther commented Aug 7, 2025

Note that the issue does not appear to be related to __structuredAttrs itself, but due the pre-existing precedence issue that was fixed here because the __structuredAttrs change would have made it worse.

I remember pkgsStatic for both postgresql and libpq only started to work when we enabled structuredAttrs - as a side effect of unknown reason. I think what happened is, that this bug worked in favor for this case - postgresql can't take -static in NIX_CFLAGS_LINK, it will then fail with:

       > /nix/store/0adymgz38j1xgca61fgg14lvaq0iwq1b-x86_64-unknown-linux-musl-binutils-2.44/bin/x86_64-unknown-linux-musl-ld: /nix/store/54m0dlvjd38xgj7sxzv9bpmfg39gpsgv-x86_64-unknown-linux-musl-gcc-14.3.0/lib/gcc/x86_64-unknown-linux-musl/14.3.0/crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when making a shared object
       > /nix/store/0adymgz38j1xgca61fgg14lvaq0iwq1b-x86_64-unknown-linux-musl-binutils-2.44/bin/x86_64-unknown-linux-musl-ld: failed to set dynamic section sizes: bad value

I am investigating it but it could possibly take some time because postgresql builds a full LLVM.

Try pkgsStatic.libpq, which doesn't need LLVM - much quicker for me.

@wolfgangwalther
Copy link
Contributor

I confirmed that dontAddStaticConfigureFlags = true doesn't help, thus the "precedence issue" was not the problem. I think it's related to NIX_CFLAGS_LINK itself as mentioned above.

But yeah.. this just means that postgresql's build system is just not going to work well with pkgsStatic and we had just been lucky to have it work for a year...

@alois31
Copy link
Contributor

alois31 commented Aug 7, 2025

I'm not even sure it is the same issue; the linked bug mentions the difference in flags for compiler-rt-libc which libpq does not build-depend on. So either this is a red herring or there are multiple issues.

@alois31
Copy link
Contributor

alois31 commented Aug 7, 2025

It seems that libpq indeed is NIX_CFLAGS_LINK after all, as that's the only difference I can see in nix-diff.

@wolfgangwalther
Copy link
Contributor

I'm not even sure it is the same issue; the linked bug mentions the difference in flags for compiler-rt-libc which libpq does not build-depend on. So either this is a red herring or there are multiple issues.

Ah, I hadn't looked that close. I guess there are multiple issues then, and once compiler-rt-libc works, postgresql would hit the same issue that libpq has now.

@symphorien
Copy link
Member Author

sorry I won't be able to fix this, I will not be near computers for a few days.

@symphorien symphorien deleted the pkstaticstructuredattrs branch August 7, 2025 21:46
@alois31
Copy link
Contributor

alois31 commented Aug 8, 2025

Ah, I hadn't looked that close. I guess there are multiple issues then, and once compiler-rt-libc works, postgresql would hit the same issue that libpq has now.

Actually the compiler-rt-libc stuff is a red herring since it doesn't create meaningful changes, and while there are in fact multiple issues they all boil down to things only working before because they were actually dynamically linked. See the comments I posted in #429535 (in fact I think this conversation should continue there).

lf- pushed a commit to lix-project/lix that referenced this pull request Aug 20, 2025
Normally `pkgsStatic` adds ` -static` to `NIX_CFLAGS_COMPILE`. Due to a bug
this did not apply with `__structuredAttrs`. As the fix [1] has not been
backported yet, put it in the package manually.

[1] NixOS/nixpkgs#428430

Fixes: https://git.lix.systems/lix-project/lix/issues/962

Change-Id: I6a6a6964c6a33f486ba3df3be16f715ad1b060c3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: stdenv Standard environment 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. 12.approvals: 2 This PR was reviewed and approved by two persons.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

7 participants