Skip to content

stdenv: don't discard string context from ContentAddressed derivations#214044

Closed
trofi wants to merge 1 commit intoNixOS:stagingfrom
trofi:ca-restore-context
Closed

stdenv: don't discard string context from ContentAddressed derivations#214044
trofi wants to merge 1 commit intoNixOS:stagingfrom
trofi:ca-restore-context

Conversation

@trofi
Copy link
Contributor

@trofi trofi commented Feb 1, 2023

Without the change build for packages that use disallowedReferences fails in contentAddressedByDefault = true mode:

$ nix build -f. ruby_3_1 --arg config '{ contentAddressedByDefault = true; }'
...
error: derivation contains an illegal reference specifier '/0j3hif3ni7zl5zhlzzr5q2q23z66136mnzp75pyiqp5c72q14im2'
error: 1 dependencies of derivation '/nix/store/39ji7qp225pxvrm8cgvzmyjqsyis2n0h-ruby-3.1.2.drv' failed to build

Original intent of #211783 was to avoid pulling in actual derivation for reference scanning purposes.

Unfortunately CA derivations's outputPath are placeholders until they are instantiated.

Let's restore string context for CA derivations for now.

Description of changes
Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.05 Release Notes (or backporting 22.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

Without the change build for packages that use `disallowedReferences`
fails in `contentAddressedByDefault = true` mode:

    $ nix build -f. ruby_3_1 --arg config '{ contentAddressedByDefault = true; }'
    ...
    error: derivation contains an illegal reference specifier '/0j3hif3ni7zl5zhlzzr5q2q23z66136mnzp75pyiqp5c72q14im2'
    error: 1 dependencies of derivation '/nix/store/39ji7qp225pxvrm8cgvzmyjqsyis2n0h-ruby-3.1.2.drv' failed to build

Original intent of NixOS#211783 was to
avoid pulling in actual derivation for reference scanning purposes.

Unfortunately CA derivations's outputPath are placeholders until they
are instantiated.

Let's restore string context for CA derivations for now.
@trofi
Copy link
Contributor Author

trofi commented Feb 1, 2023

@r-vdp I hope that does not affect your current use case.

@Ericson2314
Copy link
Member

Heh this is funny. I suppose one would need to do like an IFD to force the CA derivation to be built and get the store path.

@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Feb 1, 2023
@r-vdp
Copy link
Contributor

r-vdp commented Feb 1, 2023

Hi @trofi, thanks for the heads-up!

I am not currently using CA derivations so this will not directly affect me, but I guess that this should be an extra motivation to fix the underlying bug in nix, as it could bite us again if CA derivations would become more widely used.

@thufschmitt
Copy link
Member

I'd rather just (or “in addition” for the time being) make the check here a bit more lenient and also allow CA-derivations placeholders (and possibly discard them) since we can't resolve them to store paths in the general case.

I guess that this should be an extra motivation to fix the underlying bug in nix, as it could bite us again if CA derivations would become more widely used.

I'm not sure that would change much here unfortunately since that bug is just Nix being overzealous in checking that the references mentioned are valid. If anything it's actually nice that being a library-side thing we can easily work around it

@trofi trofi closed this Feb 11, 2023
@trofi trofi deleted the ca-restore-context branch February 11, 2023 14:18
@Mindavi
Copy link
Contributor

Mindavi commented Oct 4, 2023

I think this PR is still valid with a comment that it can be removed after nix resolves this issue. Otherwise we have to do all kinds of workarounds in nixpkgs to temporarily solve this

@trofi
Copy link
Contributor Author

trofi commented Oct 5, 2023

I'm not sure it's valid. But I find it good enough for local use.

@gador
Copy link
Member

gador commented Nov 25, 2024

We need manual fixes as in #354451 #350137 or #297466
Would this PR fix this?

@Ericson2314
Copy link
Member

Ericson2314 commented Nov 25, 2024

@gador Yes that's right. We should do this instead of those manual fixes. I am thinking about a longer-term solution.

edit NixOS/nix#11954 and NixOS/nix#11955 is my thoughts.

@Ericson2314
Copy link
Member

@gador Can you resurrect this, and revert/close all the other temp fixes?

@gador
Copy link
Member

gador commented Nov 25, 2024

Yes, I'll do so tomorrow 👍

@trofi
Copy link
Contributor Author

trofi commented Nov 25, 2024

I've restored the PR as is if it helps. I'm using it since on my systems without problems:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: stdenv Standard environment 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants