Skip to content

EvalState::realiseContext(): Allow access to the entire closure#12045

Merged
roberth merged 1 commit intoNixOS:masterfrom
DeterminateSystems:allow-closure
Dec 16, 2024
Merged

EvalState::realiseContext(): Allow access to the entire closure#12045
roberth merged 1 commit intoNixOS:masterfrom
DeterminateSystems:allow-closure

Conversation

@edolstra
Copy link
Member

Motivation

When doing IFD, we should allow access to the entire closure of the store paths built by the derivation, not just the top-level paths.

Fixes #11030.

Context


Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

@github-actions github-actions bot added the with-tests Issues related to testing. PRs with tests have some priority label Dec 13, 2024
@Ericson2314
Copy link
Member

Hmm, makes me wonder if we ever want allowPath except for in the definition of allowClosure.

@Mic92
Copy link
Member

Mic92 commented Dec 14, 2024

Hmm, makes me wonder if we ever want allowPath except for in the definition of allowClosure.

What about builtins.toFile?

Comment on lines +354 to +355
StorePathSet closure;
store->computeFSClosure(storePath, closure);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Computing the closure from scratch could cause a performance problem.
n calls to allowClosure would be O(n * closure size), which may be significantly larger than O(total allowed closure size).
The latter can be achieved by computing the closure of paths that aren't in an allowed closure yet, i.e. checking the references recursively and skipping ones that have been of the closure of any preceding allowClosure call.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cost of the call to computeFSClosure() is likely to be trivial compared to the actual build of the derivation we're importing from.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not entirely trivial to implement this optimization given the current code. I'm ok to keep this as is.

@NixOS NixOS deleted a comment from mergify bot Dec 16, 2024
@Mic92 Mic92 dismissed roberth’s stale review December 16, 2024 15:12

He is ok with it.

@roberth
Copy link
Member

roberth commented Dec 16, 2024

Hmm, makes me wonder if we ever want allowPath except for in the definition of allowClosure.

This would allow any store path to be included not just into derivations but into IFD by doing

withStorePath = storePathStr: f:
  let path = builtins.appendContext ... storePathStr;
  in builtins.seq
    (builtins.toFile "dummy" "${path}")
    (f path)

@roberth roberth enabled auto-merge December 16, 2024 15:17
@NixOS NixOS deleted a comment from mergify bot Dec 16, 2024
@mergify
Copy link
Contributor

mergify bot commented Dec 16, 2024

queue

🟠 Waiting for conditions to match

Details
  • -closed [📌 queue requirement]
  • any of: [🔀 queue conditions]
    • all of: [📌 queue conditions of queue default]
      • any of: [🛡 GitHub branch protection]
        • check-neutral = installer test on macos
        • check-skipped = installer test on macos
        • check-success = installer test on macos
      • any of: [🛡 GitHub branch protection]
        • check-neutral = installer test on ubuntu
        • check-skipped = installer test on ubuntu
        • check-success = installer test on ubuntu
      • any of: [🛡 GitHub branch protection]
        • check-neutral = tests on macos
        • check-skipped = tests on macos
        • check-success = tests on macos
      • any of: [🛡 GitHub branch protection]
        • check-neutral = tests on ubuntu
        • check-skipped = tests on ubuntu
        • check-success = tests on ubuntu
  • -conflict [📌 queue requirement]
  • -draft [📌 queue requirement]
  • any of: [📌 queue -> configuration change requirements]
    • -mergify-configuration-changed
    • check-success = Configuration changed

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-12-16-nix-team-meeting-minutes-203/57483/1

mergify bot added a commit that referenced this pull request Dec 31, 2024
…2045

EvalState::realiseContext(): Allow access to the entire closure (backport #12045)
@roberth roberth added automatic backport This PR is a backport produced by automation (does not trigger backporting) and removed automatic backport This PR is a backport produced by automation (does not trigger backporting) labels Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

with-tests Issues related to testing. PRs with tests have some priority

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Passing store paths into functions from other flakes causes errors in pure evaluation mode

5 participants