closureInfo: disallow substitutes, instead of only prefering local build#123943
closureInfo: disallow substitutes, instead of only prefering local build#123943ajs124 wants to merge 1 commit intoNixOS:masterfrom
Conversation
…uild "exportReferencesGraph" combined with "__structuredAttrs = true" "includes the sizes and hashes of paths". If the result of the closureInfo is substituted, but the contents of any path of the closure differs between the binary cache and the local store, which can easily happen, because not everything is perfectly reproducible, this leads to hash mismatches in the image builders, which pass the output of closureInfo to "nix-store --load-db".
ofborg
bot
added
10.rebuild-darwin: 11-100
roberth
left a comment
roberth
left a comment
There was a problem hiding this comment.
Nothing prevents a subsequent derivation, like the one that builds the image, from incorporating the output of closureInfo, causing it to be substitutable. So while this may help with the symptoms you're experiencing, it is not a general solution.
This derivation exacerbates all reproducibility problems, which no-one expects a derivation to do. For this reason, this functionality must only be used in derivations that produce no references to the original paths; instead incorporating all paths in an image (or similar). After all that's the only way to guarantee a consistent output. (a locally built path with unreproducible output may exist before substituting the derivation with the closureInfo dependency)
I would suggest replacing it by a hash- and size-free variant such as writeReferencesToFile wherever possible. That function only writes all closure paths and doesn't exacerbate reproducibility problems. The remaining use cases, hopefully only image building derivations, don't benefit at all from having this dangerous logic in a separate derivation. Perhaps its (limited) logic can be provided as a script instead, with some clear warnings that any derivations using it must not reference the original paths.
4 participants
"exportReferencesGraph" combined with "__structuredAttrs = true" "includes
the sizes and hashes of paths". If the result of the closureInfo is
substituted, but the contents of any path of the closure differs between
the binary cache and the local store, which can easily happen, because
not everything is perfectly reproducible, this leads to hash mismatches in
the image builders, which pass the output of closureInfo to
"nix-store --load-db".
Also see NixOS/nix#4840
Things done
sandboxinnix.confon non-NixOS linux)nix-shell -p nixpkgs-review --run "nixpkgs-review wip"./result/bin/)