Hang on large set of recursive-nix builds #7297
Description
Describe the bug
nix build hangs when building a large (e.g. 1000) set of derivations that each run one nix build recursively.
Maybe a deadlock e.g. the inner builds blocking to acquire a resource (e.g. builder uid) that is held by the outer builds making forward progress impossible? (Wild guess.)
Steps To Reproduce
Run nix build -f rec-big.nix with the file contents below.
Nix build hangs before completing the build:
$ nix build -f rec-big.nix
error (ignored): error: writing to file: Broken pipe
error (ignored): error: writing to file: Broken pipe
Expected behavior
nix build should build all derivations without errors and then terminate. (This is the behavior on this same machine when the recursive nix invocation is commented out from the script.)
nix-env --version output
nix-env (Nix) 2.11.0
Additional context
Contents of rec-big.nix:
# rec-big.nix -- large set of derivations each with one level of recursion
{ size ? 1000 }:
let
# master branch 2022-11-08
nixpkgs = fetchTarball {
url = "https://github.com/nixos/nixpkgs/archive/f6072a8d7b1520194453e27be1fd59a28c751a7f.tar.gz";
sha256 = "sha256:1zr2jm5gjbsyca9bcq7f3h354qf8d5bd6b0h4qjr0i03dvrvl10s";
};
in
with import nixpkgs {};
map (n: runCommand "outer-${toString n}"
{
requiredSystemFeatures = [ "recursive-nix" ];
nativeBuildInputs = [ nix ];
expr = writeText "inner.nix"
''
with import ${nixpkgs} {};
runCommand "inner"
{ }
"echo building inner ${toString n}.. > $out"
'';
}
''
# Run nix build and log output
nix --experimental-features "recursive-nix nix-command" \
build -L -f $expr
echo ok > $out
'')
(lib.range 1 size)Nix config in NixOS:
nix.nixPath = [ "nixpkgs=/home/luke/git/nixpkgs" ];
nix.package = pkgs.nixUnstable;
nix.extraOptions = ''
experimental-features = nix-command flakes recursive-nix
trusted-users = luke
'';
nix.settings.sandbox = "relaxed";