@ofborg build tests.trivial-builders.references
@ofborg build tests.trivial-builders.writeClosure-union
@ofborg build apptainer.tests.image-hello-cowsay

Regarding the #178717 (comment) about manual string interpolation, the paths will eventually go through string interpolation before

I guess the question I rather wanted to ask is (I'm not particularly familiar with lazy languages, this is probably very basic): does sort limit parallelism in any way? The worst case I imagine is when builds are scheduled e.g. two at a time and have to complete before sort decides which elements to compare next

Regarding the #178717 (comment) about manual string interpolation, the paths will eventually go through string interpolation before

I guess the question I rather wanted to ask is (I'm not particularly familiar with lazy languages, this is probably very basic): does sort limit parallelism in any way? The worst case I imagine is when builds are scheduled e.g. two at a time and have to complete before sort decides which elements to compare next

AFAIK, evaluation is independent from realization (build) as long as "import from derivation" (IFD) is not used.

For example, if we would like to build apptainer.tests.image-hello-cowsay, Nix will first evaluate apptainer.tests.image-hello-cowsay and all its dependencies, including apptainer, qemu, and writeClosure [ hello cowsay] (which involves tde evaluation of hello and cowsay). Whenever the built-in function derivation is called (by stdenv.mkDerivation), a store derivation (/nix/store/*.drv) is instantaneously written to the Nix Store.

After all the evaluation is finished, Nix reads the store derivation of apptainer.tests.image-hello-cowsay, and resolve all its build-time closure, and starts to realize them.

The only read/write things evaluation can affect, when IFD is not involved, is

1. The instantiation of store derivations.
2. The built-in fetchers.
3. The built-in file writers.

Those are unfortunately done sequentially, as Nix evaluation doesn't seem to support paralellism. That's probably why built-in fetchers are not used often in Nixpkgs.

AFAIK, evaluation is independent from realization (build) as long as "import from derivation" (IFD) is not used.

Indeed. I guess I was just blabbering something without giving it any actually thought.

Looking back, sort and filter do introduce an extra synchronization point when computing e.g. outPath, but this is mostly irrelevant.

However, I still don't like the idea of sorting by hashes (squinting an eye, this should be equivalent to a random shuffle, when perturbing any inputs). I'd rather sort by names first and then by hashes, just to have stable layouts.

However, I still don't like the idea of sorting by hashes (squinting an eye, this should be equivalent to a random shuffle, when perturbing any inputs). I'd rather sort by names first and then by hashes, just to have stable layouts.

It turns out that the order of the paths taken by exportReferencesGraph doesn't affect the output content, as exportReferencesGraph sorts the printed paths by hashes. However, it could affect the traceback log in terms of evaluation errors.

If we want to change the output order, we'll need to add custom sorting to the jq command pipe.

It turns out that the order of the paths taken by exportReferencesGraph doesn't affect the output content, as exportReferencesGraph sorts the printed paths by hashes.

Nice. It would change the IA outPath of writeClosure [...], however.

I guess the question I rather wanted to ask is (I'm not particularly familiar with lazy languages, this is probably very basic): does sort limit parallelism in any way? The worst case I imagine is when builds are scheduled e.g. two at a time and have to complete before sort decides which elements to compare next

Hello myself from a year back, you were lazy and also wrong about most things. In normal Nix, this would sort by IA hashes. Even in CA Nix this would sort by the unresolved outPaths, which functionally behave same as IA hashes. Sorting does is a synchronization point, but it hardly matters because (1) Nixpkgs already big, (2) Nix already slow. To conclude, I'm now in favour of sorting and deduping.

Out of aesthetic considerations, I would have preferred ORDER BY lib.getName, outPath, so paths are first "grouped" by their highly-non-random names, and then ties are resolved by their highly-random hashes. That would be, of course, just a waste of computation