Skip to content

maintainers/scripts/instantiate-constitutents.sh: init#221540

Closed
ghost wants to merge 3 commits intomasterfrom
unknown repository
Closed

maintainers/scripts/instantiate-constitutents.sh: init#221540
ghost wants to merge 3 commits intomasterfrom
unknown repository

Conversation

@ghost
Copy link

@ghost ghost commented Mar 16, 2023

This commit adds a script which lets you "do what Hydra does" using only nix. See the comment at the top of the file for sample usage.

This script relies on make -j to run large numbers of nix-instantiates in parallel. You'll need to tune the -j value appropriately for your machine's core/memory balance. On my 32-core opteron machines I find that -j64 keeps all the cores busy, using about 18GB of memory (~250MB per nix-instance).

@ghost ghost mentioned this pull request Mar 16, 2023
4 tasks
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Mar 16, 2023
@lovesegfault
Copy link
Member

@ghost
Copy link
Author

ghost commented Mar 17, 2023

Isn't this basically @Mic92's nix-eval-jobs?

I don't think so. nix-eval-jobs doesn't seem to know how to turn a Hydra job's .constituents into a pile of separate derivations, which is what this script does. Instead it just gives you a single derivation which doesn't actually build the constituents:

nix-shell -p nix-eval-jobs --command 'nix-eval-jobs --expr "(import <nixpkgs/nixos/release-combined.nix> { supportedSystems = [\"x86_64-linux\"]; }).tested"'
warning: `--gc-roots-dir' not specified
{"attr":"","attrPath":[],"drvPath":"/nix/store/iraxp77m9qgkq8xd4dyabj36n6i94vis-nixos-23.05pre56789.gfedcba.drv","name":"nixos-23.05pre56789.gfedcba","outputs":{"out":"/nix/store/5zgj84nhm8yciyx2mimf9vw5lb6d1m78-nixos-23.05pre56789.gfedcba"},"system":"x86_64-linux"}

If you run nix show-derivation on the derivation above you'll see that it has constituents as an environment variable. That derivation won't build the constituents; all it does is write their names into a file.

@ghost

This comment was marked as resolved.

@ghost ghost marked this pull request as draft April 24, 2023 03:14
This commit adds a script which lets you "do what Hydra does" using
only `nix`.  See the comment at the top of the file for sample
usage.

This script relies on `make -j` to run large numbers of
`nix-instantiate`s in parallel.  You'll need to tune the `-j` value
appropriately for your machine's core/memory balance.  On my 32-core
opteron machines I find that -j64 keeps all the cores busy, using
about 18GB of memory (~250MB per `nix-instance`).

Co-authored-by: Bernardo Meurer <[email protected]>
@ghost
Copy link
Author

ghost commented Jul 13, 2023

Squashed, rebased. Currently in use to test #237856 here

@ghost ghost marked this pull request as ready for review July 13, 2023 03:04
@ghost ghost requested a review from lovesegfault July 13, 2023 05:19
@ghost ghost requested a review from ncfavier July 25, 2023 22:38
@ghost ghost marked this pull request as draft October 26, 2023 06:10
@ghost ghost closed this Jan 23, 2024
@ghost ghost deleted the pr/instantiate-constituents branch January 23, 2024 06:48
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants