Reduce overhead of spack install pkg for develop environments.#51518
Reduce overhead of spack install pkg for develop environments.#51518vbrunini wants to merge 2 commits intospack:developfrom
spack install pkg for develop environments.#51518Conversation
By using a cache for Spec objects and reducing the number of copies of Spec objects being done. Combined with spack#51367 and the reduced install task sleep time from spack#51491 this reduces the time for a no-op install of a development environment with 94 packages (including dependencies) from 8s to 2.8s for me. It also provides a similar reduction in time for `spack build-env --dump env.txt mypkg` which we frequently use to start a shell with the build environment. Signed-off-by: Victor Brunini <[email protected]>
Signed-off-by: Victor Brunini <[email protected]>
|
This seems to have a larger scope than "develop environments". Maybe split into parts? Some changes look controversial from skimming the diff. |
In my mind it's mostly relevant for the iterative development use case because these relatively small overheads of a few seconds aren't as impactful when doing full package builds compared to incremental builds with a small number of modified files, but yes the code changes certainly impact other cases as well. Any suggestions on how to split it up? I can split the two commits into separate PRs, but most of the changes are just from the first commit. If you'd like that one split up I think the two logical pieces are:
|
2 participants
We are moving to using spack to drive day to day development of a group of related packages. In the process we have noticed that there is a someoverhead running some commands like
spack installfor an incremental build where only a small number of files in one or two packages were modified when compared to directly running the packages build system.Combined with #51367 and the reduced install task sleep time from #51491 this reduces the time for a no-op install of a development environment with 94 packages (including dependencies) from 8s to 2.8s for me. It also provides a similar reduction in time for
spack build-env --dump env.txt mypkgwhich we frequently use to start a shell with the build environment.@psakievich