Skip to content

allow managed environments to be organized into folders#50994

Merged
psakievich merged 4 commits intospack:developfrom
rbberger:feature/organize_envs
Jul 29, 2025
Merged

allow managed environments to be organized into folders#50994
psakievich merged 4 commits intospack:developfrom
rbberger:feature/organize_envs

Conversation

@rbberger
Copy link
Copy Markdown
Member

@rbberger rbberger commented Jul 12, 2025

Spack currently forces environments to have simple names. This forces a flat folder structure for all managed environments. This PR relaxes this restriction and allows a folder structure to help organize environments. This would allow pointing Spack to a known environment root for a project which may be organized this way.

Below is a typical env setup that could make use of this.

% spack env list
==> 52 environments
    darwin/docs                                                 darwin/legion/ucx/cuda/openmpi-ampere-clang-debug
    darwin/format                                               darwin/legion/ucx/cuda/openmpi-volta-clang-debug
    darwin/hpx/mpich-clang-debug                                darwin/legion/ucx/openmp/openmpi-gcc-debug
    darwin/hpx/mpich-clang-release                              darwin/legion/ucx/openmp/openmpi-gcc-release
    darwin/hpx/mpich-gcc-debug                                  darwin/legion/ucx/serial/mpich-clang-debug
    darwin/hpx/mpich-gcc-release                                darwin/legion/ucx/serial/mpich-gcc-debug
    darwin/hpx/openmpi-gcc-debug                                darwin/legion/ucx/serial/mpich-gcc-defaults
    darwin/hpx/openmpi-gcc-release                              darwin/legion/ucx/serial/mpich-gcc-release
    darwin/legion/gasnet/cuda/openmpi-ibv-ampere-clang-debug    darwin/legion/ucx/serial/openmpi-gcc-debug
    darwin/legion/gasnet/cuda/openmpi-ibv-ampere-clang-release  darwin/legion/ucx/serial/openmpi-gcc-release
    darwin/legion/gasnet/cuda/openmpi-ibv-volta-clang-debug     darwin/mpi/cuda/mpich-ampere-clang-debug
    darwin/legion/gasnet/cuda/openmpi-ibv-volta-clang-release   darwin/mpi/cuda/mpich-volta-clang-debug
    darwin/legion/gasnet/cuda/openmpi-mpi-ampere-clang-debug    darwin/mpi/cuda/openmpi-ampere-clang-debug
    darwin/legion/gasnet/cuda/openmpi-mpi-ampere-clang-release  darwin/mpi/cuda/openmpi-volta-clang-debug
    darwin/legion/gasnet/cuda/openmpi-mpi-volta-clang-debug     darwin/mpi/openmp/mpich-clang-debug
    darwin/legion/gasnet/cuda/openmpi-mpi-volta-clang-release   darwin/mpi/rocm/openmpi-mi250-gcc-debug
    darwin/legion/gasnet/openmp/openmpi-ibv-gcc-debug           darwin/mpi/serial/mpich-clang-debug
    darwin/legion/gasnet/openmp/openmpi-ibv-gcc-release         darwin/mpi/serial/mpich-gcc-debug
    darwin/legion/gasnet/rocm/openmpi-mpi-mi250-gcc-debug       darwin/mpi/serial/mpich-gcc-release
    darwin/legion/gasnet/serial/mpich-ibv-clang-debug           darwin/mpi/serial/openmpi-clang-debug
    darwin/legion/gasnet/serial/mpich-ibv-gcc-debug             darwin/mpi/serial/openmpi-gcc-debug
    darwin/legion/gasnet/serial/mpich-ibv-gcc-defaults          darwin/mpi/serial/openmpi-gcc-release
    darwin/legion/gasnet/serial/mpich-ibv-gcc-release           
    darwin/legion/gasnet/serial/openmpi-ibv-gcc-debug
    darwin/legion/gasnet/serial/openmpi-ibv-gcc-release 
    darwin/legion/ucx/cuda/mpich-volta-clang-debug
% spack env activate darwin/hpx/mpich-clang-debug

Background: I've been writing wrappers/tools to circumvent this limitation by making use of directory environments. This PR tries to bring the missing feature to Spack itself. Marking this as draft for now until I see what else I break with this.

@rbberger rbberger force-pushed the feature/organize_envs branch from 37a40df to ed89350 Compare July 12, 2025 06:03
@rbberger rbberger added environments feature A feature is missing in Spack labels Jul 12, 2025
@rbberger rbberger force-pushed the feature/organize_envs branch from ed89350 to ed78fe4 Compare July 13, 2025 05:17
@rbberger rbberger marked this pull request as ready for review July 13, 2025 06:23
@haampie haampie added this to the v1.1.0 milestone Jul 14, 2025
@psakievich
Copy link
Copy Markdown
Contributor

Looks great! Docs and test(s)? Seems pretty straightforward but definitely something we'd want to add to the documentation on environments.

Assuming spack env create will treat the separators correctly to create nested env dirs?

@rbberger
Copy link
Copy Markdown
Member Author

@psakievich yes, surprisingly it just worked. env create, env activate , etc. But I agree, probably good to add some tests to prevent future breakage.

@rbberger rbberger force-pushed the feature/organize_envs branch from 9382083 to 393e82f Compare July 23, 2025 04:22
Copy link
Copy Markdown
Contributor

@psakievich psakievich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making my request more formal.

Let's modify the docs to explain how environment projection with nested paths can be used and add in a unit-test that uses this feature to guard against breaking it with future changes.

Other than that LGTM. With 1.0 out now we should be able to merge this soon.

@rbberger rbberger force-pushed the feature/organize_envs branch from 1399c4a to 025066f Compare July 26, 2025 01:51
@psakievich psakievich merged commit a492141 into spack:develop Jul 29, 2025
32 checks passed
@rbberger rbberger deleted the feature/organize_envs branch July 29, 2025 16:20
alstar555 pushed a commit to alstar555/spack that referenced this pull request Aug 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

environments feature A feature is missing in Spack

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants