-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Spack Environments 3.0: How they should REALLY work #7944
Description
@scheibelp @tgamblin
The ideas in this Issue might get implemented, but only after the current Environnments PR (#7843) is merged.
Rather than a spack env command that increasingly duplicates much of Spack, Environments should be integrated into existing Spack commands. The following commands could be "environmentized": find, all "build packages" commands, all "environment" commands (in spack help --all), module.
Commands would be run with an environment in maybe one of a few ways:
spack --env <env> command ...--- works for all commands, but only makes a difference for commands where it makes sense.SPACK_ENV=<env> spack command ...--- get the current Spack env out of an env var, but otherwise similar to (1). Nothing else is done in Spack this way, so probably not what we want.spack command --env <env>--- Add the--envflag only for commands where an environment makes a difference.- Don't use any global env specifier; instead, get the required env from the specs on the command line (see below).
Suppose we have an Environment with user specs aa, bb and cc added to it. Then the following conventions (or something like them) can be used anywhere a spec is asked for on the command line:
env:aarefers to theaauser spec in the current environment. Ifaahas been concretized, then it will be replaced with the concretized version ofaain the env. NOTE: This is different from justaa, which does an immediate concretization.env:expands to all user specs in env.env:aa:expands to all specs in theaaDAG.env:aa:xxrefers to a spec matchingxx, which is in the DAG of the concretizedaauser spec. This kind of control allows us to easily debug, rebuild, etc. individual problem packages within an environment.env::xxrefers to a spec matchingxxanywhere in any DAG in the concretized environment.env::expands to ALL SPECS in the environment.
IN general:
- Anything wanting an environment has no colons. Eg:
myenv - Anything wanting a user spec, or list of user specs, or top spec in a DAG, has one colon. Eg:
myenv:aaormyenv: - Anything wanting any spec has two colons. Eg:
myenv:aa:openblas.
spack install would work as it currently does: it concretizes and installs all in one. spack install w.r.t an env would add the user spec to the env, concretize it and then build ("install") it. For example, spack install c1:emacs would add emacs to the c1 environment, concretize it, and then install.
spack add adds user specs to an environment. Eg: spack add c1:emacs
spack concretize concretizes one or more user specs. Eg: spack concretize c1: or spack concretize c1:aa
spack build builds ("install") concretized specs in an environment (concretizing them if not already). Eg: spack build myenv:, spack build myenv:aa, spack build myenv:aa:openblas.
Thus, spack install is equal to spack add + spack concretize + spack build.
spack location, spack find, spack load, spack module loads can now be used to find installed packages in a deterministic manner. (Thus, they are no longer "broken" as a reliable way to obtain hashes of stuff you've built and now want to use). Eg:
spack load -r myenv:aa # Loads all modules in myenv:aa DAG
spack location -i myenv::openblas # Finds the (hopefully 1) openblass in myenv
spack module loads -r myenv: # List all modules needed to load myenv
Note that spack activate should not be used with environments. You need to either use spack module loads, or spack view.
Thoughts?