spack environment (3): expose a subset of packages to a user#8231
spack environment (3): expose a subset of packages to a user#8231scheibelp wants to merge 16 commits intospack:developfrom
Conversation
|
To see how I'm "going to town" with environments, look at: (You need to switch to branch efischer/giss) |
|
I am more concerned that this PR gets merged, than I get attribution for it. Once this PR is merged, I will be actively promoting the Spack Setup PR; and once that is done, I will seek to re-do the environment API so it's more integrated into all Spack commands. @scheibelp did the heavy lifting here, with me testing and tucking in around the edges. |
scheibelp
force-pushed
the
features/environments
branch
2 times, most recently
from
0ddecbe to
41cc369
Compare
tgamblin
force-pushed
the
features/environments
branch
2 times, most recently
from
30fc088 to
acdcc29
Compare
SteVwonder
left a comment
SteVwonder
left a comment
There was a problem hiding this comment.
This is an awesome PR! Thanks @citibeth and @scheibelp for putting this together. I will definitely be taking advantage of this once it hits develop. As I was reading through the documentation, I noticed a few typos and figured I'd comment on them.
lib/spack/docs/environments.rst
Outdated
|
|
||
| An environment is used to group together a set of specs for the | ||
| purpose of building, rebuilding and deploying in a coherent fashion. | ||
| Environments provide a number of advantages over the the a la carte |
lib/spack/docs/environments.rst
Outdated
| install``. | ||
| #. An Environment that is built as a whole can be loaded as a whole. | ||
| Spack can generate a script of ``module load`` commands that load | ||
| the appropraite environment. And that script will work, without |
lib/spack/docs/environments.rst
Outdated
| <https://conda.io/docs/user-guide/tasks/manage-environments.html>`_ or | ||
| `Python Virtual Environments | ||
| <https://docs.python.org/3/tutorial/venv.html>`_. Spack environments | ||
| some distinctive features: |
There was a problem hiding this comment.
Spack environments provide some distinctive features
lib/spack/docs/environments.rst
Outdated
| <https://docs.python.org/3/tutorial/venv.html>`_. Spack environments | ||
| some distinctive features: | ||
|
|
||
| #. A spec installed "in" an environment is no different form the same |
| Here we follow a typical use case of creating, concretizing, | ||
| installing and loading an environment. | ||
|
|
||
| .. note:: |
There was a problem hiding this comment.
This doesn't seem to be rendering correctly on GitHub. I'm not familiar with restructured text, so I don't know if that is just GitHub's renderer or if the newlines need to be removed.
lib/spack/docs/environments.rst
Outdated
| The following files may be added to this directory by the user or | ||
| Spack: | ||
|
|
||
| * ``env.yaml``: Addition environment specification and configuration |
scheibelp
force-pushed
the
features/environments
branch
2 times, most recently
from
7439b78 to
3529325
Compare
|
I have rebased this to sync with develop. I plan on looking more into #8231 (review) tomorrow (thanks @SteVwonder!) Edit: it appears that the flake checks have gotten more-strict recently as well and I'll address that too. |
tgamblin
force-pushed
the
features/environments
branch
2 times, most recently
from
ed478e3 to
33bca07
Compare
|
@scheibelp: sorry for not pushing earlier -- I was waiting to do this transactionally. I pushed my changes, which:
This also:
I'll post more in a follow up, but the above is a setup for some refactors I want before we put this in and allow people to rely on the API. Most of them are in line with what @citibeth has requested. I think this is going to look really good! |
Co-authored-by: Elizabeth Fischer <[email protected]>
Co-authored-by: Elizabeth Fischer <[email protected]>
Co-authored-by: Elizabeth Fischer <[email protected]>
- Instead of one method with all parsers, each subcommand gets two functions: `setup_<cmd>_parser()` and `environment_<cmd>()` - the `setup_parser()` and `env()` functions now generate the parser based on these and a list of subcommands. - it is now easier to associate the arguments with the subcommand.
- `spack.util.environment` is the new home for routines that modify environment variables. - This is to make room for `spack.environment` to contain new routines for dealing with spack environments
- `spack.environment` is now the home for most of the infrastructure around Spack environments - refactor `cmd/env.py` to use everything from spack.environment - refactor the cmd/env test to use pytest and fixtures
…ther - logic used in `spack find` was hiding duplicate installations if their hashes were different - short hash doesn't work in this scenario, since specs are structurally identical - ConstraintAction always works on a DB query, so use the DAG hash to ensure uniqueness
- add a common argument for `-e/--env` - modify the database to support queries on subsets of hashes - allow `spack find` to be filtered by hashes in an environment
tgamblin
force-pushed
the
features/environments
branch
from
0af3375 to
438737b
Compare
|
Superseded by #9612 |
7 participants
This is the same as #7843 but with factored commits. The attribution is incorrect on them (I need to use https://help.github.com/articles/creating-a-commit-with-multiple-authors/ to add @citibeth as a coauthor on most of these).
@tgamblin ready for review. In short this PR:
spack envcommand: This maintains an environment object which tracks a set of specs, a spack configuration, and a repository. The userspack envcommand tospack build-envspack installcommand options apply tospack env foo installSpec.to_node_dictto preserve patches/concreteness, so that a spec is concrete & complete after converting to/from JSONUnfortunately since this replaces the
cmd/env.pyfile, the github diff presents this as if I had edited the file from it's original state (vs. actually I renamedenv.pyand started from scratch). That is to say: it would be easier to just readcmd/env.pyin an editor since all of its content is new.