-
Notifications
You must be signed in to change notification settings - Fork 54
Push-button deployment automation #1772
Description
Is your feature request related to a problem?
Per our 9/9/25 discussion, one key way we can help ourselves reduce spack-stack maintenance workload is to automate deployments as much as possible.
This issue is a place to further discuss needs and wants, especially from the spack-stack maintainers who do the lion's share of the deployments. Below I'll describe one possible general approach, but the key here is to, if possible, find an approach that's suitable for every partner organization that wants to get in on it (for example we could use an approach that makes more use of the existing weekly build-and-test scripting).
A possible solution
Imagine running spack stack deploy, and it will automatically detect which system it's running on, retrieve which installations (template+compiler combinations) should be present on that system (based on, say, config/sites/tier?/*/deployments.yaml), and create and install those environments not yet created. Add-on environments could be deployed either by adding templates (including any associated custom recipes) to the appropriate branch (necessitating a git pull for each deployment), or by storing templates for add-on environments in a separate repository to be accessed as needed by spack stack deploy (therefore no need to git pull into an existing spack-stack directory). Both approaches would give us the agility and flexibility needed to avoid re-release hellck in most cases.
spack stack deploy options would include specifying/excluding certain template+compiler combinations, force re-installing of selected environments, --configure-only to set up but not deploy each environment, and maybe an option for job scheduler-based installations.
A deployments.yaml file for a given site could look like:
core_deployments:
- template: "unified-dev"
compilers: ["gcc/12.4.0", "oneapi/2024.2.1"]
addon_deployments:
- template: "some-pkg-fix-dev"
compilers: ["oneapi/2024.2.1"]
upstreams: ["unified-dev"]
- template: "some-pkg-second-fix-dev"
compilers: ["oneapi/2024.2.1"]
upstreams: ["unified-dev", "some-pkg-fix-dev"]