Skip to content

Push-button deployment automation #1772

@AlexanderRichert-NOAA

Description

@AlexanderRichert-NOAA

Is your feature request related to a problem?

Per our 9/9/25 discussion, one key way we can help ourselves reduce spack-stack maintenance workload is to automate deployments as much as possible.

This issue is a place to further discuss needs and wants, especially from the spack-stack maintainers who do the lion's share of the deployments. Below I'll describe one possible general approach, but the key here is to, if possible, find an approach that's suitable for every partner organization that wants to get in on it (for example we could use an approach that makes more use of the existing weekly build-and-test scripting).

A possible solution

Imagine running spack stack deploy, and it will automatically detect which system it's running on, retrieve which installations (template+compiler combinations) should be present on that system (based on, say, config/sites/tier?/*/deployments.yaml), and create and install those environments not yet created. Add-on environments could be deployed either by adding templates (including any associated custom recipes) to the appropriate branch (necessitating a git pull for each deployment), or by storing templates for add-on environments in a separate repository to be accessed as needed by spack stack deploy (therefore no need to git pull into an existing spack-stack directory). Both approaches would give us the agility and flexibility needed to avoid re-release hellck in most cases.

spack stack deploy options would include specifying/excluding certain template+compiler combinations, force re-installing of selected environments, --configure-only to set up but not deploy each environment, and maybe an option for job scheduler-based installations.

A deployments.yaml file for a given site could look like:

core_deployments:
  - template: "unified-dev"
    compilers: ["gcc/12.4.0", "oneapi/2024.2.1"]

addon_deployments:
  - template: "some-pkg-fix-dev"
    compilers: ["oneapi/2024.2.1"]
    upstreams: ["unified-dev"]
  - template: "some-pkg-second-fix-dev"
    compilers: ["oneapi/2024.2.1"]
    upstreams: ["unified-dev", "some-pkg-fix-dev"]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions