Hi Spack Devs!

Background

We currently use spack to test builds of Climate Models and their components. Specifically, we have GitHub Actions infrastructure that builds various test manifests (different variants, versions of dependencies, etc) of climate model components on Pull Request. We use a binary cache to store intermediate dependencies so the infrastructure doesn't take too long installing everything, every time.

In some cases, components use a @git.BRANCH version specifier, or a @stable infinity version (that is actually a branch in the components spack package recipe). If changes are done to the referred branch, they are picked up in the concretization graph - specifically the reserved commit=XXX variant that was added in spack v1.0. However, the package hash resolves to the same value, and hence the binary in the buildcache is used, rather than reinstalling using the new git ref.

Concrete Example

For example, in our https://github.com/ACCESS-NRI/MOM6 repository (A component of a larger ACCESS-OM3 model), we are testing out various dependency versions and variants of MOM6. One part continued to be an issue.

We are testing that the dependency access3-share at version @stable (an infinity version that is a git branch) is rebuilt if there are changes to the linked branch of that dependency. We are currently using releases/v1.0 spack.

A workflow run before a change to the branch linked to the @stable version, the concretization was:

  -   hdbt3sf          ^access3-share@develop~ipo~openmp build_system=cmake build_type=Release commit=f2f35ce5915e82a83899b69560d826deab53b668 generator=make arch=linux-rocky8-x86_64_v3 %c,[email protected]

A workflow run after a change to the branch linked to the @stable version, the concretization was:

  -   hdbt3sf          ^access3-share@develop~ipo~openmp build_system=cmake build_type=Release commit=33c5bd243a7ca2263fabd1330429816de4ff2426 generator=make arch=linux-rocky8-x86_64_v3 %c,[email protected]

Note the different reserved commit=XXX variants, but the same package hash hdbt3sf. This leads to the package being taken from the binary cache, when it actually needs to be rebuilt due to the linked branch being updated. Can the commit=XXX variant be used when computing the package hash?

See ACCESS-NRI/MOM6#24 for more tests, and ACCESS-NRI#4 (comment) for another write up.

Linking ACCESS-NRI#4 for our own tracking of this issue, too. @harshula