build-ci v3: Infrastructure Update for spack v1.X#253
Merged
Conversation
CodeGat
added
type:feature
github-project-automation
bot
moved this to New Issues 🌅
in Spack `0.22` to `1.0` Migration
CodeGat
moved this from New Issues 🌅
to In Progress 🏗️
in Spack `0.22` to `1.0` Migration
CodeGat
force-pushed
the
spack-config-81-git-based-spack-packages-repo
branch
from
d207478 to
fbb74a7
Compare
This was referenced Nov 4, 2025
CodeGat
added a commit
to CABLE-LSM/CABLE
that referenced
this pull request
Nov 11, 2025
…nge (#653) # CABLE Thank you for submitting a pull request to the CABLE Project. ## Description We are updating our `build-ci` infrastructure so it will be able to support `spack >= v1.0` more fully. There is an incoming change (see ACCESS-NRI/spack-config#82) to `spack-config` to forward this goal, that will no longer work with `build-ci@v2`. There is a new version of `build-ci@v3` (see ACCESS-NRI/build-ci#253) that will work with this new version of `spack-config`, but it is not yet ready for merging. This interim fix for `build-ci@v2` will work while `build-ci@v3` is being tested, so there is no downtime between versions, by pinning the version of `spack-config` to a tag before the breaking change. > [!NOTE] > Open pull requests will need to be rebased to get this fix, or a similar pinning of `spack-config-ref` in the PR added ## Type of change - [x] Bug fix <!-- readthedocs-preview cable start --> ---- 📚 Documentation preview 📚: https://cable--653.org.readthedocs.build/en/653/ <!-- readthedocs-preview cable end -->
This was referenced Nov 17, 2025
…e exported env vars
… of upstream/runner
Member
Author
|
Regarding 1: Excellent |
CodeGat
force-pushed
the
spack-config-81-git-based-spack-packages-repo
branch
from
aadca03 to
c0fef23
Compare
Member
Author
|
Reviewed your comments @aidanheerdegen - thanks!! |
github-project-automation
bot
moved this from Review 👏
to Done ✅
in Spack `0.22` to `1.0` Migration
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #231
References #250, ACCESS-NRI/access-spack-packages#295
Important
This PR is a major update to the infrastructure, as package-repo-related inputs are being changed. See below for the prerequisites for this repository to be able to merge this PR.
Important
This major version change marks the end of major infrastructure updates for CI using
spack < 1.0. They will still get bug fixes.If you want to deploy to instances of
spack < 1.0, you must usebuild-ci v2.If you want to deploy to instances of
spack >= 1.0, you must usebuild-ci v3.Background
We are migrating over fully to
spack v1.X!In a similar vein to ACCESS-NRI/build-cd#326, this PR moves from keeping track of a single
ACCESS-NRI/spack-packagesrepo; to both aspack/spack-packagesrepo (which we call thebuiltinrepo), and anACCESS-NRI/access-spack-packagesrepo.Importantly, this major version change is a cut off for deployments to
spack < 1.0. If you want to deploy tospack < 1.0, you must use thev2instance of the workflow. This was done so steps in the workflows aren't doing equivalent but different commands based on the version ofspack, eg.spack repo updateforspack 1.0, vsgit -C /opt/access-spack-packages pullinspack 0.22.Just to preempt a potential niche usecase: It is still possible to test across versions of
spackinbuild-ci, but it would require creation of two separate matrices for testing - one forspack < 1.0(AKAbuild-ci@v2) and one forspack >= 1.0(AKAbuild-ci@v3).The PR
Entrypoint Workflows
inputs.spack-packages-ref, addedinputs.builtin-spack-packages-refandinputs.access-spack-packages-ref. This is a major change.inputs.access-spack-packages-reffrommaintoapi-v2, andinputs.spack-reftoreleases/v1.0.builtinandaccessspack-packagesviaspackpost-compiler enable.outputs.spack-packages-sha, replaced withoutputs.builtin-spack-packages-shaandoutputs.access-spack-packages-shaDone in f2fd031
Images
spack-managed package repositories. This is different from thebuild-ci@v2images, which had a staticspack-managedbuiltinthat couldn't be changed, and a path-basedACCESS-NRI/spack-packagesrepo.SPACK_PACKAGESconfigurableARGS. This means that thespack-configref is the sole provider of versions for the package repositories initial values. They can still be updated viainputs.*-spack-packages-ref.SPACK_PACKAGESENVs andLABELs.Done in a09416f
Spack Manifests Used To Build Images
v1images: They are equivalent compilers. Thanks @harshula for this. Done in 4dadf7cMisc. Changes...
spack-checkout-updated-ref. This was created to encapsulate thespack repo updatecommand, in a similar way togit-checkout-updated-ref. Done in 942ed78tools/mcr-changefolder for mass-creation of model component repository PRs for major updates to the infra, similar tobuild-cdstools/mdr-change. Done in d8e7dccTesting
Testing Details In This Collapsed Section
Valid Cases
New
builtinTag, Same (Not Updated)access-spack-packagesBranchbuiltinat https://github.com/ACCESS-NRI/access-test-component/actions/runs/19249735919/job/55031956514#step:9:212access-spack-packagesat https://github.com/ACCESS-NRI/access-test-component/actions/runs/19249735919/job/55031956514#step:10:311New
builtinCommit, Newaccess-spack-packagesCommitbuiltinat https://github.com/ACCESS-NRI/access-test-component/actions/runs/19255532451/job/55049311573#step:9:107access-spack-packagesat https://github.com/ACCESS-NRI/access-test-component/actions/runs/19255532451/job/55049311573#step:10:240, but it took around 40s (2.5x the usual) to update that repository, given that we need to fetch theaccess-spack-packagesrepo manually to get the SHA, so we can pass it to the--commitflag ofspack repo update. Maybe one day they will change it to--ref...No values for
spack-ref/spack-config-ref/builtin-spack-packages-ref/access-spack-packages-refaccess-spack-packages(around 30s!)Invalid Cases
Spack < 1.0
releases/v0.22. See https://github.com/ACCESS-NRI/access-test-component/actions/runs/19281455165?pr=13v0.22.2. See https://github.com/ACCESS-NRI/access-test-component/actions/runs/19281769886?pr=132bfcc69fa870d3c6919be87593f22647981b648a(a commit fromreleases/v0.23, See https://github.com/ACCESS-NRI/access-test-component/actions/runs/19282372546?pr=13access-spack-packagesRun ThroughSee ACCESS-NRI/access-spack-packages#332 - all packages that expect to pass, pass. The other failures are related to attempts to access the Gadi FS.
Requirements for Merging
ACCESS-NRI/spack-packagesas git-based packages repo spack-config#82 to be mergedupstream-v1Pods updatedRequirements for Post-Merge
rocky-v1.0-2025.12.000, as that is where the created image came from