Skip to content

Infra Update v8: spack v1 Support#163

Merged
CodeGat merged 14 commits intomainfrom
infra-update-v8
Feb 25, 2026
Merged

Infra Update v8: spack v1 Support#163
CodeGat merged 14 commits intomainfrom
infra-update-v8

Conversation

@CodeGat
Copy link
Copy Markdown
Member

@CodeGat CodeGat commented Nov 18, 2025

References issue ACCESS-NRI/build-cd#313 and PR ACCESS-NRI/build-cd#326
References rollout issue ACCESS-NRI/build-cd#328
References project Spack `0.22` to `1.0` Migration

Note

This infrastructure update created a new Release, but it maintained reproducibility with the current main

Important

This PR is a major update to the deployment infrastructure. See below for the prerequisites for this repository to be able to merge this PR.

Important

This major version change marks the end of major infrastructure updates for deployments to spack < 1.0.
If you want to deploy to instances of spack < 1.0, you must use build-cd < v8.
If you want to deploy to instances of spack >= 1.0, you must use build-cd >= v8.

Background

We are moving on up to spack v1! This update to spack contains many bug fixes, optimisations and new features that can be incorporated into build-cd.

This update also contains changes that make spack v0.X incompatible with this build-cd v8 update - the most prominent one being the splitting of spacks core codebase from it's builtin spack-packages repo. This means that for provenance, one to keep track of both builtin spack-packages and our own, renamed access-spack-packages. build-cd will centrally control the version of builtin spack-packages for a given instance, and config/versions.jsons spack-packages key is renamed to access-spack-packages.

As noted earlier, this major version of build-cd can only support spack v1, due to spack v1-specific commands in the infrastructure. If you still want to deploy to spack v0.X, you will need to change build-cd to v7, and update the config/versions.json file/spack.yaml.

Features

  • Support for spack v1: We're moving to a new, non-beta version of spack! It contains bug fixes, features and optimisations over the old version.

Prerequisites for Merging

  • Update build-cd entrypoints (this PR!)
  • Update config/versions.json with new inputs (also this PR!)
  • Validate compiler additions to the spack manifest (this PR, too!)
  • Test and Verify Model built with spack v1

🚀 The latest prerelease access-om3/pr163-6 at 7af308f is here: #163 (comment) 🚀

@CodeGat CodeGat added the type:infra Related to CI/CD label Nov 18, 2025
@CodeGat CodeGat self-assigned this Nov 18, 2025
@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Jan 5, 2026

Note, this has been rebased to incorporate features from v7

@ACCESS-NRI ACCESS-NRI deleted a comment from github-actions bot Feb 12, 2026
@github-actions
Copy link
Copy Markdown

🚀 Attempted to deploy access-om3 Prerelease pr163-1 with commit 4fdeea7

🖥️ Gadi Deployment ❌

@github-actions
Copy link
Copy Markdown

🚀 Attempted to deploy access-om3 Prerelease pr163-2 with commit 7802a27

🖥️ Gadi Deployment ✔️

Usage Instructions

access-om3, defined in spack.yaml, will be deployed to Gadi as:

  • 2025.08.001 as a Release (when merged).
  • pr163-2 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr163-2

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/1.1/spack in the access-om3-pr163-2 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

If the above was not what was expected, commit changes to config/versions.json in this PR, or propose changes to build-cds config/settings.json.

@CodeGat CodeGat marked this pull request as ready for review February 13, 2026 03:36
@CodeGat CodeGat requested a review from a team as a code owner February 13, 2026 03:36
@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Feb 13, 2026

!update-configs

@github-actions
Copy link
Copy Markdown

🔧 Opening Model Configuration PRs in ACCESS-NRI/access-om3-configs

Configurations Requested

Configurations requested from profile default:

  • dev-MC_100km_jra_ryf

  • dev-MC_100km_jra_ryf+wombatlite

  • dev-MC_25km_jra_ryf

Pull Requests Opened

The following PRs were opened:

More details can be found in the workflow run: https://github.com/ACCESS-NRI/ACCESS-OM3/actions/runs/21973699640

@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Feb 13, 2026

Hmm. Those all failed the test_access_om3_manifest_exe_in_release_spack_location test...but it's a prerelease test 🤔 do we need to rethink that test..?

@anton-seaice
Copy link
Copy Markdown
Collaborator

Thats normal @CodeGat

The repro failure was :

Unable to collect pbs node info: command timed out after 10 seconds

https://github.com/ACCESS-NRI/access-om3-configs/runs/63480848267

see payu-org/payu#688 ?

@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Feb 13, 2026

I've rerun those and unfortunately all those configs fail repro_historical tests 😔 I'll redraft this one for now...it should be equivalent!

@dougiesquire
Copy link
Copy Markdown
Collaborator

Good with me. Thanks for tracking all this down @anton-seaice

@anton-seaice
Copy link
Copy Markdown
Collaborator

anton-seaice commented Feb 25, 2026

Shall I commit to this PR @CodeGat ? I think we should do a new release & deployment for this

@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Feb 25, 2026

Yea go for it - don't forget to update the version number!

@github-actions
Copy link
Copy Markdown

🚀 Attempted to deploy access-om3 Prerelease pr163-3 with commit b8fe682

🖥️ Gadi Deployment ❌

@github-actions
Copy link
Copy Markdown

🚀 Attempted to deploy access-om3 Prerelease pr163-3 with commit bb9f7d4

🖥️ Gadi Deployment ❌

@github-actions
Copy link
Copy Markdown

🚀 Attempted to deploy access-om3 Prerelease pr163-4 with commit 7af308f

🖥️ Gadi Deployment ✔️

Usage Instructions

access-om3, defined in spack.yaml, will be deployed to Gadi as:

  • 2025.08.002 as a Release (when merged).
  • pr163-4 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr163-4

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/1.1/spack in the access-om3-pr163-4 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

If the above was not what was expected, commit changes to config/versions.json in this PR, or propose changes to build-cds config/settings.json.

@anton-seaice
Copy link
Copy Markdown
Collaborator

Soo speedy !

@anton-seaice
Copy link
Copy Markdown
Collaborator

!update-configs

@github-actions
Copy link
Copy Markdown

🔧 Opening/Updating Model Configuration PRs in ACCESS-NRI/access-om3-configs

Configurations Requested

Configurations requested from profile default:

  • dev-MC_100km_jra_ryf

  • dev-MC_100km_jra_ryf+wombatlite

  • dev-MC_25km_jra_ryf

Pull Requests Opened/Updated

The following PRs were opened/updated:

More details can be found in the workflow run: https://github.com/ACCESS-NRI/ACCESS-OM3/actions/runs/22379702097

@anton-seaice
Copy link
Copy Markdown
Collaborator

Weird bug @CodeGat - the update-configs command has tried to use pr163-5, but we are only up to pr163-4 ?

@anton-seaice
Copy link
Copy Markdown
Collaborator

anton-seaice commented Feb 25, 2026

@anton-seaice anton-seaice marked this pull request as ready for review February 25, 2026 04:02
@github-actions
Copy link
Copy Markdown

🚀 Attempted to deploy access-om3 Prerelease pr163-6 with commit 7af308f

🖥️ Gadi Deployment ✔️

Usage Instructions

access-om3, defined in spack.yaml, will be deployed to Gadi as:

  • 2025.08.002 as a Release (when merged).
  • pr163-6 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr163-6

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/1.1/spack in the access-om3-pr163-6 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

If the above was not what was expected, commit changes to config/versions.json in this PR, or propose changes to build-cds config/settings.json.

dougiesquire
dougiesquire previously approved these changes Feb 25, 2026
Copy link
Copy Markdown
Collaborator

@dougiesquire dougiesquire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @CodeGat and @anton-seaice!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:infra Related to CI/CD

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants