Skip to content

Infra Update v6: Modules Injection, More Provenance#134

Merged
CodeGat merged 1 commit intomainfrom
infra-update-v6
Aug 13, 2025
Merged

Infra Update v6: Modules Injection, More Provenance#134
CodeGat merged 1 commit intomainfrom
infra-update-v6

Conversation

@CodeGat
Copy link
Copy Markdown
Member

@CodeGat CodeGat commented Aug 5, 2025 •
edited
Loading

References issue ACCESS-NRI/build-cd#311 and PR ACCESS-NRI/build-cd#306

Important

This PR is a major update to the deployment infrastructure. See below for the prerequisites for this repository to be able to merge this PR.

Background

This update is a bundle of major-level changes.

The main new features include:

  • spack.modules Section is Now Injected At Build Time: A long-standing pain point for developers was having to update spec and dependency versions in multiple places at once. Now, this is injected at build time. Users can still specify their own projections and includes as required (with the others injected automatically to get a working build), but this makes it much simpler. We recommend removing the spack.modules section entirely - it has been done for this PR, but can be reverted if required.

  • Special Packages Now Specified In config/packages.json: Rather than a relatively obscure vars.BUILD_DB_PACKAGES, we now specify special packages for inclusion in the release provenance database in the config/packages.json file (specifically the .provenance field). Furthermore, we can include modulefiles for additional packages from the .injection field.

  • All Schema Now Specified In Workflow Entrypoint Inputs: Schema versions for manifests and files under config/ were kept track on in vars - this was obscure, repo-scoped and untracked. It is now a new, required input to the CI workflow (inputs.*-schema-version), meaning users can pick schema versions for their files at the PR level, and it is now tracked by git. We can also pick a particular manifest schema to test against via the new, optional inputs.spack-manifest-schema-path - useful for manifests that are less restrictive, like the ones for software deployment rather than model deployment.

  • .github/CODEOWNERS File Sets Required Reviewers: Currently only requires that changes to the .github folder require review from @CodeGat. This file can be changed as required by the MDR in this PR, or later. There was talk of adding CODEOWNERS for various spack.yamls.

Prerequisites for Merging

  • Update build-cd entrypoints (this PR!)
  • Add CODEOWNERS file (also this PR!)
  • Add config/packages.json file (again in this PR!)
  • Remove spack.modules section from spack.yaml, revert if there are breaking changes (yep, in this PR)
  • Delete vars.*_SCHEMA_VERSION and vars.BUILD_DB_PACKAGE only when all non-draft PRs have rebased onto this PR
  • Verify that modules are injected as appropriate for this MDR

@CodeGat CodeGat added the type:infra Related to CI/CD label Aug 5, 2025
@CodeGat CodeGat self-assigned this Aug 5, 2025
@anton-seaice
Copy link
Copy Markdown
Collaborator

anton-seaice commented Aug 6, 2025

  • We recommend removing the spack.modules section entirely - it has been done for this PR, but can be reverted if required.

Going forward, we need to include access-ww3 as a module, do we need to set that manually?

access-om3 and access3 modules are found automatically?

@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Aug 6, 2025

No, you don't have to set that manually.

Everything in the config/packages.json file is injected (both the .provenance (for the build database) and .injection (for any additional) packages).

If you want, I can add a no-op commit here once I've got the infrastructure merged into build-cd so you can see what an injected spack.yaml looks like

@anton-seaice
Copy link
Copy Markdown
Collaborator

anton-seaice commented Aug 6, 2025

Oh nice :)

@anton-seaice
Copy link
Copy Markdown
Collaborator

anton-seaice commented Aug 10, 2025

Hi @CodeGat - is this close to ready ? We should be ready for a new om3-deployment this week, shall we finish this PR first ?

@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Aug 11, 2025

I'm just awaiting a review on the build-cd PR, I'll ping Aidan for a review of it

@CodeGat CodeGat marked this pull request as ready for review August 11, 2025 22:21
@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Aug 11, 2025 •
edited
Loading

When running the current spack.yaml through the injection and prerelease scripts (which is done by the CI automatically), we get (probably only need to look at the modules section, but it's all here for posterity):

Modules Injection

spack:
  specs:
  - [email protected]
  packages:
    access3:
      require:
      - '@2025.03.1'
      - configurations=MOM6-CICE6,MOM6-CICE6-WW3
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    access-cice:
      require:
      - '@CICE6.6.0-3'
      - io_type=PIO
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    access-mom6:
      require:
      - '@2025.02.001'
      - +asymmetric_mem
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    access-ww3:
      require:
      - '@2025.03.0'
    access3-share:
      require:
      - '@2025.03.1'
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    access-generic-tracers:
      require:
      - '@git.dev-2025.05.001=development'
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    access-mocsy:
      require:
      - '@git.2017.12.0=gtracers'
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    esmf:
      require:
      - '@git.v8.7.0=8.7.0'
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    parallelio:
      require:
      - '@2.6.2'
    netcdf-c:
      require:
      - '@4.9.2'
      - build_system=cmake build_type=RelWithDebInfo
    netcdf-fortran:
      require:
      - '@4.6.1'
    fms:
      require:
      - '@git.2025.02=2025.02'
    openmpi:
      require:
      - '@4.1.7'
    fortranxml:
      require:
      - '@4.1.2'
    gcc-runtime:
      require:
      - '%gcc'
    all:
      require:
      - '%[email protected]'
      - target=x86_64_v4
  view: true
  concretizer:
    unify: true
  modules:
    default:
      tcl:
        projections:
          access-om3: '{name}/2025.05.001'
          access-cice: '{name}/CICE6.6.0-3-{hash:7}'
          access-mom6: '{name}/2025.02.001-{hash:7}'
          access-ww3: '{name}/2025.03.0-{hash:7}'
          access3: '{name}/2025.03.1-{hash:7}'
          access3-share: '{name}/2025.03.1-{hash:7}'
          openmpi: '{name}/4.1.7-{hash:7}'
        include:
        - access-om3
        - access-cice
        - access-mom6
        - access-ww3
        - access3
        - access3-share
        - openmpi

Prerelease Injection

spack:
  specs:
  - access-om3
  packages:
    access3:
      require:
      - '@2025.03.1'
      - configurations=MOM6-CICE6,MOM6-CICE6-WW3
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    access-cice:
      require:
      - '@CICE6.6.0-3'
      - io_type=PIO
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    access-mom6:
      require:
      - '@2025.02.001'
      - +asymmetric_mem
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    access-ww3:
      require:
      - '@2025.03.0'
    access3-share:
      require:
      - '@2025.03.1'
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    access-generic-tracers:
      require:
      - '@git.dev-2025.05.001=development'
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    access-mocsy:
      require:
      - '@git.2017.12.0=gtracers'
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    esmf:
      require:
      - '@git.v8.7.0=8.7.0'
      - fflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
      - cxxflags="-march=sapphirerapids -mtune=sapphirerapids -unroll"
    parallelio:
      require:
      - '@2.6.2'
    netcdf-c:
      require:
      - '@4.9.2'
      - build_system=cmake build_type=RelWithDebInfo
    netcdf-fortran:
      require:
      - '@4.6.1'
    fms:
      require:
      - '@git.2025.02=2025.02'
    openmpi:
      require:
      - '@4.1.7'
    fortranxml:
      require:
      - '@4.1.2'
    gcc-runtime:
      require:
      - '%gcc'
    all:
      require:
      - '%[email protected]'
      - target=x86_64_v4
  view: true
  concretizer:
    unify: true
  modules:
    default:
      tcl:
        projections:
          access-om3: '{name}/pr123-45'
          access-cice: 'access-om3/dependencies/pr123-45/{name}/CICE6.6.0-3-{hash:7}'
          access-mom6: 'access-om3/dependencies/pr123-45/{name}/2025.02.001-{hash:7}'
          access-ww3: 'access-om3/dependencies/pr123-45/{name}/2025.03.0-{hash:7}'
          access3: 'access-om3/dependencies/pr123-45/{name}/2025.03.1-{hash:7}'
          access3-share: 'access-om3/dependencies/pr123-45/{name}/2025.03.1-{hash:7}'
          openmpi: 'access-om3/dependencies/pr123-45/{name}/4.1.7-{hash:7}'
        include:
        - access-om3
        - access-cice
        - access-mom6
        - access-ww3
        - access3
        - access3-share
        - openmpi

@anton-seaice, how does this look?

@CodeGat CodeGat requested a review from anton-seaice August 11, 2025 22:32
@anton-seaice
Copy link
Copy Markdown
Collaborator

anton-seaice commented Aug 11, 2025

That should work fine, the only modules we need are access-om3, access3 and access-ww3 but the others will be harmless.

anton-seaice
anton-seaice approved these changes Aug 11, 2025
View reviewed changes
Copy link
Copy Markdown
Collaborator

@anton-seaice anton-seaice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good @CodeGat - thanks for the example output !

@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Aug 11, 2025

One last thing:

In this PR, I've removed the modules section from the spack.yaml. This will probably mean that there will be conflicts for the other open PRs once it's merged.

Would you rather I don't remove the modules section, and you can remove it when a release is done? I suppose it's still kicking the can down the road a bit - the open PRs will still work most likely, you'll just have to rebase. I think the !redeploy command will run off v6, which may be unexpected and might not work without the new files in the main branch.

@anton-seaice
Copy link
Copy Markdown
Collaborator

anton-seaice commented Aug 12, 2025

I think the !redeploy command will run off v6, which may be unexpected and might not work without the new files in the main branch.

What do you mean here ? It will work, or it won't work ?

@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Aug 12, 2025

Now that I've thought about it a bit more, it won't work. Because !redeploy is running on v6, which expects certain files in the PR. We then checkout the PR branch and can't find the files, and then it will fail.

@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Aug 13, 2025

Just to close off this PR - Due to the removal of the spack.modules section (as it's injected), it will create a merge conflict on all open PRs, and !redeploy will not work. Open PRs must rebase onto the new main, so they are updated to v6. Again, happy to help with this stuff, just ping me.

@CodeGat CodeGat merged commit c5aab8a into main Aug 13, 2025
@anton-seaice anton-seaice mentioned this pull request Sep 2, 2025
Closed
@minghangli-uni minghangli-uni mentioned this pull request Nov 11, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@anton-seaice anton-seaice anton-seaice approved these changes

Assignees

@CodeGat CodeGat

Labels

type:infra Related to CI/CD

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@CodeGat @anton-seaice