[feature/update_to_spack_v1] Update NRL batch install script and NRL site configs; add flag to 'spack stack create env' to promote warnings to errors; updates from develop and release/1.9.0 branches by climbfuji · Pull Request #1798 · JCSDA/spack-stack

climbfuji · 2025-10-23T17:23:07Z

Description

Add argument --treat-warnings-as-errors for spack stack create env so that warnings for non-existent packages_*.yaml are promoted to errors. The default remains unchanged - they remain warnings.
Update NRL batch install script for spack v1 updates (mostly due to site config updates).
By default, do not create spack compiler modules (not to be confused with meta modules, we definitely do want those by default), use the native compiler modules on HPCs. For user site configs and CI, explicitly add compiler modules to module includelist in configs/sites/**/modules.yaml.
Bring in updates from spack-stack develop (which are updates from the latest release 1.9.3): site configs and wgrib2 variants. All site config updates except NRL sites are from this step.
Remove explicit variants for ewok-env in configs/templates/skylab-dev/spack.yaml and configs/templates/unified-dev/spack.yaml, since they are the default values. This allows us to turn off the ecflow variant in the site configs (c.f. Atlantis).
Add missing export of environment variables from MPI packages in MPI meta modules setup-meta-modules).

Dependencies

Waiting on Bug fixes for ectrans, py-kiwisolver, py-cartopy, mct spack-packages#12

Issues addressed

Working towards #1604

Applications affected

None

Systems affected

NRL systems

Testing

CI: Note whether the automatic tests (GitHub actions tests that run automatically for every commit) pass or not
- GitHub actions CI tests pass
- GitHub actions CI tests do not pass (provide explanation)
- GitHub actions CI tests skipped (provide explanation if necessary)
New tests added: List and describe any new tests added to GitHub actions
- ...
Additional testing: Add information on any additional tests conducted
- ...

Checklist

This PR addresses one issue/problem/enhancement or has a very good reason for not doing so.
These changes have been tested on the affected systems and applications.
All dependency PRs/issues have been resolved and this PR can be merged.
~~All necessary updates to the documentation on readthedocs are included in this PR~~ deferred
- For site config updates, check in particular doc/source/PreConfiguredSites.rst and doc/source/MaintainersSection.rst
~~All necessary updates to the spack-stack wiki will be made when this PR is merged~~ deferred

…pack-stack to new api version 2 under repos/spack_stack

…ubmodule pointer for spack

…ib instead of zlib-ng for zlib-api

…associated templates

…antis/packages.yaml

…lmod

…pack_stack/spack_repo/spack_stack/packages/neptune_env/package.py, repos/spack_stack/spack_repo/spack_stack/packages/neptune_python_env/package.py

…thub.com/climbfuji/spack-stack into feature/update_from_spack_dev_3b34f964da

…/meta_modules.py

…s COMPILER_TRANSLATION_TABLE

…/spack_repo/spack_stack/packages/neptune_python_env/package.py

…es/neptune_python_env/package.py

…k/stack_env.py

…s with [email protected]

…/spack.yaml and configs/templates/unified-dev/spack.yaml, since they are the defaults

…/spack-stack into feature/nrl_batch_install_spack_v1

…r bounty

…om/climbfuji/spack-stack into feature/nrl_batch_install_spack_v1

configs/sites/tier1/blueback/packages.yaml

util/nrl/batch_install.sh

…om/climbfuji/spack-stack into feature/nrl_batch_install_spack_v1

…Intel LLVM support

climbfuji added 30 commits July 22, 2025 09:03

Add spack builtin repo as repos/builtin and migrate spack-ext/repos/s…

ad9bd39

…pack-stack to new api version 2 under repos/spack_stack

Remove logic to register repos and extensions from setup.sh, update s…

0740e30

…ubmodule pointer for spack

Update .gitmodules and submodule pointer for spack

919f01c

Update submodule pointers for repos/builtin and spack

383b63d

configs/common/packages.yaml: avoid using rust-boostrap, switch to zl…

02e3cee

…ib instead of zlib-ng for zlib-api

Saving progress

d356c73

Save progress for configs/**

7254b6b

Update spack-ext/lib/jcsda-emc/spack-stack/stack/meta_modules.py and …

e355a02

…associated templates

Update submodule pointer for repos/builtin

09e1236

[skip ci] Work in progress: common config for gcc and oneapi

399ed30

[skip ci] Atlantis: packages_gcc.yaml -> packages_gcc-13.4.0.yaml

7680f0a

[skip ci] update configs/templates/neptune-dev/spack.yaml

2832594

Bug fixes in configs/common/packages.yaml and configs/sites/tier1/atl…

16587c2

…antis/packages.yaml

Update spack-ext/lib/jcsda-emc/spack-stack/stack/meta_modules.py for …

04a10e7

…lmod

Update configs/sites/tier1/atlantis/packages_gcc-13.4.0.yaml, repos/s…

f009c4b

…pack_stack/spack_repo/spack_stack/packages/neptune_env/package.py, repos/spack_stack/spack_repo/spack_stack/packages/neptune_python_env/package.py

Merge branch 'feature/update_from_spack_dev_3b34f964da' of https://gi…

794a936

…thub.com/climbfuji/spack-stack into feature/update_from_spack_dev_3b34f964da

Incremental improvements in spack-ext/lib/jcsda-emc/spack-stack/stack…

0ee6d34

…/meta_modules.py

Incremental steps towards a better setup-meta-modules extension

f6f795a

Clean up configs/common/packages*.yaml

37bde5c

Update site config for Bounty

9bd24b6

Update configs/common/{modules_lmod.yaml,modules_tcl.yaml,packages.yaml}

9ea3d0a

spack-ext/lib/jcsda-emc/spack-stack/stack/meta_modules.py: use spack'…

eb0a254

…s COMPILER_TRANSLATION_TABLE

Temporarily comment out most Python dependencies in repos/spack_stack…

4d7f82e

…/spack_repo/spack_stack/packages/neptune_python_env/package.py

Add py-cython to module exclude list

e2063a5

configs/common/packages.yaml: update boost and netcdf-c

9fb97ec

Remove unnecessary entries in configs/common/packages_oneapi.yaml

0649cc3

Update configs/** and repos/spack_stack/spack_repo/spack_stack/packag…

5b0341f

…es/neptune_python_env/package.py

Remove filter_compilers from spack-ext/lib/jcsda-emc/spack-stack/stac…

4d4e671

…k/stack_env.py

configs/common/packages.yaml: Update so that neptune-python-env build…

9678b8c

…s with [email protected]

Add jedi-neptune-env and crtm to configs/templates/neptune-dev

cfecd57

climbfuji mentioned this pull request Oct 28, 2025

Bug fixes for ectrans, py-kiwisolver, py-cartopy, mct JCSDA/spack-packages#12

Merged

4 tasks

climbfuji added 4 commits October 28, 2025 14:22

Revert .gitmodules and update submodule pointer for repos/builtin

304b3b3

[skip ci] Bug fixes in util/nrl/batch_install.sh for Nautilus/GCC

49cb5da

Remove explicit variants for ewok-env in configs/templates/skylab-dev…

c99137a

…/spack.yaml and configs/templates/unified-dev/spack.yaml, since they are the defaults

Turn off ecflow variant for ewok-env on Atlantis

2bd62fa

This was referenced Oct 29, 2025

repos/builtin/ submodule update (py-mpi4py) #1802

Closed

fetch_cargo_deps.py: ensure CARGO_HOME is used #1804

Merged

climbfuji added 8 commits October 30, 2025 05:28

Merge branch 'feature/update_to_spack_v1' of https://github.com/jcsda…

ee33413

…/spack-stack into feature/nrl_batch_install_spack_v1

[skip ci] Updates for neptune-ops in batch_install script, updates fo…

2663683

…r bounty

Update util/nrl/batch_install.sh: blackpearl

2c060d2

Write MPI environment variables from packages_*.yaml to MPI meta modules

669e0d0

Don't use external zlib on blackpearl or bounty

ee8c433

Clean up Narwhal site config

a48fb38

Merge branch 'feature/nrl_batch_install_spack_v1' of https://github.c…

d1a3fff

…om/climbfuji/spack-stack into feature/nrl_batch_install_spack_v1

Update submodule pointer for repos/builtin

6816cbe

climbfuji commented Oct 31, 2025

View reviewed changes

configs/sites/tier1/blueback/packages.yaml Outdated Show resolved Hide resolved

util/nrl/batch_install.sh Outdated Show resolved Hide resolved

util/nrl/batch_install.sh Outdated Show resolved Hide resolved

Apply suggestions from code review

bca7082

climbfuji marked this pull request as ready for review October 31, 2025 14:14

climbfuji requested review from AlexanderRichert-NOAA, eap, mathomp4 and rickgrubin-noaa October 31, 2025 14:15

climbfuji added 2 commits November 3, 2025 09:56

Merge branch 'feature/nrl_batch_install_spack_v1' of https://github.c…

8e0d3fa

…om/climbfuji/spack-stack into feature/nrl_batch_install_spack_v1

Update w3emc from 2.10.0 to 2.11.0 in configs/common/packages.yaml - …

903096e

…Intel LLVM support

AlexanderRichert-NOAA approved these changes Nov 3, 2025

View reviewed changes

mathomp4 approved these changes Nov 3, 2025

View reviewed changes

climbfuji enabled auto-merge (squash) November 3, 2025 20:35

climbfuji merged commit 687d84e into JCSDA:feature/update_to_spack_v1 Nov 3, 2025
7 checks passed

github-project-automation bot moved this from In Progress to Done in spack-stack-2.0.x (2025 Q4) Nov 3, 2025

climbfuji deleted the feature/nrl_batch_install_spack_v1 branch November 3, 2025 21:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feature/update_to_spack_v1] Update NRL batch install script and NRL site configs; add flag to 'spack stack create env' to promote warnings to errors; updates from develop and release/1.9.0 branches#1798

climbfuji commented Oct 23, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

climbfuji commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Issues addressed

Applications affected

Systems affected

Testing

Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

climbfuji commented Oct 23, 2025 •

edited

Loading