[feature/update_to_spack_v1] External zlib, external runtime libraries, MPI meta-modules updates, NRL site config updates round 2 of X by climbfuji · Pull Request #1782 · JCSDA/spack-stack

climbfuji · 2025-09-30T14:24:29Z

Description

This PR is for branch feature/update_to_spack_v1.

spack-packages submodule update for Intel oneAPI FPP bug affecting NASA packages, to allow for external Intel oneAPI runtime libraries, for ESMF bug (linking to MPI C++ libraries)
Numerous additional spack-packages updates from the authoritative repo (contributed by @AlexanderRichert-NOAA)
Return control over zlib libraries to site configs (can be external or spack-built)
Return control over runtime libraries to site configs (can be external or spack-built)
Updates to MPI meta-modules: set correct compiler wrapper and backend environment variables, set additional environment variables that define the MPI install prefix
NRL site config updates, second round (more to come in a follow-up PR)

Dependencies

Issues addressed

Working towards #1604
Working towards #771 (NRL platforms)

Applications affected

Potentially all

Systems affected

Potentially all, definitely NRL systems

Testing

CI: Note whether the automatic tests (GitHub actions tests that run automatically for every commit) pass or not
- GitHub actions CI tests pass
- GitHub actions CI tests do not pass (provide explanation)
- GitHub actions CI tests skipped (provide explanation if necessary)
New tests added: List and describe any new tests added to GitHub actions
- ...
Additional testing: Add information on any additional tests conducted
- Built environments and ran NEPTUNE against them on Blueback, Bounty with GNU, LLVM, oneAPI

Checklist

This PR addresses one issue/problem/enhancement or has a very good reason for not doing so.
These changes have been tested on the affected systems and applications.
All dependency PRs/issues have been resolved, and this PR can be merged.
All necessary updates to the documentation on readthedocs are included in this PR
- For site config updates, check in particular doc/source/PreConfiguredSites.rst and doc/source/MaintainersSection.rst
~~All necessary updates to the spack-stack wiki will be made when this PR is merged~~ deferred

…pack-stack to new api version 2 under repos/spack_stack

…ubmodule pointer for spack

…ib instead of zlib-ng for zlib-api

…associated templates

…antis/packages.yaml

…lmod

…pack_stack/spack_repo/spack_stack/packages/neptune_env/package.py, repos/spack_stack/spack_repo/spack_stack/packages/neptune_python_env/package.py

…thub.com/climbfuji/spack-stack into feature/update_from_spack_dev_3b34f964da

…/meta_modules.py

…s COMPILER_TRANSLATION_TABLE

…/spack_repo/spack_stack/packages/neptune_python_env/package.py

…es/neptune_python_env/package.py

…k/stack_env.py

…s with [email protected]

configs/common/packages_oneapi.yaml

spack-ext/lib/jcsda-emc/spack-stack/stack/meta_modules.py

…1/ursa/packages_oneapi.yaml for building unified environment on Ursa for spack-stack develop as of 2025/10/02 (JCSDA#1787)

… insert missing 'bin' in MPI compiler wrapper paths

…eature/update_from_spack_dev_3b34f964da_ADDON_DOM_from_20250923

…eview and testing

…te config

climbfuji · 2025-10-09T11:48:21Z

@AlexanderRichert-NOAA Would you be able to give this another look so that I merge? The vast majority of the changes are NRL site config updates. Would be really good if we could get this in before I go on leave for the entire next week. Thanks!

mathomp4 · 2025-10-09T18:22:42Z

I approved in that my current testing at NAS is "above" this. At the moment I have my MPT fix (spack/spack-packages#1430 which I'll eventually have to bring in) and then NAS specific updates.

eap

Thanks for this update.

climbfuji added 30 commits July 22, 2025 09:03

Add spack builtin repo as repos/builtin and migrate spack-ext/repos/s…

ad9bd39

…pack-stack to new api version 2 under repos/spack_stack

Remove logic to register repos and extensions from setup.sh, update s…

0740e30

…ubmodule pointer for spack

Update .gitmodules and submodule pointer for spack

919f01c

Update submodule pointers for repos/builtin and spack

383b63d

configs/common/packages.yaml: avoid using rust-boostrap, switch to zl…

02e3cee

…ib instead of zlib-ng for zlib-api

Saving progress

d356c73

Save progress for configs/**

7254b6b

Update spack-ext/lib/jcsda-emc/spack-stack/stack/meta_modules.py and …

e355a02

…associated templates

Update submodule pointer for repos/builtin

09e1236

[skip ci] Work in progress: common config for gcc and oneapi

399ed30

[skip ci] Atlantis: packages_gcc.yaml -> packages_gcc-13.4.0.yaml

7680f0a

[skip ci] update configs/templates/neptune-dev/spack.yaml

2832594

Bug fixes in configs/common/packages.yaml and configs/sites/tier1/atl…

16587c2

…antis/packages.yaml

Update spack-ext/lib/jcsda-emc/spack-stack/stack/meta_modules.py for …

04a10e7

…lmod

Update configs/sites/tier1/atlantis/packages_gcc-13.4.0.yaml, repos/s…

f009c4b

…pack_stack/spack_repo/spack_stack/packages/neptune_env/package.py, repos/spack_stack/spack_repo/spack_stack/packages/neptune_python_env/package.py

Merge branch 'feature/update_from_spack_dev_3b34f964da' of https://gi…

794a936

…thub.com/climbfuji/spack-stack into feature/update_from_spack_dev_3b34f964da

Incremental improvements in spack-ext/lib/jcsda-emc/spack-stack/stack…

0ee6d34

…/meta_modules.py

Incremental steps towards a better setup-meta-modules extension

f6f795a

Clean up configs/common/packages*.yaml

37bde5c

Update site config for Bounty

9bd24b6

Update configs/common/{modules_lmod.yaml,modules_tcl.yaml,packages.yaml}

9ea3d0a

spack-ext/lib/jcsda-emc/spack-stack/stack/meta_modules.py: use spack'…

eb0a254

…s COMPILER_TRANSLATION_TABLE

Temporarily comment out most Python dependencies in repos/spack_stack…

4d7f82e

…/spack_repo/spack_stack/packages/neptune_python_env/package.py

Add py-cython to module exclude list

e2063a5

configs/common/packages.yaml: update boost and netcdf-c

9fb97ec

Remove unnecessary entries in configs/common/packages_oneapi.yaml

0649cc3

Update configs/** and repos/spack_stack/spack_repo/spack_stack/packag…

5b0341f

…es/neptune_python_env/package.py

Remove filter_compilers from spack-ext/lib/jcsda-emc/spack-stack/stac…

4d4e671

…k/stack_env.py

configs/common/packages.yaml: Update so that neptune-python-env build…

9678b8c

…s with [email protected]

Add jedi-neptune-env and crtm to configs/templates/neptune-dev

cfecd57

climbfuji commented Oct 2, 2025

View reviewed changes

configs/common/packages_oneapi.yaml Outdated Show resolved Hide resolved

climbfuji marked this pull request as ready for review October 2, 2025 19:34

climbfuji requested review from AlexanderRichert-NOAA, eap, mathomp4 and rickgrubin-noaa October 2, 2025 19:34

AlexanderRichert-NOAA reviewed Oct 2, 2025

View reviewed changes

spack-ext/lib/jcsda-emc/spack-stack/stack/meta_modules.py Outdated Show resolved Hide resolved

climbfuji added 4 commits October 2, 2025 20:02

Update configs/sites/tier1/ursa/compilers.yaml and configs/sites/tier…

0d908a4

…1/ursa/packages_oneapi.yaml for building unified environment on Ursa for spack-stack develop as of 2025/10/02 (JCSDA#1787)

Bug fix in spack-ext/lib/jcsda-emc/spack-stack/stack/meta_modules.py:…

e79f10b

… insert missing 'bin' in MPI compiler wrapper paths

Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…

1fc9e04

…eature/update_from_spack_dev_3b34f964da_ADDON_DOM_from_20250923

Update submodule pointer for repos/builtin

4d47ebe

climbfuji requested a review from AlexanderRichert-NOAA October 3, 2025 16:32

climbfuji mentioned this pull request Oct 6, 2025

ESMF: workaround for legacy linking to MPI C++ interface JCSDA/spack-packages#7

Merged

climbfuji added 9 commits October 6, 2025 13:33

Update .gitmodules and submodule pointer for repos/builtin for code r…

e0b0502

…eview and testing

Update Nautilus GCC site config and documentation

d100362

Update submodule pointer for repos/builtin

010ae82

Revert .gitmodules and update submodule pointer for repos/builtin

c5ae580

Remove comment in configs/common/packages_oneapi.yaml

3dc2e15

Update submodule pointer for repos/builtin

d5ad23e

Remove repos/spack-stack-dev-ref

df5e5ba

[skip ci] Update Narwhal site config (untested), clean up Blueback si…

4e94cb7

…te config

Update submodule pointer for repos/builtin

351a337

mathomp4 approved these changes Oct 9, 2025

View reviewed changes

rickgrubin-noaa approved these changes Oct 9, 2025

View reviewed changes

eap approved these changes Oct 9, 2025

View reviewed changes

climbfuji merged commit ef32e8e into JCSDA:feature/update_to_spack_v1 Oct 9, 2025
7 checks passed

github-project-automation bot moved this from In Progress to Done in spack-stack-2.0.x (2025 Q4) Oct 9, 2025

climbfuji deleted the feature/update_from_spack_dev_3b34f964da_ADDON_DOM_from_20250923 branch October 9, 2025 22:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feature/update_to_spack_v1] External zlib, external runtime libraries, MPI meta-modules updates, NRL site config updates round 2 of X#1782

[feature/update_to_spack_v1] External zlib, external runtime libraries, MPI meta-modules updates, NRL site config updates round 2 of X#1782
climbfuji merged 164 commits intoJCSDA:feature/update_to_spack_v1from
climbfuji:feature/update_from_spack_dev_3b34f964da_ADDON_DOM_from_20250923

climbfuji commented Sep 30, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

climbfuji commented Oct 9, 2025

Uh oh!

mathomp4 commented Oct 9, 2025

Uh oh!

eap left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

climbfuji commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Issues addressed

Applications affected

Systems affected

Testing

Checklist

Uh oh!

Uh oh!

Uh oh!

climbfuji commented Oct 9, 2025

Uh oh!

mathomp4 commented Oct 9, 2025

Uh oh!

eap left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

climbfuji commented Sep 30, 2025 •

edited

Loading