Add aws-plcuster[-aarch64] stacks #37627
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| ci: | ||
| pipeline-gen: | ||
| - any-job: | ||
| variables: | ||
| SPACK_TARGET_ARCH: icelake | ||
| - build-job: | ||
| before_script: | ||
| - - curl -LfsS "https://github.com/JuliaBinaryWrappers/GNUMake_jll.jl/releases/download/GNUMake-v4.3.0+1/GNUMake.v4.3.0.x86_64-linux-gnu.tar.gz" -o gmake.tar.gz | ||
| - printf "fef1f59e56d2d11e6d700ba22d3444b6e583c663d6883fd0a4f63ab8bd280f0f gmake.tar.gz" | sha256sum --check --strict --quiet | ||
| - tar -xzf gmake.tar.gz -C /usr bin/make 2> /dev/null | ||
| tags: ["x86_64_v4"] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| ci: | ||
| pipeline-gen: | ||
| - any-job: | ||
| variables: | ||
| SPACK_TARGET_ARCH: neoverse_n1 | ||
| - build-job: | ||
| tags: ["aarch64", "graviton2"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| ci: | ||
| pipeline-gen: | ||
| - any-job: | ||
| variables: | ||
| SPACK_TARGET_ARCH: neoverse_v1 | ||
| - build-job: | ||
| tags: ["aarch64", "graviton3"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| ci: | ||
| pipeline-gen: | ||
| - any-job: | ||
| variables: | ||
| SPACK_TARGET_ARCH: skylake_avx512 | ||
| - build-job: | ||
| before_script: | ||
| - - curl -LfsS "https://github.com/JuliaBinaryWrappers/GNUMake_jll.jl/releases/download/GNUMake-v4.3.0+1/GNUMake.v4.3.0.x86_64-linux-gnu.tar.gz" -o gmake.tar.gz | ||
| - printf "fef1f59e56d2d11e6d700ba22d3444b6e583c663d6883fd0a4f63ab8bd280f0f gmake.tar.gz" | sha256sum --check --strict --quiet | ||
| - tar -xzf gmake.tar.gz -C /usr bin/make 2> /dev/null | ||
| tags: ["x86_64_v4"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| spack: | ||
| view: false | ||
|
|
||
| definitions: | ||
| - compiler_specs: | ||
| - gcc | ||
| - gettext | ||
|
|
||
| - compiler_target: | ||
| - '%[email protected] target=x86_64_v3' | ||
|
Contributor
There was a problem hiding this comment. This doesn't seem consistent with the icelake configs above. Am I missing something?
Contributor
Author
There was a problem hiding this comment. The architecture used is defined by the cluster head node. But clusters may have different architecture compute nodes, so I want the compilers to be as broad as possible. If I made this
Contributor
There was a problem hiding this comment. if that is a concern, then I don't understand why this stack is specifically building for icelake.. |
||
|
|
||
| - optimized_configs: | ||
| # - gromacs | ||
| - lammps | ||
| # - mpas-model | ||
| - openfoam | ||
| # - palace | ||
| # - py-devito | ||
| # - quantum-espresso | ||
| # - wrf | ||
|
|
||
| - optimized_libs: | ||
| - mpich | ||
| - openmpi | ||
|
|
||
| specs: | ||
| - matrix: | ||
| - - $compiler_specs | ||
| - - $compiler_target | ||
| - $optimized_configs | ||
| # - $optimized_libs | ||
|
|
||
| mirrors: { "mirror": "s3://spack-binaries/develop/aws-pcluster-icelake" } | ||
|
|
||
| ci: | ||
| pipeline-gen: | ||
| - build-job: | ||
|
Contributor
There was a problem hiding this comment. It seems like this part could be moved into
Contributor
Contributor
Author
There was a problem hiding this comment. Yes, I can do that. |
||
| image: { "name": "ghcr.io/spack/pcluster-amazonlinux-2:latest", "entrypoint": [""] } | ||
| before_script: | ||
| - - . "./share/spack/setup-env.sh" | ||
| - . /etc/profile.d/modules.sh | ||
| - spack --version | ||
| - spack arch | ||
| # Setup postinstall Spack as upstream installation | ||
| - - if [[ -f /bootstrap/spack/etc/spack/packages.yaml ]]; then cp /bootstrap/spack/etc/spack/packages.yaml ./etc/spack/; fi | ||
| - if [[ -f /bootstrap/spack/etc/spack/config.yaml ]]; then cp /bootstrap/spack/etc/spack/config.yaml ./etc/spack/; fi | ||
| - if [[ -f /bootstrap/spack/etc/spack/modules.yaml ]]; then cp /bootstrap/spack/etc/spack/modules.yaml ./etc/spack/; fi | ||
| - if [[ -f /bootstrap/spack/etc/spack/mirrors.yaml ]]; then cp /bootstrap/spack/etc/spack/mirrors.yaml ./etc/spack/; fi | ||
| - if [[ -d /bootstrap/spack/opt/spack ]]; then spack config add "upstreams:postinstall:install_tree:/bootstrap/spack/opt/spack"; fi | ||
| - - /bin/bash "${SPACK_ARTIFACTS_ROOT}/postinstall.sh" -fg | ||
| - spack config --scope site add "packages:all:target:\"target=${SPACK_TARGET_ARCH}\"" | ||
| - signing-job: | ||
| before_script: | ||
| # Do not distribute Intel & ARM binaries | ||
| - - for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/ | grep intel-oneapi | awk '{print $4}' | sed -e 's?^.*build_cache/??g'); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/$i; done | ||
| - for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/ | grep armpl | awk '{print $4}' | sed -e 's?^.*build_cache/??g'); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/$i; done | ||
|
Contributor
There was a problem hiding this comment. Do I understand this correctly? You are deleting from the mirror some (possibly large) subset of the binaries just built, in order to avoid re-distribution? I thought it was going to be ok to leave these in the stack-specific mirrors, as long as we don't included them at the root. The way it is here, isn't it going to force every matching spec to be rebuilt from source on every pipeline?
Contributor
Author
There was a problem hiding this comment. No, I am only deleting software that's distributed as binaries from Intel/ARM anyway. This is only oneapi and armPL software. On the one hand we are on the save side for re-distribution and on the hand the installation "from source" is downloading pre-built binaries anyway.
Contributor
There was a problem hiding this comment. What I should have said here, but I just didn't think of at the time, is that some time ago we made it an error (see here) for dependencies in a pipeline rebuild job to be installed from source. The definition of "from source" there doesn't consider that it might be downloading pre-built binaries, it's either from a spack buildcache or it's "from source". |
||
| cdash: | ||
| build-group: AWS Packages | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| spack: | ||
| view: false | ||
|
|
||
| definitions: | ||
| - compiler_specs: | ||
|
Contributor
There was a problem hiding this comment. Since this stuff is identical between the stacks, but may be worth while to merge the aarch64 stacks and the x86_64_v4 and use a matrix spec and submappings to assign the runner tags. appropriately. ci:
pipeline-gen:
- match_behavior: first
submapping
- match:
- target=neoverse_n1
build_job:
tags: ["graviton2"]
- match:
- target=neoverse_v1
build_job:
tags: ["graviton3"]I am not sure if both
Contributor
Author
There was a problem hiding this comment. Let me take a look at this after the deadline. It seems I need to learn some more syntax. |
||
| - gcc | ||
kwryankrattiger marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - gettext | ||
|
|
||
| - compiler_target: | ||
| - '%[email protected] target=aarch64' | ||
|
|
||
| - optimized_configs: | ||
| - gromacs | ||
| # - lammps | ||
| # - mpas-model | ||
| - openfoam | ||
| - palace | ||
| # - py-devito | ||
| - quantum-espresso | ||
| # - wrf | ||
|
|
||
| - optimized_libs: | ||
| - mpich | ||
| - openmpi | ||
|
|
||
| specs: | ||
| - matrix: | ||
| - - $compiler_specs | ||
| - - $compiler_target | ||
| - $optimized_configs | ||
| - $optimized_libs | ||
|
|
||
|
|
||
| mirrors: { "mirror": "s3://spack-binaries/develop/aws-pcluster-neoverse_n1" } | ||
|
|
||
| ci: | ||
| pipeline-gen: | ||
| - build-job: | ||
| image: { "name": "ghcr.io/spack/pcluster-amazonlinux-2:latest", "entrypoint": [""] } | ||
| tags: ["aarch64"] | ||
| before_script: | ||
| - - . "./share/spack/setup-env.sh" | ||
| - . /etc/profile.d/modules.sh | ||
| - spack --version | ||
| - spack arch | ||
| # Setup postinstall Spack as upstream installation | ||
| - - if [[ -f /bootstrap/spack/etc/spack/packages.yaml ]]; then cp /bootstrap/spack/etc/spack/packages.yaml ./etc/spack/; fi | ||
| - if [[ -f /bootstrap/spack/etc/spack/config.yaml ]]; then cp /bootstrap/spack/etc/spack/config.yaml ./etc/spack/; fi | ||
| - if [[ -f /bootstrap/spack/etc/spack/modules.yaml ]]; then cp /bootstrap/spack/etc/spack/modules.yaml ./etc/spack/; fi | ||
| - if [[ -f /bootstrap/spack/etc/spack/mirrors.yaml ]]; then cp /bootstrap/spack/etc/spack/mirrors.yaml ./etc/spack/; fi | ||
| - if [[ -d /bootstrap/spack/opt/spack ]]; then spack config add "upstreams:postinstall:install_tree:/bootstrap/spack/opt/spack"; fi | ||
| - - /bin/bash "${SPACK_ARTIFACTS_ROOT}/postinstall.sh" -fg | ||
| - spack config --scope site add "packages:all:target:\"target=${SPACK_TARGET_ARCH}\"" | ||
| - signing-job: | ||
| before_script: | ||
| # Do not distribute Intel & ARM binaries | ||
| - - for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/ | grep intel-oneapi | awk '{print $4}' | sed -e 's?^.*build_cache/??g'); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/$i; done | ||
| - for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/ | grep armpl | awk '{print $4}' | sed -e 's?^.*build_cache/??g'); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/$i; done | ||
|
|
||
| cdash: | ||
| build-group: AWS Packages | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| spack: | ||
| view: false | ||
|
|
||
| definitions: | ||
| - compiler_specs: | ||
| - gcc | ||
| - gettext | ||
|
|
||
| - compiler_target: | ||
| - '%[email protected] target=aarch64' | ||
|
|
||
| - optimized_configs: | ||
| - gromacs | ||
| # - lammps | ||
| # - mpas-model | ||
| - openfoam | ||
| - palace | ||
| # - py-devito | ||
| - quantum-espresso | ||
| # - wrf | ||
|
|
||
| - optimized_libs: | ||
| - mpich | ||
| - openmpi | ||
|
|
||
| specs: | ||
| - matrix: | ||
| - - $compiler_specs | ||
| - - $compiler_target | ||
| - $optimized_configs | ||
| - $optimized_libs | ||
|
|
||
|
|
||
| mirrors: { "mirror": "s3://spack-binaries/develop/aws-pcluster-neoverse_v1" } | ||
|
|
||
| ci: | ||
| pipeline-gen: | ||
| - build-job: | ||
| image: { "name": "ghcr.io/spack/pcluster-amazonlinux-2:latest", "entrypoint": [""] } | ||
| tags: ["aarch64"] | ||
| before_script: | ||
| - - . "./share/spack/setup-env.sh" | ||
| - . /etc/profile.d/modules.sh | ||
| - spack --version | ||
| - spack arch | ||
| # Setup postinstall Spack as upstream installation | ||
| - - if [[ -f /bootstrap/spack/etc/spack/packages.yaml ]]; then cp /bootstrap/spack/etc/spack/packages.yaml ./etc/spack/; fi | ||
| - if [[ -f /bootstrap/spack/etc/spack/config.yaml ]]; then cp /bootstrap/spack/etc/spack/config.yaml ./etc/spack/; fi | ||
| - if [[ -f /bootstrap/spack/etc/spack/modules.yaml ]]; then cp /bootstrap/spack/etc/spack/modules.yaml ./etc/spack/; fi | ||
| - if [[ -f /bootstrap/spack/etc/spack/mirrors.yaml ]]; then cp /bootstrap/spack/etc/spack/mirrors.yaml ./etc/spack/; fi | ||
| - if [[ -d /bootstrap/spack/opt/spack ]]; then spack config add "upstreams:postinstall:install_tree:/bootstrap/spack/opt/spack"; fi | ||
| - - /bin/bash "${SPACK_ARTIFACTS_ROOT}/postinstall.sh" -fg | ||
| - spack config --scope site add "packages:all:target:\"target=${SPACK_TARGET_ARCH}\"" | ||
| - signing-job: | ||
| before_script: | ||
| # Do not distribute Intel & ARM binaries | ||
| - - for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/ | grep intel-oneapi | awk '{print $4}' | sed -e 's?^.*build_cache/??g'); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/$i; done | ||
| - for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/ | grep armpl | awk '{print $4}' | sed -e 's?^.*build_cache/??g'); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/$i; done | ||
|
|
||
| cdash: | ||
| build-group: AWS Packages |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems to me like
icelakeandskylakecould be reduced tox86_64_v4.The stack
spack.yamls can specifiy theSPACK_TARGET_ARCHunder it's own variables.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are right. I did not have a separate stack
spack.yamlwhen I made these files.