Skip to content

Re enable aws pcluster buildcache stack#38931

Merged
kwryankrattiger merged 40 commits intospack:developfrom
stephenmsachs:re-enable-aws-pcluster-2
Mar 21, 2024
Merged

Re enable aws pcluster buildcache stack#38931
kwryankrattiger merged 40 commits intospack:developfrom
stephenmsachs:re-enable-aws-pcluster-2

Conversation

@stephenmsachs
Copy link
Copy Markdown
Contributor

Fresh PR of #38729 based on latest develop. The intel compiler packages have changed while #38729 was open leaving it in an inconsistent state.

  • Use compilers from pre-installed spack store such that compiler path relocation works when downloading from buildcache.
  • Install gcc from hash so there is no risk of building gcc from source in pipleine.
  • packages.yaml files are now part of the pipelines.
  • No more external postinstall.sh. The necessary steps are in setup-pcluster.sh and will be version controlled within this repo.
  • Re-enable pipelines.

@spackbot-app spackbot-app bot added core PR affects Spack core functionality gitlab Issues related to gitlab integration shell-support labels Jul 17, 2023
@stephenmsachs stephenmsachs force-pushed the re-enable-aws-pcluster-2 branch from 9262e03 to 63bbb56 Compare September 11, 2023 13:34
@stephenmsachs
Copy link
Copy Markdown
Contributor Author

#40561 should fix the pipeline errors.

@stephenmsachs
Copy link
Copy Markdown
Contributor Author

gettext failed at downloading source archive @spackbot re-run pipeline.

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Oct 18, 2023

I've started that pipeline for you!

@stephenmsachs
Copy link
Copy Markdown
Contributor Author

Checks seem stuck @spackbot run pipeline.

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Oct 20, 2023

I'm sorry, gitlab does not have your latest revision yet, I can't run that pipeline for you right now.

One likely possibility is that your PR pipeline has been temporarily deferred, in which case, it is awaiting a develop pipeline, and will be run when that finishes.

Please check the gitlab commit status message to see if more information is available.

Details
pr head: bc9133c, gitlab commit parents: ['67b8dd0', 'fc1d179']

@stephenmsachs stephenmsachs force-pushed the re-enable-aws-pcluster-2 branch from 40e61a9 to ffb6d88 Compare December 12, 2023 08:35
@stephenmsachs stephenmsachs changed the title Re enable aws pcluster Re enable aws pcluster buildcache stack Dec 13, 2023
@stephenmsachs stephenmsachs reopened this Dec 15, 2023
@stephenmsachs stephenmsachs force-pushed the re-enable-aws-pcluster-2 branch from 2d4d1ff to 1d3fe9d Compare December 18, 2023 12:06
@stephenmsachs
Copy link
Copy Markdown
Contributor Author

Re-base might fix codecov issues.

@stephenmsachs
Copy link
Copy Markdown
Contributor Author

stephenmsachs commented Dec 19, 2023

I have added several changes compared to the old version in order to improve the stability of the build pipeline:

  • No more scripts from external repo. Everything that runs in this build pipleine is part of this repo.
  • The GCC used for the stacks is pre-installed into container through gitlab-runners.
  • I am installing the intel compiler from a separate older spack installation here. This means there are no unwanted effects when the intel compiler packages or their dependencies change. The compilers used in these stacks can be considered external to spack.
  • Addressed Merge aws-pcluster pipelines & stacks per generic architecture #37746 by only usig two stacks neoverse_v1 and x86_64_v4.

@stephenmsachs
Copy link
Copy Markdown
Contributor Author

Thanks for the review. I have addressed all your comments.

@kwryankrattiger
Copy link
Copy Markdown
Contributor

@spackbot run pipeline

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Mar 15, 2024

I'm sorry, gitlab does not have your latest revision yet, I can't run that pipeline for you right now.

One likely possibility is that your PR pipeline has been temporarily deferred, in which case, it is awaiting a develop pipeline, and will be run when that finishes.

Please check the gitlab commit status message to see if more information is available.

Details
Unexpected response from gitlab: {'message': '404 Commit Not Found'}

@stephenmsachs
Copy link
Copy Markdown
Contributor Author

Re-base to trigger pipeline.

@stephenmsachs stephenmsachs force-pushed the re-enable-aws-pcluster-2 branch from 6a412f7 to e5ba3e5 Compare March 18, 2024 08:27
@stephenmsachs
Copy link
Copy Markdown
Contributor Author

Closing and re-opening to trigger pipeline.

@stephenmsachs
Copy link
Copy Markdown
Contributor Author

@kwryankrattiger pipeline rebuilt all packages without errors.

@kwryankrattiger kwryankrattiger merged commit aa39465 into spack:develop Mar 21, 2024
mathomp4 pushed a commit to mathomp4/spack that referenced this pull request Mar 27, 2024
* Changes to re-enable aws-pcluster pipelines

- Use compilers from pre-installed spack store such that compiler path relocation works when downloading from buildcache.
- Install gcc from hash so there is no risk of building gcc from source in pipleine.
- `packages.yam` files are now part of the pipelines.
- No more eternal `postinstall.sh`. The necessary steps are in `setup=pcluster.sh` and will be version controlled within this repo.
- Re-enable pipelines.

* Add  and

* Debugging output & mv skylake -> skylake_avx512

* Explicilty check for packages

* Handle case with no intel compiler

* compatibility when using setup-pcluster.sh on a pre-installed cluster.

* Disable palace as parser cannot read require clause at the moment

* ifort cannot build superlu in buildcache

`ifort` is unable to handle such long file names as used when cmake compiles
test programs inside build cache.

* Fix spack commit for intel compiler installation

* Need to fetch other commits before using them

* fix style

* Add TODO

* Update packages.yaml to not use 'compiler:', 'target:' or 'provider:'

Synchronize with changes in https://github.com/spack/spack-configs/blob/main/AWS/parallelcluster/

* Use Intel compiler from later version (orig commit no longer found)

* Use envsubst to deal with quoted newlines

This is cleaner than the `eval` command used.

* Need to fetch tags for checkout on version number

* Intel compiler needs to be from version that has compatible DB

* Install intel compiler with commit that has DB ver 7

* Decouple the intel compiler installation from current commit

- Use a completely different spack installation such that this current pipeline
commit remains untouched.
- Make the script suceed even if the compiler installation fails (e.g. because
the Database version has been updated)
- Make the install targets fall back to gcc in case the compiler did not install
correctly.

* Use generic target for x86_64_vX

There is no way to provision a skylake/icelake/zen runner. They are all in the
same pools under x86_64_v3 and x86_64_v4.

* Find the intel compiler in the current spack installation

* Remove SPACK_TARGET_ARCH

* Fix virtual package index & use package.yaml for intel compiler

* Use only one stack & pipeline per generic architecture

* Fix yaml format

* Cleanup typos

* Include fix for ifx.cfg to get the right gcc toolchain when linking

* [removeme] Adding timeout to debug hang in make (palace)

* Revert "[removeme] Adding timeout to debug hang in make (palace)"

This reverts commit fee8a01.

* palace x86_64_v4 gets stuck when compiling try newer oneapi

* Update comment

* Use the latest container image

* Update gcc_hashes to match new container

* Use only one tag providing tags per extends call

Also removed an unnecessary tag.

* Move generic setup script out of individual stack

* Cleanup from last commit

* Enable checking signature for packages available on the container

* Remove commented packages / Add comment for palace

* Enable openmpi@5 which needs pmix>3

* don't look for intel compiler on aarch64
teaguesterling pushed a commit to teaguesterling/spack that referenced this pull request Jun 15, 2024
* Changes to re-enable aws-pcluster pipelines

- Use compilers from pre-installed spack store such that compiler path relocation works when downloading from buildcache.
- Install gcc from hash so there is no risk of building gcc from source in pipleine.
- `packages.yam` files are now part of the pipelines.
- No more eternal `postinstall.sh`. The necessary steps are in `setup=pcluster.sh` and will be version controlled within this repo.
- Re-enable pipelines.

* Add  and

* Debugging output & mv skylake -> skylake_avx512

* Explicilty check for packages

* Handle case with no intel compiler

* compatibility when using setup-pcluster.sh on a pre-installed cluster.

* Disable palace as parser cannot read require clause at the moment

* ifort cannot build superlu in buildcache

`ifort` is unable to handle such long file names as used when cmake compiles
test programs inside build cache.

* Fix spack commit for intel compiler installation

* Need to fetch other commits before using them

* fix style

* Add TODO

* Update packages.yaml to not use 'compiler:', 'target:' or 'provider:'

Synchronize with changes in https://github.com/spack/spack-configs/blob/main/AWS/parallelcluster/

* Use Intel compiler from later version (orig commit no longer found)

* Use envsubst to deal with quoted newlines

This is cleaner than the `eval` command used.

* Need to fetch tags for checkout on version number

* Intel compiler needs to be from version that has compatible DB

* Install intel compiler with commit that has DB ver 7

* Decouple the intel compiler installation from current commit

- Use a completely different spack installation such that this current pipeline
commit remains untouched.
- Make the script suceed even if the compiler installation fails (e.g. because
the Database version has been updated)
- Make the install targets fall back to gcc in case the compiler did not install
correctly.

* Use generic target for x86_64_vX

There is no way to provision a skylake/icelake/zen runner. They are all in the
same pools under x86_64_v3 and x86_64_v4.

* Find the intel compiler in the current spack installation

* Remove SPACK_TARGET_ARCH

* Fix virtual package index & use package.yaml for intel compiler

* Use only one stack & pipeline per generic architecture

* Fix yaml format

* Cleanup typos

* Include fix for ifx.cfg to get the right gcc toolchain when linking

* [removeme] Adding timeout to debug hang in make (palace)

* Revert "[removeme] Adding timeout to debug hang in make (palace)"

This reverts commit fee8a01.

* palace x86_64_v4 gets stuck when compiling try newer oneapi

* Update comment

* Use the latest container image

* Update gcc_hashes to match new container

* Use only one tag providing tags per extends call

Also removed an unnecessary tag.

* Move generic setup script out of individual stack

* Cleanup from last commit

* Enable checking signature for packages available on the container

* Remove commented packages / Add comment for palace

* Enable openmpi@5 which needs pmix>3

* don't look for intel compiler on aarch64
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core PR affects Spack core functionality gitlab Issues related to gitlab integration shell-support

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants