Skip to content

[DO NOT MERGE] HPCToolkit software stacks#31598

Closed
blue42u wants to merge 54 commits intospack:developfrom
blue42u:hpctoolkit-stack
Closed

[DO NOT MERGE] HPCToolkit software stacks#31598
blue42u wants to merge 54 commits intospack:developfrom
blue42u:hpctoolkit-stack

Conversation

@blue42u
Copy link
Copy Markdown
Contributor

@blue42u blue42u commented Jul 15, 2022

Experimenting with Spack CI for HPCToolkit, don't mind me.

@spackbot-app spackbot-app bot added the gitlab Issues related to gitlab integration label Jul 15, 2022
@blue42u blue42u force-pushed the hpctoolkit-stack branch from 6bac87c to a026f81 Compare July 16, 2022 05:23
@spackbot-app spackbot-app bot added the core PR affects Spack core functionality label Jul 16, 2022
@blue42u blue42u force-pushed the hpctoolkit-stack branch from fc25ab8 to 250b871 Compare July 16, 2022 16:21
@blue42u blue42u force-pushed the hpctoolkit-stack branch 3 times, most recently from bdf7ce5 to a2ee6ce Compare July 21, 2022 01:21
@blue42u blue42u force-pushed the hpctoolkit-stack branch from da93365 to 59d3f5f Compare July 26, 2022 03:12
@blue42u blue42u force-pushed the hpctoolkit-stack branch 2 times, most recently from f0c84f7 to 22bca6d Compare July 26, 2022 16:47
@blue42u blue42u force-pushed the hpctoolkit-stack branch 8 times, most recently from 4196bd2 to 49b7b52 Compare August 5, 2022 14:15
@blue42u blue42u force-pushed the hpctoolkit-stack branch 8 times, most recently from a27a1cd to 69f7467 Compare August 19, 2022 15:15
blue42u and others added 23 commits August 21, 2022 18:47
Apparently concretizer:targets:granularity:generic doesn't work well
when you explicitly set the target= variant. But only on ARM?

Also reformat the specs list to make single-element lists more obvious
Not sure why the specific arch refuses to concretize in CI...
This should fix the Git failures in CentOS 7
Let's see if this even works...
Repeated, somehow it got lost in the shuffle just now
Apparently definitions: entries are additive, not overwrites.
I didn't realize that. Whoops.
Until I find out how to access the runners for it
@blue42u
Copy link
Copy Markdown
Contributor Author

blue42u commented Sep 6, 2022

This PR has served its purpose and has been superseded by #32327.

@blue42u blue42u closed this Sep 6, 2022
scheibelp pushed a commit that referenced this pull request Oct 15, 2022
Currently "spack ci generate" chooses the first matching entry in
gitlab-ci:mappings to fill attributes for a generated build-job,
requiring that the entire configuration matrix is listed out
explicitly. This unfortunately causes significant problems in
environments with large configuration spaces, for example the
environment in #31598 (spack.yaml) supports 5 operating systems,
3 architectures and 130 packages with explicit size requirements,
resulting in 1300 lines of configuration YAML.

This patch adds a configuraiton option to the gitlab-ci schema called
"match_behavior"; when it is set to "merge", all matching entries
are applied in order to the final build-job, allowing a few entries
to cover an entire matrix of configurations.

The default for "match_behavior" is "first", which behaves as before
this commit (only the runner attributes of the first match are used).

In addition, match entries may now include a "remove-attributes"
configuration, which allows matches to remove tags that have been
aggregated by prior matches. This only makes sense to use with
"match_behavior:merge". You can combine "runner-attributes" with
"remove-attributes" to effectively override prior tags.
luke-dt added a commit to dantaslab/spack that referenced this pull request Oct 17, 2022
* py-execnet: 1.9.0 (spack#33282)

* py-execnet: 1.9.0

* bounds

* add version 1.12.3 of parallel-netcdf (spack#33286)

* Add checksum for py-psutil 5.9.2 (spack#33139)

* gitlab ci: Print better information about broken specs (spack#33124)

When a pipeline generation job is automatically failed because it
generated jobs for specs known to be broken on develop, print better
information about the broken specs that were encountered.  Include
at a minimum the hash and the url of the job whose failure caused it
to be put on the broken specs list in the first place.

* meson: remove slash in path (spack#33292)

* Add checksum for py-gitpython 3.1.27 (spack#33285)

* Add checksum for py-gitpython 3.1.27

* Update package.py

* Update var/spack/repos/builtin/packages/py-gitpython/package.py

Co-authored-by: Adam J. Stewart <[email protected]>

Co-authored-by: Adam J. Stewart <[email protected]>

* UPC++/GASNet-EX 2022.9.0 update (spack#33277)

* gasnet: Add new release hash
* upcxx: Add new release hash
* gasnet: misc updates
* upcxx: misc updates

* Fix [email protected] sha (spack#33307)

* [email protected] onwards: set prefix properly (spack#33257)

* hip-set-prefix-rocm5.2.0-onwards

* Update var/spack/repos/builtin/packages/hip/package.py

Update description

Co-authored-by: Satish Balay <[email protected]>

* petsc,py-petsc4py,slepc,py-slepc4py: add version 3.18.0 (spack#32938)

* petsc,py-petsc4py,slepc,py-slepc4py: add version 3.18.0

* workaround for dealii build failure [with petsc version check]

* pism: add compatibility fix to for [email protected]

* add in hipsolver dependency

* py-libensemble: updating package for v0.9.3 (spack#33298)

* commit updating py-libensemble package for 0.9.3

* removed commented-out lines

* Add checksum for py-oauthlib 3.2.1 (spack#33201)

* Add checksum for py-oauthlib 3.2.1

* Update package.py

* [@spackbot] updating style on behalf of iarspider

* Update package.py

* Update package.py

Co-authored-by: iarspider <[email protected]>

* ninja: New version 1.11.1 (spack#33215)

* seacas: update to latest release (spack#33330)

Add checksum for latest tag/release

* gptl: new version 8.1.1; use the correct mpi fortran compiler (spack#33235)

* gptl: new version 8.1.1; use the correct `mpifc`
* add `F90` and `$F77`

* glib: add 2.74.0 and 2.72.4 (spack#33332)

* depfile: update docs (spack#33279)

* rocksdb: add 7.7.3 (spack#33341)

* Add checksum for py-seaborn 0.12.0 (spack#33145)

* Add checksum for py-seaborn 0.12.0

* Update package.py

* Update package.py

* [@spackbot] updating style on behalf of iarspider

* Update package.py

Co-authored-by: iarspider <[email protected]>

* CI: allow multiple matches to combine tags (spack#32290)

Currently "spack ci generate" chooses the first matching entry in
gitlab-ci:mappings to fill attributes for a generated build-job,
requiring that the entire configuration matrix is listed out
explicitly. This unfortunately causes significant problems in
environments with large configuration spaces, for example the
environment in spack#31598 (spack.yaml) supports 5 operating systems,
3 architectures and 130 packages with explicit size requirements,
resulting in 1300 lines of configuration YAML.

This patch adds a configuraiton option to the gitlab-ci schema called
"match_behavior"; when it is set to "merge", all matching entries
are applied in order to the final build-job, allowing a few entries
to cover an entire matrix of configurations.

The default for "match_behavior" is "first", which behaves as before
this commit (only the runner attributes of the first match are used).

In addition, match entries may now include a "remove-attributes"
configuration, which allows matches to remove tags that have been
aggregated by prior matches. This only makes sense to use with
"match_behavior:merge". You can combine "runner-attributes" with
"remove-attributes" to effectively override prior tags.

* meson: update OneAPI compiler support patch (spack#33293)

* py-tensorflow: fix zlib (spack#33349)

* py-tensorflow: fix zlib

* [@spackbot] updating style on behalf of haampie

Co-authored-by: haampie <[email protected]>

* py-meson-python: add new versions (spack#33294)

* tasmanian: disable openmp by default (spack#33345)

* octopus: upgrade to 12.1 (spack#33343)

* py-sphinx: add v5.3 and v5.2 (spack#33356)

* py-setuptools: add v65.5.0 (spack#33353)

* libblastrampoline: Add versions 5.1.1, 5.2.0 (spack#33352)

* nextflow: add v20.10.0 (spack#33354)

* sdl2: add v2.0.22 and v2.24.1 (spack#33351)

* mariadb-c-client: add 3.3.2, 3.2.7, 3.1.18, 3.0.10 (spack#33335)

* py-tensorflow-hub: zlib, again. (spack#33359)

* Add checksum for py-sniffio 1.3.0 (spack#32975)

* py-numpy: add v1.23.4 (spack#33260)

* py-jupyterlab-pygments: install from wheel to avoid cyclic dependency (spack#33278)

* py-jupyterlab-pygments: avoid cyclic dependency

* Fix style

* Update package.py

* Update package.py

* [@spackbot] updating style on behalf of iarspider

* Update package.py

* Flake-8

* fix

Co-authored-by: iarspider <[email protected]>

* installer.py: show timers for binary install (spack#33305)

Print a message of the form
```
Fetch mm:ss.  Build: mm:ss.  Total: mm:ss
```
when installing from buildcache. 

Previously this only happened for source builds.

* Add checksum for py-astroid 2.12.7, py-astroid 2.12.10, py-setuptools 62.6.0, py-wrapt 1.14.1, py-pylint 2.15.0 (spack#32976)

* Add checksum for py-astroid 2.12.7, py-setuptools 62.6.0

* Also add checksum for py-wrapt

* Update package.py

* Update package.py (spack#57)

* Update package.py

* Update package.py

* Update package.py

* gitlab ci: Do not force protected build jobs to run on aws runners (spack#33314)

* installer.py: fix/test get_deptypes (spack#33363)

Fixing an oversight in spack#32537

`get_deptypes` should depend on new `package/dependencies_cache_only`
props.

* Add checksum for py-grpcio-tools 1.48.1 (spack#33358)

* Add checksum for py-prompt-toolkit 3.0.31 (spack#33362)

Co-authored-by: Harmen Stoppels <[email protected]>
Co-authored-by: Jim Edwards <[email protected]>
Co-authored-by: iarspider <[email protected]>
Co-authored-by: Scott Wittenburg <[email protected]>
Co-authored-by: Adam J. Stewart <[email protected]>
Co-authored-by: Dan Bonachea <[email protected]>
Co-authored-by: Auriane R <[email protected]>
Co-authored-by: eugeneswalker <[email protected]>
Co-authored-by: Satish Balay <[email protected]>
Co-authored-by: John-Luke Navarro <[email protected]>
Co-authored-by: iarspider <[email protected]>
Co-authored-by: Erik Schnetter <[email protected]>
Co-authored-by: Greg Sjaardema <[email protected]>
Co-authored-by: WuK <[email protected]>
Co-authored-by: Michael Kuhn <[email protected]>
Co-authored-by: Jonathon Anderson <[email protected]>
Co-authored-by: haampie <[email protected]>
Co-authored-by: Miroslav Stoyanov <[email protected]>
Co-authored-by: Hans Fangohr <[email protected]>
Co-authored-by: Mosè Giordano <[email protected]>
Co-authored-by: Diego Alvarez <[email protected]>
Co-authored-by: Wouter Deconinck <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

commands core PR affects Spack core functionality docker gitlab Issues related to gitlab integration tests General test capability(ies) workflow

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant