Skip to content

ACCESS-OM2 Dependencies Update#60

Merged
CodeGat merged 13 commits intomainfrom
om2-dependencies-update
Jun 26, 2024
Merged

ACCESS-OM2 Dependencies Update#60
CodeGat merged 13 commits intomainfrom
om2-dependencies-update

Conversation

@CodeGat
Copy link
Copy Markdown
Member

@CodeGat CodeGat commented Apr 26, 2024

Updated compiler to [email protected], updated package versions

Related issue #61 (doesn't close until compiler is also updated to OneAPI version, ifx)

@CodeGat CodeGat self-assigned this Apr 26, 2024
@github-actions
Copy link
Copy Markdown
Contributor

This access-om2 model will be deployed with the following versions:

  • 2024.04.0 as a Release (when merged).
  • pr60-4 as a Prerelease (during this PR). This can be accessed on Gadi via spack at /g/data/vk83/prerelease/apps/spack/0.20/spack once deployed.

It will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

@github-actions
Copy link
Copy Markdown
Contributor

This access-om2 model will be deployed with the following versions:

  • 2024.04.0 as a Release (when merged).
  • pr60-4 as a Prerelease (during this PR). This can be accessed on Gadi via spack at /g/data/vk83/prerelease/apps/spack/0.20/spack once deployed.

It will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

@github-actions
Copy link
Copy Markdown
Contributor

This access-om2 model will be deployed with the following versions:

  • 2024.04.0 as a Release (when merged).
  • pr60-5 as a Prerelease (during this PR). This can be accessed on Gadi via spack at /g/data/vk83/prerelease/apps/spack/0.20/spack once deployed.

It will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

@CodeGat CodeGat temporarily deployed to Gadi Prerelease April 26, 2024 04:55 — with GitHub Actions Inactive
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 26, 2024

This access-om2 model will be deployed with the following versions:

  • 2024.04.0 as a Release (when merged).
  • pr60-6 as a Prerelease (during this PR). This can be accessed on Gadi via spack at /g/data/vk83/prerelease/apps/spack/0.21/spack once deployed.

It will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

@CodeGat CodeGat temporarily deployed to Gadi Prerelease April 26, 2024 06:22 — with GitHub Actions Inactive
@harshula
Copy link
Copy Markdown
Collaborator

@harshula
Copy link
Copy Markdown
Collaborator

Hi @CodeGat , Should we also use Spack v0.21 instead of v0.20?

@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Apr 29, 2024

It is using that version - if you have access, check the vars in the Gadi Prerelease environment setting - they're pointing to the 0.21 prerelease spack on Gadi

@aidanheerdegen
Copy link
Copy Markdown
Member

Am I correct, updating all those dependencies didn't change bit reproducibility?

@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented May 1, 2024

I don't know if we'd tested the bit repro yet?

@aidanheerdegen
Copy link
Copy Markdown
Member

I don't know if we'd tested the bit repro yet?

Yeah that was a mistake on my part.

I've created an issue for that

ACCESS-NRI/access-om2-configs#107

I can make a PR to test it, or if you want to use this as a test for an automated PR workflow that would also be cool @CodeGat

@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented May 1, 2024

That workflow might take a bit of thonking to figure out, so I'm happy for this one to be manual!

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented May 6, 2024

This access-om2 model will be deployed with the following versions:

  • 2024.04.0 as a Release (when merged).
  • pr60-7 as a Prerelease (during this PR). This can be accessed on Gadi via spack at /g/data/vk83/prerelease/apps/spack/0.20/spack once deployed.

It will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented May 6, 2024

Gadi is currently feeling sluggish and is failing to deploy our things. Will wait until it is better, then try again

@aidanheerdegen
Copy link
Copy Markdown
Member

I have manually run the repro tests with the release-1deg_jra55_ryf branch, using the binaries from this PR:

$ git diff HEAD^^ manifests/exe.yaml
diff --git a/manifests/exe.yaml b/manifests/exe.yaml
index 0aeb8c6..80ed3f5 100644
--- a/manifests/exe.yaml
+++ b/manifests/exe.yaml
@@ -2,17 +2,17 @@ format: yamanifest
 version: 1.0
 ---
 work/atmosphere/yatm.exe:
-  fullpath: /g/data/vk83/apps/spack/0.20/release/linux-rocky8-x86_64/intel-19.0.5.281/libaccessom2-git.2023.10.26=2023.10.26-ieiy3e7hidn4dzaqly3ly2yu45mecgq4/bin/yatm.exe
+  fullpath: /g/data/vk83/apps/spack/0.20/release/linux-rocky8-x86_64/intel-19.0.5.281/libaccessom2-git.2023.10.26=2023.10.26-ltfg7jcn6t4cefotvj3kjnyu5nru26xo/bin/yatm.exe
   hashes:
-    binhash: 61890159678b68a1699cb989d2c2864e
-    md5: 7bbd1014eb48b16406877d1b93ae1a47
+    binhash: 4e8b4ef76e971c4af3b26cfac632e160
+    md5: 5baa1d417fe6708fc30cbeaa57d82f96
 work/ice/cice_auscom_360x300_24x1_24p.exe:
-  fullpath: /g/data/vk83/apps/spack/0.20/release/linux-rocky8-x86_64/intel-19.0.5.281/cice5-git.2023.10.19=2023.10.19-rh3xfkrgajya3ghtliacuhlx3pgvrzqs/bin/cice_auscom_360x300_24x1_24p.exe
+  fullpath: /g/data/vk83/apps/spack/0.20/release/linux-rocky8-x86_64/intel-19.0.5.281/cice5-git.2023.10.19=2023.10.19-v3zncpqjj2gyseudbwiudolcjq3k3leo/bin/cice_auscom_360x300_24x1_24p.exe
   hashes:
-    binhash: 6bff005e04c23c579f37b7b2c0189793
-    md5: 5e7c7ba864da95cd1329d098f1e47776
+    binhash: 3a65f67d21152e77034da28c22a94c66
+    md5: 37866455b057c85c3ea50c0ef0ea840b
 work/ocean/fms_ACCESS-OM.x:
-  fullpath: /g/data/vk83/apps/spack/0.20/release/linux-rocky8-x86_64/intel-19.0.5.281/mom5-git.2023.11.09=2023.11.09-ewcdbrfukblyjxpkhd3mfkj4yxfolal4/bin/fms_ACCESS-OM.x
+  fullpath: /g/data/vk83/apps/spack/0.20/release/linux-rocky8-x86_64/intel-19.0.5.281/mom5-git.2023.11.09=2023.11.09-qji4nlmr6utrribaiyhewe4je6mifguz/bin/fms_ACCESS-OM.x
   hashes:
-    binhash: c5c24bc2ab515b08e47d6d3ab2b54cc6
-    md5: 2432c23048525b384fd57e8cd2ed7e3b
+    binhash: 92ce1ff1a38f44f92ceafd67e8e7142c
+    md5: a3f10baeadb88e813b4a8121f61a6226
$ model-config-tests -m checksum
============================================================ test session starts =============================================================
platform linux -- Python 3.11.7, pytest-8.2.1, pluggy-1.5.0                            
rootdir: /g/data/tm70/aph502/envs/model-config-tests/lib/python3.11/site-packages/model_config_tests
collected 39 items / 37 deselected / 2 selected                                                                                              
                                     
../../../../../g/data/tm70/aph502/envs/model-config-tests/lib/python3.11/site-packages/model_config_tests/test_bit_reproducibility.py . [ 50%]
.                                                                                                                                      [100%]
                      
================================================ 2 passed, 37 deselected in 685.56s (0:11:25) ================================================
Details of python `venv` used for testing

``` $ pip list Package Version ------------------------- ----------- attrs 23.2.0 certifi 2024.6.2 cftime 1.6.3 charset-normalizer 3.3.2 f90nml 1.4.4 gitdb 4.0.11 GitPython 3.1.43 idna 3.7 iniconfig 2.0.0 jsonschema 4.22.0 jsonschema-specifications 2023.12.1 model_config_tests 0.0.1 nchash 0.1.5 netCDF4 1.6.5 numpy 1.26.4 packaging 24.0 payu 1.1.3 pip 23.2.1 pluggy 1.5.0 pytest 8.2.1 python-dateutil 2.9.0.post0 PyYAML 6.0.1 referencing 0.35.1 requests 2.32.3 rpds-py 0.18.1 ruamel.yaml 0.18.6 ruamel.yaml.clib 0.2.8 setuptools 65.5.0 six 1.16.0 smmap 5.0.1 tenacity 8.3.0 urllib3 2.2.1 ```

Based on the reproducibility and performance testing, I think this should be merged.

@aidanheerdegen aidanheerdegen requested a review from harshula June 5, 2024 07:14
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 5, 2024

This access-om2 model will be deployed as:

  • 2024.06.0 as a Release (when merged).
  • pr60-10 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using module use /g/data/vk83/prerelease/modules/access-models/ && module load access-om2/pr60-10, where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.21/spack in the access-om2-pr60-10 environment.

It will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

@CodeGat CodeGat temporarily deployed to Gadi Prerelease June 5, 2024 23:51 — with GitHub Actions Inactive
@aidanheerdegen
Copy link
Copy Markdown
Member

aidanheerdegen commented Jun 6, 2024

We probably want to update to 2024.06.0 if that is when we're merging the PR.

Or ... we could use 2023.03.1, as we're not updating any model components, and we've since established it is bit repro and doesn't affect performance.

Sorry! And I'm not particularly sold on one over the other.

@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Jun 6, 2024

@aidanheerdegen Good idea. I'll use 2024.03.1 instead.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 6, 2024

This access-om2 model will be deployed as:

  • 2024.03.1 as a Release (when merged).
  • pr60-11 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using module use /g/data/vk83/prerelease/modules/access-models/ && module load access-om2/pr60-11, where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.21/spack in the access-om2-pr60-11 environment.

It will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

@CodeGat CodeGat temporarily deployed to Gadi Prerelease June 6, 2024 01:16 — with GitHub Actions Inactive
@CodeGat CodeGat temporarily deployed to Gadi Prerelease June 7, 2024 03:10 — with GitHub Actions Inactive
@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Jun 7, 2024

The above commit is testing out #60 (comment) - it can be removed if it is not needed, and we can go back to 4ed4c7b

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 7, 2024

This access-om2 model will be deployed as:

  • 2024.03.1 as a Release (when merged).
  • pr60-12 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using module use /g/data/vk83/prerelease/modules/access-models/ && module load access-om2/pr60-12, where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.21/spack in the access-om2-pr60-12 environment.

It will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

@github-actions
Copy link
Copy Markdown
Contributor

This access-om2 model will be deployed as:

  • 2024.03.1 as a Release (when merged).
  • pr60-13 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using module use /g/data/vk83/prerelease/modules/access-models/ && module load access-om2/pr60-13, where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.21/spack in the access-om2-pr60-13 environment.

It will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

@CodeGat CodeGat temporarily deployed to Gadi Prerelease June 11, 2024 04:18 — with GitHub Actions Inactive
@CodeGat CodeGat requested a review from harshula June 11, 2024 04:37
@CodeGat CodeGat merged commit 32ef741 into main Jun 26, 2024
@CodeGat CodeGat temporarily deployed to Gadi Prerelease June 26, 2024 03:55 — with GitHub Actions Inactive
@CodeGat
Copy link
Copy Markdown
Member Author

CodeGat commented Jun 26, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants