Skip to content

Add developer docs#63

Closed
aidanheerdegen wants to merge 14 commits intomainfrom
aidanheerdegen/issue58
Closed

Add developer docs#63
aidanheerdegen wants to merge 14 commits intomainfrom
aidanheerdegen/issue58

Conversation

@aidanheerdegen
Copy link
Copy Markdown
Member

@aidanheerdegen aidanheerdegen commented Apr 30, 2024

Initial version to address #58

@aidanheerdegen aidanheerdegen force-pushed the aidanheerdegen/issue58 branch from 24f8ca2 to c3aec10 Compare April 30, 2024 10:22
@aidanheerdegen
Copy link
Copy Markdown
Member Author

Currently the dual component development build errors:

==> Error: ProcessError: Command exited with status 2:                                                                                                         
    '/g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/spack-build.sh' 'auscom' '360x300' '24x1' '24'                                        
                                                                                                                                                              
4 errors found in build log:                                                                                                                                   
     186    cd /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/build_auscom_360x300_24x1_24p                                                   
     187    cat                                                                                                                                               
     188    cc -o makdep /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/makdep.c                                                          
     189    In file included from /usr/include/bits/floatn.h(119),                                                                                             
     190                     from /usr/include/stdlib.h(55),                                                                                                  
     191                     from /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/makdep.c(20):                                            
  >> 192    /usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers                                                             
     193      typedef float _Float32;                                                                                                                          
     194                    ^                                                                                                                                 
     195                                                                                                                                                      
     196    In file included from /usr/include/bits/floatn.h(119),                                                                          
     197                     from /usr/include/stdlib.h(55),                                                                                                  
     198                     from /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/makdep.c(20):                                             
  >> 199    /usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers                                                              
     200      typedef double _Float64;                                                                                                                         
     201                     ^                                                                                                                                
     202                                                                                                                                                      
     203    In file included from /usr/include/bits/floatn.h(119),                                                                                            
     204                     from /usr/include/stdlib.h(55),                                                                                             
     205                     from /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/makdep.c(20):                                             
  >> 206    /usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers                                                             
     207      typedef double _Float32x;                                                                                                                       
     208                     ^                                                                                                                    
     209                                                                                                                                                       
     210    In file included from /usr/include/bits/floatn.h(119),                                                                                            
     211                     from /usr/include/stdlib.h(55),                                                                                                  
     212                     from /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/makdep.c(20):
  >> 213    /usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers                                                              
     214      typedef long double _Float64x;                                                                                                                  
     215                          ^                                                                                                                           
     216                                                                                                                                    
     217    compilation aborted for /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/makdep.c (code 2)                                      
     218    exit 2                                                                                                                                             
                                                                                                                                                              
See build log for details:                                                                                                                                     
  /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/spack-build-out.txt 

There are similar errors for the mom5 build.

@aidanheerdegen
Copy link
Copy Markdown
Member Author

Another problem I came across: the instructions only seem to work for references like [email protected], when I tried [email protected] I got errors like

$ spack concretize -f                                                                                                                                                                     
==> Error: concretization failed for the following reasons:                                                                                                   
                                                                                                                                                               
   1. cannot satisfy a requirement for package 'mom5'.

@aidanheerdegen aidanheerdegen requested review from CodeGat and harshula May 1, 2024 01:18
@aidanheerdegen
Copy link
Copy Markdown
Member Author

Another problem I came across: the instructions only seem to work for references like [email protected], when I tried [email protected] I got errors

Now I think about it, maybe that is because I still had a package: depends in there from the deployment spack.yaml.

Which brings up another point. Should we provide a dev.yaml with less in it, and all the components already added as a spec to make it simpler for developers to create a suitable environment?

Copy link
Copy Markdown
Member

@CodeGat CodeGat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good to me so far. Some small changes.

@aidanheerdegen
Copy link
Copy Markdown
Member Author

Seems good to me so far. Some small changes.

Thanks for the review. What do you think structurally? Covers the right ground? Too much detail? Does there need to be a TL;DR at the top that just says how to get going super quickly?

Is it worth including all the spack output? It helps in some ways, but also creates a lot of visual clutter. Also the way it is done means they can't cut n' paste those code blocks and just run them for themselves.

Any opinion on providing a dev environment spack.yaml?

@CodeGat
Copy link
Copy Markdown
Member

CodeGat commented May 2, 2024

TLDRs are always good.
Now that you mention it, the logging is excessive. Could possibly just do something like

# output trimmed for brevity... 

On some of the logging. For example, a lot of the [+] lines are redundant - just showing that packages have been installed earlier

@aekiss
Copy link
Copy Markdown
Contributor

aekiss commented May 2, 2024

This is looking great, thanks guys. Could this document also state explicitly where to find the newly-compiled executable and how to add it to config.yaml so it can be run? (Does anything else need to change in config.yaml?)

2. Concretize the updated environment. This will update what spack thinks is required to build the defined specs, in this case `access-om2` and `mom5`. Note that the `mom5` spec has a `dev_path` argument that points to the location of the sources it will use to build the package.

```bash
$ spack concretize -f
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

==> Error: concretization failed for the following reasons:

   1. Cannot select a single "version" for package "mom5"
   2. Cannot satisfy '[email protected]=2023.11.09'
   3. Cannot satisfy '[email protected]=2023.11.09'
   4. Cannot satisfy '[email protected]=2023.11.09'

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also received this. @harshula's suggestion to use

spack develop [email protected]

also fixed things for me (both with and without adding the spec).

Comment on lines +100 to +101
$ spack add [email protected]
==> Adding [email protected]=2023.11.09 to environment /g/data/.../dev_instructions/envs/mom5-dev
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double check if you need to add a package, since it is already included in the spack.yaml and you have added it with a version via develop.

@harshula
Copy link
Copy Markdown
Collaborator

harshula commented May 7, 2024

The following worked:

[root@1cb2279fcb5a /]# git clone https://github.com/ACCESS-NRI/ACCESS-OM2.git
Cloning into 'ACCESS-OM2'...
remote: Enumerating objects: 250, done.
remote: Counting objects: 100% (60/60), done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 250 (delta 43), reused 30 (delta 24), pack-reused 190
Receiving objects: 100% (250/250), 84.42 KiB | 6.03 MiB/s, done.
Resolving deltas: 100% (94/94), done.

[root@1cb2279fcb5a /]# mkdir mom5-dev

[root@1cb2279fcb5a /]# cd mom5-dev

[root@1cb2279fcb5a mom5-dev]# spack env create -d . ../ACCESS-OM2/spack.yaml
==> Created environment in /mom5-dev
==> You can activate this environment with:
==>   spack env activate /mom5-dev

[root@1cb2279fcb5a mom5-dev]# spack env activate .

[root@1cb2279fcb5a mom5-dev]# spack env status
==> Using spack.yaml in current directory: /mom5-dev

[root@1cb2279fcb5a mom5-dev]# spack develop [email protected]

==> Configuring spec [email protected]=2023.11.09 for development at path mom5

[root@1cb2279fcb5a mom5-dev]# spack concretize -f

==> Concretized [email protected]=2024.03.0
 -   75s5qbm  [email protected]=2024.03.0%[email protected]~deterministic build_system=bundle arch=linux-rocky8-x86_64
 -   2tvb2i6      ^[email protected]=2023.10.19%[email protected]~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64
 -   ojigw3h          ^[email protected]%[email protected] build_system=autotools patches=80b9577 arch=linux-rocky8-x86_64
[+]  l7h7kzk          ^[email protected]%[email protected]~guile build_system=generic arch=linux-rocky8-x86_64
 -   iov3ic2          ^[email protected]%[email protected]~blosc~byterange~dap~fsync~hdf4~jna+mpi~nczarr_zip+optimize~parallel-netcdf+pic+shared~szip~zstd build_system=autotools arch=linux-rocky8-x86_64
 -   2zxw3rg              ^[email protected]%[email protected]~cxx~fortran+hl~ipo~java~map+mpi+shared~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64
 -   w6fac4x          ^[email protected]%[email protected]~doc+pic+shared build_system=autotools patches=b050dbd arch=linux-rocky8-x86_64
 -   v2ddwt6          ^[email protected]=2023.11.09%[email protected]~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64
[+]  zsvealv          ^[email protected]%[email protected]~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix+romio+rsh~singularity+static+vt+wrapper-rpath build_system=autotools fabrics=none patches=073477a,60ce20b schedulers=none arch=linux-rocky8-x86_64
[+]  m7o4usy              ^[email protected]%[email protected]~cairo~cuda~gl~libudev+libxml2~netloc~nvml~oneapi-level-zero~opencl+pci~rocm build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
[+]  arw22dt                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  5x73eit                      ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  ztbumxg                  ^[email protected]%[email protected]+pic~python+shared build_system=autotools arch=linux-rocky8-x86_64
[+]  6czv5xg                      ^[email protected]%[email protected] build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
[+]  mamryan                      ^[email protected]%[email protected]~pic build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
[+]  hi4pfkb              ^[email protected]%[email protected] build_system=autotools patches=4e1d78c,62fc8a8,ff37630 arch=linux-rocky8-x86_64
[+]  acsxl2d                  ^[email protected]%[email protected] build_system=autotools patches=35c4492,7793209,a49dd5b arch=linux-rocky8-x86_64
[+]  ulelybd                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  aibo2hi                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  bl5vgn2                  ^[email protected]%[email protected]+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7 arch=linux-rocky8-x86_64
[+]  3qlauq6                      ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  nxptqrm                      ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  ijsfwt7              ^[email protected]%[email protected]+gssapi build_system=autotools arch=linux-rocky8-x86_64
[+]  zbzsece                  ^[email protected]%[email protected]+shared build_system=autotools arch=linux-rocky8-x86_64
[+]  oljratz                      ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  wxdsanc                      ^[email protected]%[email protected] build_system=autotools patches=440b954 arch=linux-rocky8-x86_64
[+]  ulr3uy3                      ^[email protected]%[email protected]+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools arch=linux-rocky8-x86_64
[+]  m4kzokr                          ^[email protected]%[email protected]~debug~pic+shared build_system=generic arch=linux-rocky8-x86_64
[+]  7ohhm7g                          ^[email protected]%[email protected] build_system=autotools zip=pigz arch=linux-rocky8-x86_64
[+]  kw2iugs                              ^[email protected]%[email protected] build_system=makefile arch=linux-rocky8-x86_64
[+]  clqsbru                              ^[email protected]%[email protected]+programs build_system=makefile compression=none libs=shared,static arch=linux-rocky8-x86_64
[+]  mgpwduu                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  b3wuw76                  ^[email protected]%[email protected]~obsolete_api build_system=autotools patches=4885da3 arch=linux-rocky8-x86_64
[+]  obhhhkg                  ^[email protected]%[email protected]~docs+shared build_system=generic certs=mozilla arch=linux-rocky8-x86_64
[+]  noqg2rf                      ^ca-certificates-mozilla@2023-05-30%[email protected] build_system=generic arch=linux-rocky8-x86_64
[e]  u7fofm2              ^[email protected]%[email protected]~cpanm+opcode+open+shared+threads build_system=generic patches=8cf4302 arch=linux-rocky8-x86_64
[+]  2tsjfk3              ^[email protected]%[email protected]~docs+pmi_backwards_compatibility~python~restful build_system=autotools arch=linux-rocky8-x86_64
[+]  r5biad2                  ^[email protected]%[email protected]+openssl build_system=autotools arch=linux-rocky8-x86_64
[+]  7swvjd2              ^[email protected]%[email protected]+compat+opt build_system=autotools arch=linux-rocky8-x86_64
 -   taofugt          ^[email protected]%[email protected]+fortran~ipo~logging+mpi~pnetcdf~shared~timing build_system=cmake build_type=Release generator=make patches=55a6d7a arch=linux-rocky8-x86_64
 -   xs5nghf      ^[email protected]=2023.10.26%[email protected]~deterministic~ipo~optimisation_report build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64
[+]  qlvugmf          ^[email protected]%[email protected]~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-rocky8-x86_64
[+]  3vvvrhz              ^[email protected]%[email protected]~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs=shared,static tls=openssl arch=linux-rocky8-x86_64
[+]  hrjpfs2                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  3b277qb              ^[email protected]%[email protected]~symlinks+termlib abi=none build_system=autotools arch=linux-rocky8-x86_64
 -   a2vylfw          ^[email protected]%[email protected]~ipo build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64
[+]  2ibagqa          ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
 -   q6evhui      ^[email protected]=2023.11.09%[email protected]~deterministic~optimisation_report+restart_repro build_system=makefile dev_path=/mom5-dev/mom5 type=ACCESS-OM arch=linux-rocky8-x86_64

==> Updating view at /mom5-dev/.spack-env/view

[root@1cb2279fcb5a mom5-dev]# spack install
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/gmake-4.4.1-l7h7kzk72xrid2uwjc4xaupuvxmcsw7h
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/bzip2-1.0.8-m4kzokrqyxvn7aeppdoypno364gg6bby
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/xz-5.4.1-mamryan4pg6psyxxcyr4djwcxuietwps
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/zlib-ng-2.1.4-7swvjd22aiggqhf2eu5osnxcvahyrv7w
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/pkgconf-1.9.5-2ibagqa3rb5spjcoopnoxqhbpndalrsd
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/zstd-1.5.5-clqsbrujtgzvooh5sutsqoomxcv6vqn5
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libxcrypt-4.4.35-b3wuw76yov75k6rxbnmpoiplcycyhzby
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/numactl-2.0.14-hi4pfkbi3qlkkarineyk2o7qfohmr44h
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libiconv-1.17-6czv5xgtld5kdoab2rfsd6kbppoznzfm
==> Installing datetime-fortran-1.7.0-ojigw3hps67jzwhr5by4ldeq2ddevypi [10/37]
==> No binary for datetime-fortran-1.7.0-ojigw3hps67jzwhr5by4ldeq2ddevypi found: installing from source
==> Fetching https://github.com/wavebitscientific/datetime-fortran/releases/download/v1.7.0/datetime-fortran-1.7.0.tar.gz
==> Applied patch /opt/spack-packages/packages/datetime-fortran/0001-Enable-deterministic-builds-using-D-flag-for-ar.patch
==> datetime-fortran: Executing phase: 'autoreconf'
==> datetime-fortran: Executing phase: 'configure'
==> datetime-fortran: Executing phase: 'build'
==> datetime-fortran: Executing phase: 'install'
==> datetime-fortran: Successfully installed datetime-fortran-1.7.0-ojigw3hps67jzwhr5by4ldeq2ddevypi
  Stage: 3.26s.  Autoreconf: 0.00s.  Configure: 1.33s.  Build: 1.45s.  Install: 0.04s.  Post-install: 0.03s.  Total: 6.14s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/datetime-fortran-1.7.0-ojigw3hps67jzwhr5by4ldeq2ddevypi
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/pigz-2.7-kw2iugsyeuywllqhgcw2gcn6hoird2bw
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/openssl-3.1.3-obhhhkgf27fa2qh5f4ui2e5np4k4f6cg
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libpciaccess-0.17-arw22dtj7arogotgmvj5udwclr6h5ifb
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/nghttp2-1.57.0-hrjpfs2lx7evq6bxnworz4v6xnynp2zo
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/ncurses-6.4-3b277qbzudjq6dxnnbjjktjqkdn7oc3w
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libxml2-2.10.3-ztbumxg43utno4dbwbz4b626s32wxsgp
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/tar-1.34-7ohhm7ga5agbvkj23f3egescr6tl7kyh
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libevent-2.1.12-r5biad2gihqnxjxylaaw5rdo4v7nlu4m
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/curl-8.4.0-3vvvrhzgjh3xbduumo6e2l3xw3o6tgfs
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libedit-3.1-20210216-mgpwduuernhhu2qyennfzqj3w2ce3oj5
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/hwloc-2.9.1-m7o4usyq2rajo72swf2srp3v7yjbkrr2
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/gettext-0.22.3-ulr3uy3sebp6n7y6ai6zuttswhwejbys
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/cmake-3.24.2-qlvugmfoczvvhzssbm5mob46z4jmzlx7
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/pmix-4.2.2-2tsjfk3t337rdc4xkv5mgqvz2vhrhnzb
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/krb5-1.20.1-zbzseceiavw4axvxgnx7wqzi5dpy6p7q
==> Installing json-fortran-8.3.0-a2vylfwg5e6nzwva4s4xoxjdxid34fv6 [26/37]
==> No binary for json-fortran-8.3.0-a2vylfwg5e6nzwva4s4xoxjdxid34fv6 found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/5f/5fe9ad709a726416cec986886503e0526419742e288c4e43f63c1c22026d1e8a.tar.gz
==> No patches needed for json-fortran
==> json-fortran: Executing phase: 'cmake'
==> json-fortran: Executing phase: 'build'
==> json-fortran: Executing phase: 'install'
==> json-fortran: Successfully installed json-fortran-8.3.0-a2vylfwg5e6nzwva4s4xoxjdxid34fv6
  Stage: 3.97s.  Cmake: 0.33s.  Build: 2.66s.  Install: 0.07s.  Post-install: 0.06s.  Total: 7.10s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/json-fortran-8.3.0-a2vylfwg5e6nzwva4s4xoxjdxid34fv6
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/openssh-9.5p1-ijsfwt7iygrmdra7ir7sszgiigwiee2q
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/openmpi-4.0.2-zsvealvafpe45lbzzg7yf42oiompduwo
==> Installing hdf5-1.14.3-2zxw3rgf3ls4emchlblhjkb66smyns4x [29/37]
==> No binary for hdf5-1.14.3-2zxw3rgf3ls4emchlblhjkb66smyns4x found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/09/09cdb287aa7a89148c1638dd20891fdbae08102cf433ef128fd345338aa237c7.tar.gz
==> Ran patch() for hdf5
==> hdf5: Executing phase: 'cmake'
==> hdf5: Executing phase: 'build'
==> hdf5: Executing phase: 'install'
==> hdf5: Successfully installed hdf5-1.14.3-2zxw3rgf3ls4emchlblhjkb66smyns4x
  Stage: 5.51s.  Cmake: 20.89s.  Build: 1m 6.71s.  Install: 0.79s.  Post-install: 0.21s.  Total: 1m 34.28s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/hdf5-1.14.3-2zxw3rgf3ls4emchlblhjkb66smyns4x
==> Installing netcdf-c-4.7.4-iov3ic2jttncqfyzl2p4ofrw5krqcc5c [30/37]
==> No binary for netcdf-c-4.7.4-iov3ic2jttncqfyzl2p4ofrw5krqcc5c found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/99/99930ad7b3c4c1a8e8831fb061cb02b2170fc8e5ccaeda733bd99c3b9d31666b.tar.gz
==> No patches needed for netcdf-c
==> netcdf-c: Executing phase: 'autoreconf'
==> netcdf-c: Executing phase: 'configure'
==> netcdf-c: Executing phase: 'build'
==> netcdf-c: Executing phase: 'install'
==> netcdf-c: Successfully installed netcdf-c-4.7.4-iov3ic2jttncqfyzl2p4ofrw5krqcc5c
  Stage: 6.33s.  Autoreconf: 0.00s.  Configure: 18.55s.  Build: 27.52s.  Install: 0.74s.  Post-install: 0.03s.  Total: 53.27s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/netcdf-c-4.7.4-iov3ic2jttncqfyzl2p4ofrw5krqcc5c
==> Installing netcdf-fortran-4.5.2-w6fac4x63uugrl4v2o2wmr4vpoxvw66q [31/37]
==> No binary for netcdf-fortran-4.5.2-w6fac4x63uugrl4v2o2wmr4vpoxvw66q found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/b9/b959937d7d9045184e9d2040a915d94a7f4d0185f4a9dceb8f08c94b0c3304aa.tar.gz
==> Applied patch /opt/spack/var/spack/repos/builtin/packages/netcdf-fortran/no_parallel_build.patch
==> netcdf-fortran: Executing phase: 'autoreconf'
==> netcdf-fortran: Executing phase: 'configure'
==> netcdf-fortran: Executing phase: 'build'
==> netcdf-fortran: Executing phase: 'install'
==> netcdf-fortran: Successfully installed netcdf-fortran-4.5.2-w6fac4x63uugrl4v2o2wmr4vpoxvw66q
  Stage: 3.57s.  Autoreconf: 0.00s.  Configure: 14.33s.  Build: 33.86s.  Install: 0.31s.  Post-install: 0.05s.  Total: 52.23s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/netcdf-fortran-4.5.2-w6fac4x63uugrl4v2o2wmr4vpoxvw66q
==> Waiting for oasis3-mct-git.2023.11.09=2023.11.09-v2ddwt6mumin6lsx5nkfj6b74nq==> Installing oasis3-mct-git.2023.11.09=2023.11.09-v2ddwt6mumin6lsx5nkfj6b74nqzfae6 [32/37]
==> No binary for oasis3-mct-git.2023.11.09=2023.11.09-v2ddwt6mumin6lsx5nkfj6b74nqzfae6 found: installing from source
==> No patches needed for oasis3-mct
==> oasis3-mct: Executing phase: 'edit'
==> oasis3-mct: Executing phase: 'build'
==> oasis3-mct: Executing phase: 'install'
==> oasis3-mct: Successfully installed oasis3-mct-git.2023.11.09=2023.11.09-v2ddwt6mumin6lsx5nkfj6b74nqzfae6
  Stage: 4.29s.  Edit: 0.00s.  Build: 1m 30.16s.  Install: 0.02s.  Post-install: 0.04s.  Total: 1m 34.60s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/oasis3-mct-git.2023.11.09_2023.11.09-v2ddwt6mumin6lsx5nkfj6b74nqzfae6
==> Installing parallelio-2.5.2-taofugtgphqhfk2jrrr47bvb5yp2ns4o [33/37]
==> No binary for parallelio-2.5.2-taofugtgphqhfk2jrrr47bvb5yp2ns4o found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/93/935bc120ef3bf4fe09fb8bfdf788d05fb201a125d7346bf6b09e27ac3b5f345c.tar.gz
==> Moving resource stage
	source: /tmp/root/spack-stage/resource-genf90-taofugtgphqhfk2jrrr47bvb5yp2ns4o/spack-src/
	destination: /tmp/root/spack-stage/spack-stage-parallelio-2.5.2-taofugtgphqhfk2jrrr47bvb5yp2ns4o/spack-src/genf90
==> Moving resource stage
	source: /tmp/root/spack-stage/resource-CMake_Fortran_utils-taofugtgphqhfk2jrrr47bvb5yp2ns4o/spack-src/
	destination: /tmp/root/spack-stage/spack-stage-parallelio-2.5.2-taofugtgphqhfk2jrrr47bvb5yp2ns4o/spack-src/CMake_Fortran_utils
==> Applied patch /opt/spack/var/spack/repos/builtin/packages/parallelio/remove_redefinition_of_mpi_offset.patch
==> parallelio: Executing phase: 'cmake'
==> parallelio: Executing phase: 'build'
==> parallelio: Executing phase: 'install'
==> parallelio: Successfully installed parallelio-2.5.2-taofugtgphqhfk2jrrr47bvb5yp2ns4o
  Stage: 10.89s.  Cmake: 5.37s.  Build: 5.86s.  Install: 0.13s.  Post-install: 0.03s.  Total: 22.39s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/parallelio-2.5.2-taofugtgphqhfk2jrrr47bvb5yp2ns4o
==> Waiting for libaccessom2-git.2023.10.26=2023.10.26-xs5nghfcy53dpfgomzutr34iy==> Installing libaccessom2-git.2023.10.26=2023.10.26-xs5nghfcy53dpfgomzutr34iym4vpjkd [34/37]
==> No binary for libaccessom2-git.2023.10.26=2023.10.26-xs5nghfcy53dpfgomzutr34iym4vpjkd found: installing from source
==> Ran patch() for libaccessom2
==> libaccessom2: Executing phase: 'cmake'
==> libaccessom2: Executing phase: 'build'
==> libaccessom2: Executing phase: 'install'
==> libaccessom2: Successfully installed libaccessom2-git.2023.10.26=2023.10.26-xs5nghfcy53dpfgomzutr34iym4vpjkd
  Stage: 5.10s.  Cmake: 2.16s.  Build: 4.63s.  Install: 0.14s.  Post-install: 0.04s.  Total: 12.16s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libaccessom2-git.2023.10.26_2023.10.26-xs5nghfcy53dpfgomzutr34iym4vpjkd
==> Installing mom5-git.2023.11.09=2023.11.09-q6evhuii4knpdh3f3wb6mizyh22e2kxn [35/37]
==> No binary for mom5-git.2023.11.09=2023.11.09-q6evhuii4knpdh3f3wb6mizyh22e2kxn found: installing from source
==> No patches needed for mom5
==> mom5: Executing phase: 'edit'
==> mom5: Executing phase: 'build'
==> mom5: Executing phase: 'install'
==> mom5: Successfully installed mom5-git.2023.11.09=2023.11.09-q6evhuii4knpdh3f3wb6mizyh22e2kxn
  Stage: 0.00s.  Edit: 0.01s.  Build: 4m 16.12s.  Install: 0.04s.  Post-install: 0.08s.  Total: 4m 16.40s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/mom5-git.2023.11.09_2023.11.09-q6evhuii4knpdh3f3wb6mizyh22e2kxn
==> Installing cice5-git.2023.10.19=2023.10.19-2tvb2i6vqfugcwuoq6zkgtoqr33xrug3 [36/37]
==> No binary for cice5-git.2023.10.19=2023.10.19-2tvb2i6vqfugcwuoq6zkgtoqr33xrug3 found: installing from source
==> No patches needed for cice5
==> cice5: Executing phase: 'edit'
==> cice5: Executing phase: 'build'
==> cice5: Executing phase: 'install'
==> cice5: Successfully installed cice5-git.2023.10.19=2023.10.19-2tvb2i6vqfugcwuoq6zkgtoqr33xrug3
  Stage: 7.66s.  Edit: 0.01s.  Build: 11m 39.08s.  Install: 0.04s.  Post-install: 0.07s.  Total: 11m 46.98s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/cice5-git.2023.10.19_2023.10.19-2tvb2i6vqfugcwuoq6zkgtoqr33xrug3
==> Waiting for access-om2-git.2024.03.0=2024.03.0-75s5qbm3peejwou6u7e237swm7wek==> Installing access-om2-git.2024.03.0=2024.03.0-75s5qbm3peejwou6u7e237swm7wek4aq [37/37]
==> No binary for access-om2-git.2024.03.0=2024.03.0-75s5qbm3peejwou6u7e237swm7wek4aq found: installing from source
==> No patches needed for access-om2
==> access-om2: Executing phase: 'install'
==> access-om2: Successfully installed access-om2-git.2024.03.0=2024.03.0-75s5qbm3peejwou6u7e237swm7wek4aq
  Stage: 0.00s.  Install: 0.00s.  Post-install: 0.05s.  Total: 0.14s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/access-om2-git.2024.03.0_2024.03.0-75s5qbm3peejwou6u7e237swm7wek4aq
==> Updating view at /mom5-dev/.spack-env/view

dougiesquire
dougiesquire previously approved these changes May 7, 2024
Copy link
Copy Markdown
Collaborator

@dougiesquire dougiesquire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this @aidanheerdegen! A couple of very minor comments to take or leave

2. Concretize the updated environment. This will update what spack thinks is required to build the defined specs, in this case `access-om2` and `mom5`. Note that the `mom5` spec has a `dev_path` argument that points to the location of the sources it will use to build the package.

```bash
$ spack concretize -f
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also received this. @harshula's suggestion to use

spack develop [email protected]

also fixed things for me (both with and without adding the spec).

@harshula
Copy link
Copy Markdown
Collaborator

harshula commented May 8, 2024

I was able to successfully change the version of a dependency. e.g. Change the version of parallelio from 2.5.2 to 2.5.10 by doing:

  1. Edit the environment's spack.yaml file.
  2. Remove the section:
    parallelio:
      require: '@2.5.2'
  1. Modify the view and concretizer to be (It should also work if you set view: false.):
  view:
    default:
      root: spack-view
      projections:
        all: '{name}/{version}'
  concretizer:
    unify: when_possible
  1. Run:
spack add [email protected]
spack concretize -f
spack install

CodeGat
CodeGat previously approved these changes May 9, 2024
@aidanheerdegen aidanheerdegen dismissed stale reviews from CodeGat and dougiesquire via 5eca3aa May 12, 2024 02:52
@aidanheerdegen
Copy link
Copy Markdown
Member Author

Thanks for the thorough review and feedback

The following worked:

[root@1cb2279fcb5a /]# git clone https://github.com/ACCESS-NRI/ACCESS-OM2.git
[root@1cb2279fcb5a mom5-dev]# spack env create -d . ../ACCESS-OM2/spack.yaml
[root@1cb2279fcb5a mom5-dev]# spack env activate .
[root@1cb2279fcb5a mom5-dev]# spack env status
[root@1cb2279fcb5a mom5-dev]# spack develop [email protected]
[root@1cb2279fcb5a mom5-dev]# spack concretize -f
[root@1cb2279fcb5a mom5-dev]# spack install

I got an error at concretisation:

$ spack concretize -f
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.5/build_cache/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstr
ap-spack-bhqgwuvef354fwuxq7heeighavunpber.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.5/build_cache/linux-centos7-x86_64/gcc-10.2.1/clingo-bootstr
ap-spack/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstrap-spack-bhqgwuvef354fwuxq7heeighavunpber.spack
==> Installing "clingo-bootstrap@=spack%gcc@=10.2.1~docs+ipo+optimized+python+static_libstdcpp build_system=cmake build_type=
Release generator=make patches=bebb819,ec99431 arch=linux-centos7-x86_64" from a buildcache
==> Error: concretization failed for the following reasons:
                                         
   1. No valid value for variant 'deterministic' of package 'access-om2'

I am chaining the 0.20 spack tree to this install (though didn't document in instructions) like so

$ cat spack/etc/spack/upstreams.yaml 
upstreams:
  spack-vk83-0.20:
    install_tree: /g/data/vk83/apps/spack/0.20/release

If I delete that upstream it concretises ok. I guess that is due to a hash mismatches between 0.20 and 0.21? It worked when I was using spack v0.20.

Installation works, but is slow because it has to compile all the dependencies.

So I think I've established that we can't mix spack versions when chaining.

@aidanheerdegen
Copy link
Copy Markdown
Member Author

aidanheerdegen commented May 12, 2024

I was able to successfully change the version of a dependency. e.g. Change the version of parallelio from 2.5.2 to 2.5.10

Thanks again for working through this. I can confirm that this installed parallel-2.5.10, but doesn't rebuild cice5 to use the updated dependency:

$ spack find --paths --deps cice5
==> In environment /g/data/tm70/aph502/dev_instructions/envs/mom5-dev
==> Root specs
[email protected]=2024.03.0

[email protected]


==> Installed packages
-- linux-rocky8-x86_64_v4 / [email protected] --------------------
[email protected]=2023.10.19             /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/cice5-git.2023.10.19_2023.10.19-ckoimsofpfnlwizd3q6zcdd6zsdbu2hv
    [email protected]                  /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/datetime-fortran-1.7.0-oy47sjopcbyjxejfinwe3ept564idest
    [email protected]                             /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/gmake-4.4.1-ggoluv3v37ruoyyjo4idshs6lcfcnpyb
    [email protected]=2023.10.26  /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/libaccessom2-git.2023.10.26_2023.10.26-hve4wjl273pya4j4s6dbvdjistrid2oa
        [email protected]                        /apps/cmake/3.24.2
        [email protected]                  /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/json-fortran-8.3.0-dlmgtabad5koosoqhjfcs7323itzzrxx
        [email protected]                       /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/pkgconf-1.9.5-3f2l3qwlgatzrt6kdcigbdiadyydy6ld
    [email protected]                          /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/netcdf-c-4.7.4-lzrdcziljsdeosfzwpti2lkrad5ixdju
        [email protected]                         /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/hdf5-1.14.3-45zs6fv3fmmilvlb753kfscuxlqmlrq4
            [email protected]                   /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/zlib-ng-2.1.4-jugwcxhg3jwqkp4rzchowtra6zlmoxn3
    [email protected]                    /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/netcdf-fortran-4.5.2-g3tf2uevw73rzufozccjav5bumvebkto
    [email protected]=2023.11.09    /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/oasis3-mct-git.2023.11.09_2023.11.09-bhk6guqf66zqonmikcecpkpb3hmg4uaa
    [email protected]                           /apps/openmpi/4.0.2
    [email protected]                        /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/parallelio-2.5.2-ylrx5soeqhfqvdszal3dchfarzzk5wvq

==> 1 installed package

Tried

$ spack install --overwrite
==> The following package specs will be reinstalled:

-- linux-rocky8-x86_64_v4 / [email protected] --------------------
coujqai [email protected]=2024.03.0%intel ~deterministic build_system=bundle
==> Do you want to proceed? [y/N] y
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/datetime-fortran-1.7.0-oy47sjopcbyjxejfinwe3ept564idest
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/json-fortran-8.3.0-dlmgtabad5koosoqhjfcs7323itzzrxx
==> [email protected] : has external module in ['openmpi/4.0.2']
[+] /apps/openmpi/4.0.2 (external openmpi-4.0.2-skdxaycsjbzb33gqqp252d7vu6zuai5e)
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/pkgconf-1.9.5-3f2l3qwlgatzrt6kdcigbdiadyydy6ld
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/zlib-ng-2.1.4-jugwcxhg3jwqkp4rzchowtra6zlmoxn3
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/hdf5-1.14.3-45zs6fv3fmmilvlb753kfscuxlqmlrq4
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/netcdf-c-4.7.4-lzrdcziljsdeosfzwpti2lkrad5ixdju
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/netcdf-fortran-4.5.2-g3tf2uevw73rzufozccjav5bumvebkto
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/parallelio-2.5.2-ylrx5soeqhfqvdszal3dchfarzzk5wvq
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/oasis3-mct-git.2023.11.09_2023.11.09-bhk6guqf66zqonmikcecpkpb3hmg4uaa
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/libaccessom2-git.2023.10.26_2023.10.26-hve4wjl273pya4j4s6dbvdjistrid2oa
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/mom5-git.2023.11.09_2023.11.09-6m2cz6ai7znxxup7hvt5cornxgg7cwei
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/cice5-git.2023.10.19_2023.10.19-ckoimsofpfnlwizd3q6zcdd6zsdbu2hv
==> Installing access-om2-git.2024.03.0=2024.03.0-coujqair3vkzv5jickbilp5trnvjugi3 [14/14]
==> No binary for access-om2-git.2024.03.0=2024.03.0-coujqair3vkzv5jickbilp5trnvjugi3 found: installing from source
==> No patches needed for access-om2    
==> access-om2: Executing phase: 'install'
==> access-om2: Successfully installed access-om2-git.2024.03.0=2024.03.0-coujqair3vkzv5jickbilp5trnvjugi3
  Stage: 0.00s.  Install: 0.00s.  Post-install: 0.32s.  Total: 7.13s
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/access-om2-git.2024.03.0_2024.03.0-coujqair3vkzv5jickbilp5trnvjugi3
==> Warning: Module file /g/data/tm70/aph502/dev_instructions/release/modules/linux-rocky8-x86_64_v4/access-om2/2024.03.0 exists and will not be overwritten

But it didn't install a new version of cice5 with the new parallelio version.

Adding

    parallelio:
      require: '@2.5.10'

to the spack.yaml didn't change the outcome.

Copy link
Copy Markdown
Member

@ccarouge ccarouge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was reading through the developer docs and I decided to put in my own comments. They are on the form only, since I'm reading through to learn the content. I haven't finished reading through but sending what I have already so it doesn't get lost.

Feel free to ignore if it isn't the right time for these.

@aidanheerdegen aidanheerdegen marked this pull request as draft May 23, 2024 04:55
@aidanheerdegen
Copy link
Copy Markdown
Member Author

It was becoming unwieldy trying to investigate the issues with multi-component and updating dependencies, so I've turned this into a draft so it isn't merged, but we retain all the review comments in place.

I've created a new PR: #67 to get a simpler version merged and work on the other examples individually.

@harshula
Copy link
Copy Markdown
Collaborator

Hi @aidanheerdegen , There's no target being set in https://github.com/ACCESS-NRI/ACCESS-OM2/blob/main/spack.yaml . You need PR #60 to be merged or use branch om2-dependencies-update to get target=x86_64.

@harshula
Copy link
Copy Markdown
Collaborator

Thanks again for working through this. I can confirm that this installed parallel-2.5.10, but doesn't rebuild cice5 to use the updated dependency:

First try doing spack uninstall --dependents [email protected] %<compiler>

@harshula
Copy link
Copy Markdown
Collaborator

Hi @aidanheerdegen , I was able to successfully change the version of a dependency and rebuild the dependents. e.g. Change the version of parallelio from 2.5.2 to 2.5.10:

  1. Edit the environment's spack.yaml file and make the following change:
diff --git a/spack.yaml b/spack.yaml
index 0754065..c8f2ab0 100644
--- a/spack.yaml
+++ b/spack.yaml
@@ -20,7 +20,7 @@ spack:
     netcdf-fortran:
       require: '@4.5.2'
     parallelio:
-      require: '@2.5.2'
+      require: '@2.5.10'
     openmpi:
       require: '@4.0.2'
     all:
  1. Run:
spack uninstall --dependents [email protected] %<compiler>
spack concretize -f
spack install

@harshula
Copy link
Copy Markdown
Collaborator

CORRECTION:

  1. Run:
spack concretize -f --fresh
spack install

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants