Tutorial setup

If you have not done the prior sections, you’ll need to start the docker image:

docker run -it ghcr.io/spack/tutorial:sc25

and then set Spack up like this:

git clone --depth=2 --branch=releases/v1.1 https://github.com/spack/spack
. spack/share/spack/setup-env.sh
spack repo update builtin --tag v2025.11.0
spack tutorial -y
spack bootstrap now
spack compiler find

See the Basic Installation Tutorial for full details on setup. For more help, join us in the #tutorial channel on Slack – get an invitation at slack.spack.io

Developer Workflows Tutorial

This tutorial will guide you through the process of using the spack develop command to develop software from local source code within a Spack environment. With this command, Spack will manage your dependencies while you focus on testing changes to your library and/or application.

Installing from local source

The spack install command, as you know, fetches source code from a mirror or the internet before building and installing your package. As developers, we want to build from local source, which we will constantly change, build, and test.

Let’s imagine, for a second, we’re working on scr. scr is a library used to implement scalable checkpointing in application codes. It supports writing/reading checkpoints quickly and efficiently using MPI and high-bandwidth file I/O. We’d like to test changes to scr within an actual application, so we’ll test with macsio, a proxy application written to mimic typical HPC I/O workloads. We’ve chosen scr and macsio because together they are quick to build.

We’ll start by making an environment for our development. We need to build macsio with scr support, and we’d like everything to be built without Fortran support for the time being. Let’s set up that development workflow.

$ cd ~
$ mkdir devel-env
$ cd devel-env
$ spack env create -d .
==> Created independent environment in: /home/spack/devel-env
==> Activate with: spack env activate .
$ spacktivate .
$ # for now, disable fortran support in all packages
$ spack config add "packages:all:variants: ~fortran"
$ spack add macsio+scr
==> Adding macsio+scr to environment /home/spack/devel-env
$ spack install
==> Concretized 1 spec
 -   2qq3umg  [email protected]~exodus~hdf5~ipo+mpi~pdb+scr+silo~szip~typhonio~zfp~zlib build_system=cmake build_type=Release generator=make patches:=59479b9 platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  ivvor7v	  ^[email protected]~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  isdtvvd	      ^[email protected]~gssapi~ldap~libidn2~librtmp~libssh+libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=mbedtls,openssl platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  txa2olx		  ^[email protected]+shared build_system=autotools crypto=mbedtls platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  bz3ghzh		  ^[email protected]+pic build_system=makefile build_type=Release libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  ft5kpbd		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  gv7wpik		  ^[email protected]~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  etqlnw5		      ^ca-certificates-mozilla@2025-08-12 build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  ncdxq3j	      ^[email protected]~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  qlavhjb	      ^[email protected] cflags=-O3 +compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  ntccuj2	  ^[email protected] build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[e]  ml7cem5	  ^[email protected]~binutils+bootstrap~graphite~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  nokfxva	  ^[email protected] build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[e]  qg7qyaz	  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  4obn7cg	  ^[email protected]~guile build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
 -   xb3rsbk	  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  fgsf5ij	      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  pzmnwzm	      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  yt7ajy4	      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  irvryts		  ^[email protected] build_system=autotools patches:=440b954 platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  lxvpwti	      ^[email protected]+sigsegv build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  cih4xrz		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  qtepnkr		      ^[email protected] build_system=autotools libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  bs5ujst		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  tqxbnvo	  ^[email protected]+atomics~cuda~debug+fortran~gpfs~internal-hwloc~internal-libevent~internal-pmix~ipv6~java~lustre~memchecker~openshmem~rocm~romio+rsh~static~two_level_namespace+vt+wrapper-rpath build_system=autotools fabrics:=none romio-filesystem:=none schedulers:=none platform=linux os=ubuntu22.04 target=x86_64_v3 %c,cxx,[email protected]
[+]  4hos372	      ^[email protected]~cairo~cuda~gl~level_zero~libudev+libxml2~nvml~opencl+pci~rocm build_system=autotools libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  txbwcin		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  hwxnwvm		      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  5trxrsw		  ^[email protected]~http+pic~python+shared build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  lprginh	      ^[email protected]+openssl build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  yzbm5q6	      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  cxdcxo5	      ^[email protected]+gssapi build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  hnmy4fw		  ^[email protected]+shared build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  n7yzkyl		      ^[email protected]~color build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  hdzcfgi		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  yiij42p		  ^[email protected]~obsolete_api build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  62kt5y4	      ^[email protected]+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  vdgigsw		  ^[email protected]+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  x7t4naj		  ^[email protected]~debug~pic+shared build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  aq7qwy6		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  f4qiprw	      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  yqlblh6	      ^[email protected]~munge~python build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  buin62m	      ^[email protected] build_system=autotools schedulers:=none platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  gmhq65u		  ^[email protected]+lex~nls build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  f57s4no	  ^[email protected]+dtcmp~fortran~ipo+libyogrt async_api=NONE build_system=cmake build_type=Release cache_base=/dev/shm cntl_base=/dev/shm copy_config=none file_lock=FLOCK generator=make resource_manager=SLURM scr_config=scr.conf platform=linux os=ubuntu22.04 target=x86_64_v3 %c,cxx,[email protected]
[+]  2kcmd2x	      ^[email protected]+shared build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  bms4pwx		  ^[email protected]+shared build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  ylblit5	      ^[email protected]~static build_system=autotools scheduler=slurm platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  tc55ttm		  ^slurm@25-05-1-1~cgroup~gtk~hdf5~hwloc~mariadb~nvml~pam~pmix+readline~restd~rsmi build_system=autotools sysconfdir=PREFIX/etc platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  arsxi4d		      ^[email protected]+introspection~libmount~strip build_system=meson buildtype=release default_library:=shared tracing:=none platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  brbwl6l			  ^[email protected]~debuginfod+exeprefix+nls build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  zktpj7s			  ^[email protected]~strip build_system=meson buildtype=release default_library:=shared platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  dcark7n			      ^[email protected]~strip build_system=meson buildtype=release default_library:=shared platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  muzysky			  ^[email protected] build_system=python_pip patches:=0f0b1bd platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  hjc6yfd			      ^[email protected] build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  lnndrge			      ^[email protected] build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  hpfn7k7			      ^[email protected] build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  2stmhpc			  ^[email protected]+re2c build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  axmwics			      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  z5vjj7q			  ^[email protected]~jit+multibyte+pic build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  essoyki			  ^[email protected] build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  5viqezv		      ^[email protected]~ipo build_system=cmake build_type=Release generator=make platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  7m7ijno		      ^[email protected]+pic build_system=makefile libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  itsnhnf		      ^[email protected] build_system=autotools localstatedir=PREFIX/var platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  4j7zjzs			  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  jerwaho			      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  irhvzch				  ^[email protected]~nls build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  mj4re3l				      ^[email protected]+cxx build_system=autotools libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  m3nwalt				      ^[email protected] build_system=autotools libs:=shared,static patches:=3ec29a6 platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  k6ygirs					  ^[email protected]~xs build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  eicmuq5	      ^[email protected]+ssh+static_modules build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
 -   vsguusy	  ^[email protected]~fortran+fpzip+hdf5+hzip+mpi+pic+python+shared~silex build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,cxx,[email protected]
[+]  7p7gq7u	      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  7mm6knn	      ^[email protected]~cxx~fortran~hl~ipo~java~map~mpi+shared~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  rbf2x4i	      ^[email protected]+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches:=13fa8bf,b0615b2,ebdca64,f2fd060 platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  7yr5v6w		  ^[email protected]+libbsd build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  ucrhsyw		      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  gth3ii5			  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  cakgj4n		  ^[email protected]+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  lfgvgva		      ^[email protected] build_system=autotools zip=pigz platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  kaz756e			  ^[email protected] build_system=makefile platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  foiizhd			  ^[email protected]+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  zgro4tw		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  elvodyb		  ^[email protected]+column_metadata+fts+rtree build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  n45otd3		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  yzaocbs		  ^[email protected]~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  c6d2zlj	      ^[email protected] build_system=autotools patches:=21f0a03 platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]

[+] /usr (external glibc-2.35-qg7qyazcn2fwrck5vgr3mxq3i4uxkhlo)
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/compiler-wrapper-1.0-ntccuj2fi3y7asqifeq3i4iylrbxakw2
[+] /usr (external gcc-11.4.0-ml7cem5pfeoluzbhb7jsiisemvoauaz5)
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/autoconf-archive-2023.02.20-7p7gq7uzoaurflydkojv6kvjyue777zs
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gcc-runtime-11.4.0-nokfxvaa4aklxfhoymvz6hlvbqw3kwnh
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libmd-1.1.0-gth3ii5boey5akbsen5la4rgqpeaqkmx
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libiconv-1.18-qtepnkrdvqazp3jmfnheapckbemejrhq
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/berkeley-db-18.1.40-vdgigswy4fvttrivbedl4cp7bl72esma
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gmake-4.4.1-4obn7cgqfwoxyvmnpeedud63bsjhoqma
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/zlib-ng-2.0.7-qlavhjbsgqyboovfvsultjdwzz5nvthw
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/bzip2-1.0.8-x7t4najic2jb46srjiebkrf55utda3nl
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/nghttp2-1.48.0-ft5kpbdiz6kctcooz4ksam37pskskehg
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libbsd-0.12.2-ucrhsywwalxwcqigisozcu2xvvhx3zwx
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/lz4-1.10.0-7m7ijnovafqfvcxitqrbveal5qgz57cy
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libffi-3.5.2-zgro4twqg3qnq6hx6ohsjlejatth3rxa
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/numactl-2.0.18-yzbm5q6nblzcjccy7kddijqdnkgkxvtp
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/json-c-0.18-5viqezvbwekimujkx7dj3xc6qlwzmqes
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libgpg-error-1.55-jerwahojdgk4ga5b5vopwmu4oto356qx
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pcre2-10.44-z5vjj7qwopnby63vu35fgh5ojadzfc4x
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libpciaccess-0.17-txbwcin227323qmxmg4opnj4r4tvenz4
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/findutils-4.10.0-irvrytslmleivpkraggqq5brluhluert
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/mbedtls-2.28.9-bz3ghzheol2ecbi3vpldgbb3yd7yblle
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libsigsegv-2.14-bs5ujst3rrdcbj3r726bekzjfdddck4w
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/zstd-1.5.7-foiizhdg2mc4jdjtuddksbzvr64roxea
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/xz-5.6.3-yzaocbs7geczi5d7qyvmqwrxi4r7dvph
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pdsh-2.31-eicmuq5waz3gebhupmaxeo5bw5cl2gkm
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/ncurses-6.5-20250705-ncdxq3juvboefpavgmovg5rb5bx76ohz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/util-linux-uuid-2.41-n45otd3wy4fv7iawv52hiogfs63hxfpz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pkgconf-2.5.1-f4qiprwjcd2q3fp63uzgp47f3kw66r5h
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libxcrypt-4.4.38-yiij42powrfh2mwjebwiqxmkvinutofr
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pigz-2.8-kaz756eya6nj3ardls4b6aiaclkaux7i
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/openssl-3.6.0-gv7wpik32unrnickoxbgm5iqza572w6s
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/expat-2.7.3-7yr5v6w2xu5f4bohfxjcsnb2snnwwj5m
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libgcrypt-1.11.2-4j7zjzsf5hbmbypjyd4gqmpy6yp2esmn
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libtool-2.4.7-yt7ajy4c3zwh2ks6dawvycaxrydjibel
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/m4-1.4.20-lxvpwtivihagmvso5cxgs3b7olqmac2g
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libssh2-1.11.1-txa2olxuoxfkv7jdjrdw27djcf6fa5sz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/readline-8.3-c6d2zljdklrjdya4xy236brmpdonc2xo
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libxml2-2.13.5-5trxrsws5dig3bf63uc4xzcbsywsqtjo
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/hdf5-1.14.6-7mm6knnnjxquabuuhme4sjpe6d6vfawh
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libedit-3.1-20240808-hdzcfgipukcqxsw7hjkksstbbz5otx3x
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/tar-1.35-lfgvgvawnpdxxz7prdfn4vsfgtwruxjc
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libevent-2.1.12-lprginh6npwjirdhmghjqjckaa7ukrri
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/munge-0.5.15-itsnhnf3jn5ual4k7intv62hnu54gplq
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/curl-8.15.0-isdtvvdziidtozwacfgtdcv3cukd7uuz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/sqlite-3.50.4-elvodybm3egmq2kdk74ca3jdhzcnxu77
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gdbm-1.25-aq7qwy6yezwhtwdrsyqen65rmjmlth27
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/hwloc-2.12.2-4hos3725nynk2bsskofvtcmk6pmtcjg4
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gettext-0.23.1-cakgj4ntlncc4zsgty5z6sa7f7mwvlt7
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/cmake-3.31.9-ivvor7vdsqvplv65yyfragoxppyfi33t
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/perl-5.42.0-62kt5y4kl7uhwljviixr3m4dyf5vre7t
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pmix-6.0.0-yqlblh6p4u6vwmfhtffyrnxyqrliw2jm
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/elfutils-0.193-brbwl6lkh323duhprbcxlmwwk7zz2cs2
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/python-3.11.14-rbf2x4id5wuf7yihgm6fv5vtm2qoc2bz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/automake-1.16.5-pzmnwzmdyfj2g64cb6fycltaeiz2edau
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/autoconf-2.72-fgsf5ijzhnbaj2fqngm57aoe6civ72uz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/krb5-1.21.3-hnmy4fwwly5s4xifp5roodj6opppwwiu
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/python-venv-1.0-essoyki3al5aam2wqamlejoclcd7jf6o
==> Fetching file:///mirror/blobs/sha256/48/48fa302f6c1994e22f46eb1883f243e7fb6d168ef16dc9591715cb5fb43e32b0
    [100%]  296.57 KB @	 642.5 MB/s
==> Extracting json-cwx-0.12-xb3rsbkkz6mb4b2rapjkeq5iginffvwi from binary cache
==> json-cwx: Successfully installed json-cwx-0.12-xb3rsbkkz6mb4b2rapjkeq5iginffvwi
  Search: 0.00s.  Fetch: 0.17s.	 Install: 0.09s.  Extract: 0.05s.  Relocate: 0.01s.  Total: 0.26s
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/json-cwx-0.12-xb3rsbkkz6mb4b2rapjkeq5iginffvwi
==> Installing json-cwx-0.12-xb3rsbkkz6mb4b2rapjkeq5iginffvwi [59/70]
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/prrte-4.0.0-buin62mintd2cczzj6vsvai45vhpgg6w
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/openssh-9.9p1-cxdcxo5pdxc2nk7domxigdqiygg4757e
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/glib-2.86.1-arsxi4dynj54sx2h3mcryrco7pml4yhf
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/openmpi-5.0.8-tqxbnvoiw3nzds4xypcualamx3cvisln
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/slurm-25-05-1-1-tc55ttmephfx5nx3s7o7lem6w2hbke34
==> Fetching file:///mirror/blobs/sha256/65/65ae9a603ce1917040c73608811bb5779ec04211baedabfe384ef482fc407230
    [100%]   65.58 MB @	 351.2 GB/s
==> Extracting silo-4.11.1-vsguusygbz767eclcacoq2hzu3ivoeez from binary cache
==> silo: Successfully installed silo-4.11.1-vsguusygbz767eclcacoq2hzu3ivoeez
  Search: 0.00s.  Fetch: 0.01s.	 Install: 0.21s.  Extract: 0.11s.  Relocate: 0.05s.  Total: 0.23s
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/silo-4.11.1-vsguusygbz767eclcacoq2hzu3ivoeez
==> Installing silo-4.11.1-vsguusygbz767eclcacoq2hzu3ivoeez [65/70]
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/lwgrp-1.0.6-bms4pwx7whtr74fbz6idyyk5lbgpv72z
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libyogrt-1.35-ylblit5vwtdqn2zoc7ghzfmmf46bgm5g
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/dtcmp-1.1.5-2kcmd2x76so5ites4co66qnkqukjenot
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/scr-2.0.0-f57s4noxfkoiizusmhjyjue2h3fn2yrc
==> Fetching file:///mirror/blobs/sha256/90/90e2346d1c5b126a904a0fe6ff4dd0a471ca34c462b9e63ad4f1c70079afb1f0
    [100%]  109.67 KB @	 417.8 MB/s
==> Extracting macsio-1.1-2qq3umgemah2sniqgfdlkdjtitxrrumm from binary cache
==> macsio: Successfully installed macsio-1.1-2qq3umgemah2sniqgfdlkdjtitxrrumm
  Search: 0.00s.  Fetch: 0.01s.	 Install: 0.13s.  Extract: 0.01s.  Relocate: 0.07s.  Total: 0.14s
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/macsio-1.1-2qq3umgemah2sniqgfdlkdjtitxrrumm
==> Installing macsio-1.1-2qq3umgemah2sniqgfdlkdjtitxrrumm [70/70]
==> Updating view at /home/spack/devel-env/.spack-env/view
$ spack find -cv scr
==> In environment /home/spack/devel-env
==> 1 root specs
-- no arch / no compilers ---------------------------------------
[+] macsio+scr

-- linux-ubuntu22.04-x86_64_v3 / %c,cxx,[email protected] ------
[+]  [email protected]+dtcmp~fortran~ipo+libyogrt async_api=NONE build_system=cmake build_type=Release cache_base=/dev/shm cntl_base=/dev/shm copy_config=none file_lock=FLOCK generator=make resource_manager=SLURM scr_config=scr.conf
==> 1 installed package
==> 0 concretized packages to be installed

Before we do any work, we verify that this all builds. Spack ends up building the entire development tree below, and links everything together for you.

digraph G { labelloc = "b" rankdir = "TB" ranksep = "1" edge[ penwidth=4 ] node[ fontname=Monaco, penwidth=4, fontsize=24, margin=.2, shape=box, fillcolor=lightblue, style="rounded,filled" ] "4sh6pymrm2ms4auu3ajbjjr6fiuhz5g7" [label="pkgconf"] "7tkgwjvu2mi4ea2wsdetunq7g4k4r2nh" [label="json-cwx"] "4ihuiazsglf22f3pntq5hc4kyszqzexn" [label="berkeley-db"] "jearpk4xci4zc7dkrza4fufaqfkq7rfl" [label="libiconv"] "d2krmb5gweivlnztcymhklzsqbrpatt6" [label="automake"] "gs6ag7ktdoiirb62t7bcagjw62szrrg2" [label="util-macros"] "yn2r3wfhiilelyulh5toteicdtxjhw7d" [label="libxml2"] "lbrx7lnfz46ukewxbhxnucmx76g23c6q" [label="libsigsegv"] "bob4o5m3uku6vtdil5imasprgy775zg7" [label="libpciaccess"] "pmsyupw6w3gql4loaor25gfumlmvkl25" [label="openmpi"] "mkc3u4x2p2wie6jfhuku7g5rkovcrxps" [label="m4"] "jdxbjftheiotj6solpomva7dowrhlerl" [label="libtool"] "mm33a3ocsv3jsh2tfxc4mlab4xsurtdd" [label="autoconf"] "zfdvt2jjuaees43ffrrtphqs2ky3o22t" [label="perl"] "t54jzdy2jj4snltjazlm3br2urcilc6v" [label="readline"] "4av4gywgpaspkhy3dvbb62nulqogtzbb" [label="gdbm"] "crhlefo3dv7lmsv5pf4icsy4gepkdorm" [label="ncurses"] "bltycqwh5oofai4f6o42q4uuj4w5zb3j" [label="cmake"] "zqwfzhw5k2ollygh6nrjpsi7u4d4g6lu" [label="hwloc"] "vedchc5aoqyu3ydbp346qrbpe6kg46rq" [label="hdf5"] "wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" [label="numactl"] "komekkmyciga3kl24edjmredhj3uyt7v" [label="xz"] "es377uqsqougfc67jyg7yfjyyuukin52" [label="openssl"] "vfrf7asfclt7epufnoxibfqbkntbk5k3" [label="silo"] "smoyzzo2qhzpn6mg6rd3l2p7b23enshg" [label="zlib"] "sz72vygmht66khd5aa4kihz5alg4nrbm" [label="macsio"] "wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" -> "jdxbjftheiotj6solpomva7dowrhlerl" "zqwfzhw5k2ollygh6nrjpsi7u4d4g6lu" -> "4sh6pymrm2ms4auu3ajbjjr6fiuhz5g7" "sz72vygmht66khd5aa4kihz5alg4nrbm" -> "vfrf7asfclt7epufnoxibfqbkntbk5k3" "vfrf7asfclt7epufnoxibfqbkntbk5k3" -> "t54jzdy2jj4snltjazlm3br2urcilc6v" "crhlefo3dv7lmsv5pf4icsy4gepkdorm" -> "4sh6pymrm2ms4auu3ajbjjr6fiuhz5g7" "7tkgwjvu2mi4ea2wsdetunq7g4k4r2nh" -> "mkc3u4x2p2wie6jfhuku7g5rkovcrxps" "sz72vygmht66khd5aa4kihz5alg4nrbm" -> "pmsyupw6w3gql4loaor25gfumlmvkl25" "zqwfzhw5k2ollygh6nrjpsi7u4d4g6lu" -> "wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" "7tkgwjvu2mi4ea2wsdetunq7g4k4r2nh" -> "d2krmb5gweivlnztcymhklzsqbrpatt6" "es377uqsqougfc67jyg7yfjyyuukin52" -> "smoyzzo2qhzpn6mg6rd3l2p7b23enshg" "bltycqwh5oofai4f6o42q4uuj4w5zb3j" -> "crhlefo3dv7lmsv5pf4icsy4gepkdorm" "mm33a3ocsv3jsh2tfxc4mlab4xsurtdd" -> "zfdvt2jjuaees43ffrrtphqs2ky3o22t" "es377uqsqougfc67jyg7yfjyyuukin52" -> "zfdvt2jjuaees43ffrrtphqs2ky3o22t" "7tkgwjvu2mi4ea2wsdetunq7g4k4r2nh" -> "jdxbjftheiotj6solpomva7dowrhlerl" "mkc3u4x2p2wie6jfhuku7g5rkovcrxps" -> "lbrx7lnfz46ukewxbhxnucmx76g23c6q" "bltycqwh5oofai4f6o42q4uuj4w5zb3j" -> "es377uqsqougfc67jyg7yfjyyuukin52" "vedchc5aoqyu3ydbp346qrbpe6kg46rq" -> "smoyzzo2qhzpn6mg6rd3l2p7b23enshg" "wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" -> "d2krmb5gweivlnztcymhklzsqbrpatt6" "zfdvt2jjuaees43ffrrtphqs2ky3o22t" -> "4av4gywgpaspkhy3dvbb62nulqogtzbb" "vedchc5aoqyu3ydbp346qrbpe6kg46rq" -> "pmsyupw6w3gql4loaor25gfumlmvkl25" "d2krmb5gweivlnztcymhklzsqbrpatt6" -> "mm33a3ocsv3jsh2tfxc4mlab4xsurtdd" "bob4o5m3uku6vtdil5imasprgy775zg7" -> "jdxbjftheiotj6solpomva7dowrhlerl" "yn2r3wfhiilelyulh5toteicdtxjhw7d" -> "komekkmyciga3kl24edjmredhj3uyt7v" "pmsyupw6w3gql4loaor25gfumlmvkl25" -> "smoyzzo2qhzpn6mg6rd3l2p7b23enshg" "wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" -> "mm33a3ocsv3jsh2tfxc4mlab4xsurtdd" "vfrf7asfclt7epufnoxibfqbkntbk5k3" -> "vedchc5aoqyu3ydbp346qrbpe6kg46rq" "bob4o5m3uku6vtdil5imasprgy775zg7" -> "gs6ag7ktdoiirb62t7bcagjw62szrrg2" "d2krmb5gweivlnztcymhklzsqbrpatt6" -> "zfdvt2jjuaees43ffrrtphqs2ky3o22t" "7tkgwjvu2mi4ea2wsdetunq7g4k4r2nh" -> "mm33a3ocsv3jsh2tfxc4mlab4xsurtdd" "vfrf7asfclt7epufnoxibfqbkntbk5k3" -> "smoyzzo2qhzpn6mg6rd3l2p7b23enshg" "zfdvt2jjuaees43ffrrtphqs2ky3o22t" -> "4ihuiazsglf22f3pntq5hc4kyszqzexn" "bob4o5m3uku6vtdil5imasprgy775zg7" -> "4sh6pymrm2ms4auu3ajbjjr6fiuhz5g7" "vfrf7asfclt7epufnoxibfqbkntbk5k3" -> "pmsyupw6w3gql4loaor25gfumlmvkl25" "zqwfzhw5k2ollygh6nrjpsi7u4d4g6lu" -> "bob4o5m3uku6vtdil5imasprgy775zg7" "yn2r3wfhiilelyulh5toteicdtxjhw7d" -> "jearpk4xci4zc7dkrza4fufaqfkq7rfl" "sz72vygmht66khd5aa4kihz5alg4nrbm" -> "bltycqwh5oofai4f6o42q4uuj4w5zb3j" "pmsyupw6w3gql4loaor25gfumlmvkl25" -> "wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" "sz72vygmht66khd5aa4kihz5alg4nrbm" -> "7tkgwjvu2mi4ea2wsdetunq7g4k4r2nh" "yn2r3wfhiilelyulh5toteicdtxjhw7d" -> "smoyzzo2qhzpn6mg6rd3l2p7b23enshg" "t54jzdy2jj4snltjazlm3br2urcilc6v" -> "crhlefo3dv7lmsv5pf4icsy4gepkdorm" "pmsyupw6w3gql4loaor25gfumlmvkl25" -> "zqwfzhw5k2ollygh6nrjpsi7u4d4g6lu" "4av4gywgpaspkhy3dvbb62nulqogtzbb" -> "t54jzdy2jj4snltjazlm3br2urcilc6v" "jdxbjftheiotj6solpomva7dowrhlerl" -> "mkc3u4x2p2wie6jfhuku7g5rkovcrxps" "yn2r3wfhiilelyulh5toteicdtxjhw7d" -> "4sh6pymrm2ms4auu3ajbjjr6fiuhz5g7" "mm33a3ocsv3jsh2tfxc4mlab4xsurtdd" -> "mkc3u4x2p2wie6jfhuku7g5rkovcrxps" "zqwfzhw5k2ollygh6nrjpsi7u4d4g6lu" -> "yn2r3wfhiilelyulh5toteicdtxjhw7d" "pmsyupw6w3gql4loaor25gfumlmvkl25" -> "4sh6pymrm2ms4auu3ajbjjr6fiuhz5g7" "wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" -> "mkc3u4x2p2wie6jfhuku7g5rkovcrxps" }

Now we are ready to begin work on the actual application.

Development iteration cycles

Let’s assume that scr has a bug, and we’d like to patch scr to find out what the problem is. First, we tell Spack that we’d like to check out the version of scr in our environment. In this case, it will be the 2.0.0 release that we want to write a patch for:

$ spack develop scr
==> Cloning source code for scr@=2.0.0
==> Fetching https://mirror.spack.io/_source-cache/archive/47/471978ae0afb56a20847d3989b994fbd680d1dea21e77a5a46a964b6e3deed6b.tar.gz
    [100%]  955.92 KB @	  81.0 MB/s
==> Updating view at /home/spack/devel-env/.spack-env/view
$ spack config blame develop
---				     develop:
/home/spack/devel-env/spack.yaml:16    scr:
/home/spack/devel-env/spack.yaml:17	 spec: scr@=2.0.0
$ spack find -cv scr
==> In environment /home/spack/devel-env
==> 1 root specs
-- no arch / no compilers ---------------------------------------
 -  macsio+scr

-- linux-ubuntu22.04-x86_64_v3 / %c,cxx,[email protected] ------
 -   [email protected]+dtcmp~fortran~ipo+libyogrt async_api=NONE build_system=cmake build_type=Release cache_base=/dev/shm cntl_base=/dev/shm copy_config=none dev_path=/home/spack/devel-env/scr file_lock=FLOCK generator=make resource_manager=SLURM scr_config=scr.conf
==> 0 installed packages
==> 1 concretized package to be installed

The spack develop command marks the package as being a “development” package based on the supplied develop spec. Develop specs are listed in their own develop section inside the spack.yaml. The mechanics of how this section is used to enforce development are as follows:

  1. Specs in the environment that satisfy the develop specs are selected for development.

  2. Any specs selected in step 1 receive a dev_path= variant. This variant tells Spack where to find the source code for the spec. Adding the variant modifies the DAG hash of the spec and all of its downstream dependencies.

  3. Calls to spack install will now use the source code at dev_path when building that package.

  4. Spack doesn’t clean this build up after a successful build so subsequent calls to spack install trigger incremental builds.

If the environment is already concretized, spack develop performs step 1 and 2 in situ and updates the spack.lock file by default. If the environment is not yet concretized, the selection of develop specs and assignment of dev_path are handled during concretization.

So how does Spack determine the value of the dev_path variant? By default, the source code is downloaded into a subdirectory of the environment using Spack’s staging functionality. You can change the location of this source directory by modifying the path: attribute of the develop configuration in the environment or by passing the --path options when calling spack develop.

Now that we have this done, we tell Spack to rebuild both scr and macsio by running spack install.

$ spack install
[+] /usr (external glibc-2.35-qg7qyazcn2fwrck5vgr3mxq3i4uxkhlo)
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/compiler-wrapper-1.0-ntccuj2fi3y7asqifeq3i4iylrbxakw2
[+] /usr (external gcc-11.4.0-ml7cem5pfeoluzbhb7jsiisemvoauaz5)
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gcc-runtime-11.4.0-nokfxvaa4aklxfhoymvz6hlvbqw3kwnh
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/json-cwx-0.12-xb3rsbkkz6mb4b2rapjkeq5iginffvwi
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/numactl-2.0.18-yzbm5q6nblzcjccy7kddijqdnkgkxvtp
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libgpg-error-1.55-jerwahojdgk4ga5b5vopwmu4oto356qx
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/zstd-1.5.7-foiizhdg2mc4jdjtuddksbzvr64roxea
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pcre2-10.44-z5vjj7qwopnby63vu35fgh5ojadzfc4x
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pdsh-2.31-eicmuq5waz3gebhupmaxeo5bw5cl2gkm
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/mbedtls-2.28.9-bz3ghzheol2ecbi3vpldgbb3yd7yblle
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/berkeley-db-18.1.40-vdgigswy4fvttrivbedl4cp7bl72esma
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/ncurses-6.5-20250705-ncdxq3juvboefpavgmovg5rb5bx76ohz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libxcrypt-4.4.38-yiij42powrfh2mwjebwiqxmkvinutofr
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/zlib-ng-2.0.7-qlavhjbsgqyboovfvsultjdwzz5nvthw
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/xz-5.6.3-yzaocbs7geczi5d7qyvmqwrxi4r7dvph
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/lz4-1.10.0-7m7ijnovafqfvcxitqrbveal5qgz57cy
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gmake-4.4.1-4obn7cgqfwoxyvmnpeedud63bsjhoqma
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libssh2-1.11.1-txa2olxuoxfkv7jdjrdw27djcf6fa5sz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libedit-3.1-20240808-hdzcfgipukcqxsw7hjkksstbbz5otx3x
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libffi-3.5.2-zgro4twqg3qnq6hx6ohsjlejatth3rxa
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/json-c-0.18-5viqezvbwekimujkx7dj3xc6qlwzmqes
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/openssl-3.6.0-gv7wpik32unrnickoxbgm5iqza572w6s
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pkgconf-2.5.1-f4qiprwjcd2q3fp63uzgp47f3kw66r5h
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pigz-2.8-kaz756eya6nj3ardls4b6aiaclkaux7i
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/bzip2-1.0.8-x7t4najic2jb46srjiebkrf55utda3nl
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libmd-1.1.0-gth3ii5boey5akbsen5la4rgqpeaqkmx
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/readline-8.3-c6d2zljdklrjdya4xy236brmpdonc2xo
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libgcrypt-1.11.2-4j7zjzsf5hbmbypjyd4gqmpy6yp2esmn
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libiconv-1.18-qtepnkrdvqazp3jmfnheapckbemejrhq
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libevent-2.1.12-lprginh6npwjirdhmghjqjckaa7ukrri
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/hdf5-1.14.6-7mm6knnnjxquabuuhme4sjpe6d6vfawh
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/util-linux-uuid-2.41-n45otd3wy4fv7iawv52hiogfs63hxfpz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libpciaccess-0.17-txbwcin227323qmxmg4opnj4r4tvenz4
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/nghttp2-1.48.0-ft5kpbdiz6kctcooz4ksam37pskskehg
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libbsd-0.12.2-ucrhsywwalxwcqigisozcu2xvvhx3zwx
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gdbm-1.25-aq7qwy6yezwhtwdrsyqen65rmjmlth27
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/sqlite-3.50.4-elvodybm3egmq2kdk74ca3jdhzcnxu77
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/munge-0.5.15-itsnhnf3jn5ual4k7intv62hnu54gplq
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libxml2-2.13.5-5trxrsws5dig3bf63uc4xzcbsywsqtjo
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/tar-1.35-lfgvgvawnpdxxz7prdfn4vsfgtwruxjc
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/curl-8.15.0-isdtvvdziidtozwacfgtdcv3cukd7uuz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/expat-2.7.3-7yr5v6w2xu5f4bohfxjcsnb2snnwwj5m
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/perl-5.42.0-62kt5y4kl7uhwljviixr3m4dyf5vre7t
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/hwloc-2.12.2-4hos3725nynk2bsskofvtcmk6pmtcjg4
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gettext-0.23.1-cakgj4ntlncc4zsgty5z6sa7f7mwvlt7
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/cmake-3.31.9-ivvor7vdsqvplv65yyfragoxppyfi33t
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pmix-6.0.0-yqlblh6p4u6vwmfhtffyrnxyqrliw2jm
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/krb5-1.21.3-hnmy4fwwly5s4xifp5roodj6opppwwiu
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/python-3.11.14-rbf2x4id5wuf7yihgm6fv5vtm2qoc2bz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/elfutils-0.193-brbwl6lkh323duhprbcxlmwwk7zz2cs2
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/prrte-4.0.0-buin62mintd2cczzj6vsvai45vhpgg6w
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/openssh-9.9p1-cxdcxo5pdxc2nk7domxigdqiygg4757e
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/python-venv-1.0-essoyki3al5aam2wqamlejoclcd7jf6o
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/openmpi-5.0.8-tqxbnvoiw3nzds4xypcualamx3cvisln
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/glib-2.86.1-arsxi4dynj54sx2h3mcryrco7pml4yhf
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/silo-4.11.1-vsguusygbz767eclcacoq2hzu3ivoeez
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/lwgrp-1.0.6-bms4pwx7whtr74fbz6idyyk5lbgpv72z
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/slurm-25-05-1-1-tc55ttmephfx5nx3s7o7lem6w2hbke34
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/dtcmp-1.1.5-2kcmd2x76so5ites4co66qnkqukjenot
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libyogrt-1.35-ylblit5vwtdqn2zoc7ghzfmmf46bgm5g
==> No binary for scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw found: installing from source
==> Installing scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw [62/63]
==> No patches needed for scr
==> scr: Executing phase: 'cmake'
==> scr: Executing phase: 'build'
==> scr: Executing phase: 'install'
==> scr: Successfully installed scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw
  Stage: 0.00s.	 Cmake: 1.41s.	Build: 4.38s.  Install: 0.18s.	Post-install: 0.04s.  Total: 6.09s
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw
==> No binary for macsio-1.1-3uiweakny3apccoagw7fczwrjwoeqxwu found: installing from source
==> Installing macsio-1.1-3uiweakny3apccoagw7fczwrjwoeqxwu [63/63]
==> Fetching https://mirror.spack.io/_source-cache/archive/a8/a86249b0f10647c0b631773db69568388094605ec1a0af149d9e61e95e6961ec.tar.gz
    [100%]   64.50 MB @	 239.3 MB/s
==> Applied patch /home/spack/.spack/package_repos/fncqgg4/repos/spack_repo/builtin/packages/macsio/cast.patch
==> macsio: Executing phase: 'cmake'
==> macsio: Executing phase: 'build'
==> macsio: Executing phase: 'install'
==> macsio: Successfully installed macsio-1.1-3uiweakny3apccoagw7fczwrjwoeqxwu
  Stage: 0.16s.	 Cmake: 1.16s.	Build: 1.29s.  Install: 0.05s.	Post-install: 0.02s.  Total: 2.76s
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/macsio-1.1-3uiweakny3apccoagw7fczwrjwoeqxwu
==> Updating view at /home/spack/devel-env/.spack-env/view

This rebuilds scr from the subdirectory we specified. If your package uses CMake, Spack will build the package in a build directory that matches the hash for your package. From here you can change into the appropriate directory and perform your own build/test cycles.

Now, we can develop our code. For the sake of this demo, we’re just going to intentionally introduce an error. Let’s edit a file and remove the first semi-colon we find.

$ $EDITOR scr/src/scr_copy.c

Once you have a development package, spack install also works much like “make”. Since Spack knows the source code directory of the package, it checks the filetimes on the source directory to see if we’ve made recent changes. If the file times are newer, it will rebuild scr and any other package that depends on scr.

$ spack install
[+] /usr (external glibc-2.35-qg7qyazcn2fwrck5vgr3mxq3i4uxkhlo)
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/compiler-wrapper-1.0-ntccuj2fi3y7asqifeq3i4iylrbxakw2
[+] /usr (external gcc-11.4.0-ml7cem5pfeoluzbhb7jsiisemvoauaz5)
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gcc-runtime-11.4.0-nokfxvaa4aklxfhoymvz6hlvbqw3kwnh
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gmake-4.4.1-4obn7cgqfwoxyvmnpeedud63bsjhoqma
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libgpg-error-1.55-jerwahojdgk4ga5b5vopwmu4oto356qx
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/ncurses-6.5-20250705-ncdxq3juvboefpavgmovg5rb5bx76ohz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pdsh-2.31-eicmuq5waz3gebhupmaxeo5bw5cl2gkm
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/berkeley-db-18.1.40-vdgigswy4fvttrivbedl4cp7bl72esma
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/json-cwx-0.12-xb3rsbkkz6mb4b2rapjkeq5iginffvwi
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/bzip2-1.0.8-x7t4najic2jb46srjiebkrf55utda3nl
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libmd-1.1.0-gth3ii5boey5akbsen5la4rgqpeaqkmx
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/json-c-0.18-5viqezvbwekimujkx7dj3xc6qlwzmqes
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/xz-5.6.3-yzaocbs7geczi5d7qyvmqwrxi4r7dvph
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pcre2-10.44-z5vjj7qwopnby63vu35fgh5ojadzfc4x
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/mbedtls-2.28.9-bz3ghzheol2ecbi3vpldgbb3yd7yblle
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/numactl-2.0.18-yzbm5q6nblzcjccy7kddijqdnkgkxvtp
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libffi-3.5.2-zgro4twqg3qnq6hx6ohsjlejatth3rxa
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/zlib-ng-2.0.7-qlavhjbsgqyboovfvsultjdwzz5nvthw
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/nghttp2-1.48.0-ft5kpbdiz6kctcooz4ksam37pskskehg
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/zstd-1.5.7-foiizhdg2mc4jdjtuddksbzvr64roxea
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libxcrypt-4.4.38-yiij42powrfh2mwjebwiqxmkvinutofr
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libiconv-1.18-qtepnkrdvqazp3jmfnheapckbemejrhq
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libpciaccess-0.17-txbwcin227323qmxmg4opnj4r4tvenz4
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pkgconf-2.5.1-f4qiprwjcd2q3fp63uzgp47f3kw66r5h
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/lz4-1.10.0-7m7ijnovafqfvcxitqrbveal5qgz57cy
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libgcrypt-1.11.2-4j7zjzsf5hbmbypjyd4gqmpy6yp2esmn
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/readline-8.3-c6d2zljdklrjdya4xy236brmpdonc2xo
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libbsd-0.12.2-ucrhsywwalxwcqigisozcu2xvvhx3zwx
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pigz-2.8-kaz756eya6nj3ardls4b6aiaclkaux7i
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libssh2-1.11.1-txa2olxuoxfkv7jdjrdw27djcf6fa5sz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/openssl-3.6.0-gv7wpik32unrnickoxbgm5iqza572w6s
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/hdf5-1.14.6-7mm6knnnjxquabuuhme4sjpe6d6vfawh
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libxml2-2.13.5-5trxrsws5dig3bf63uc4xzcbsywsqtjo
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/util-linux-uuid-2.41-n45otd3wy4fv7iawv52hiogfs63hxfpz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libedit-3.1-20240808-hdzcfgipukcqxsw7hjkksstbbz5otx3x
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gdbm-1.25-aq7qwy6yezwhtwdrsyqen65rmjmlth27
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/sqlite-3.50.4-elvodybm3egmq2kdk74ca3jdhzcnxu77
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/expat-2.7.3-7yr5v6w2xu5f4bohfxjcsnb2snnwwj5m
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/tar-1.35-lfgvgvawnpdxxz7prdfn4vsfgtwruxjc
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/curl-8.15.0-isdtvvdziidtozwacfgtdcv3cukd7uuz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libevent-2.1.12-lprginh6npwjirdhmghjqjckaa7ukrri
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/munge-0.5.15-itsnhnf3jn5ual4k7intv62hnu54gplq
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/hwloc-2.12.2-4hos3725nynk2bsskofvtcmk6pmtcjg4
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/perl-5.42.0-62kt5y4kl7uhwljviixr3m4dyf5vre7t
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gettext-0.23.1-cakgj4ntlncc4zsgty5z6sa7f7mwvlt7
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/cmake-3.31.9-ivvor7vdsqvplv65yyfragoxppyfi33t
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pmix-6.0.0-yqlblh6p4u6vwmfhtffyrnxyqrliw2jm
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/krb5-1.21.3-hnmy4fwwly5s4xifp5roodj6opppwwiu
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/elfutils-0.193-brbwl6lkh323duhprbcxlmwwk7zz2cs2
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/python-3.11.14-rbf2x4id5wuf7yihgm6fv5vtm2qoc2bz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/prrte-4.0.0-buin62mintd2cczzj6vsvai45vhpgg6w
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/openssh-9.9p1-cxdcxo5pdxc2nk7domxigdqiygg4757e
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/python-venv-1.0-essoyki3al5aam2wqamlejoclcd7jf6o
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/openmpi-5.0.8-tqxbnvoiw3nzds4xypcualamx3cvisln
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/glib-2.86.1-arsxi4dynj54sx2h3mcryrco7pml4yhf
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/lwgrp-1.0.6-bms4pwx7whtr74fbz6idyyk5lbgpv72z
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/silo-4.11.1-vsguusygbz767eclcacoq2hzu3ivoeez
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/slurm-25-05-1-1-tc55ttmephfx5nx3s7o7lem6w2hbke34
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/dtcmp-1.1.5-2kcmd2x76so5ites4co66qnkqukjenot
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libyogrt-1.35-ylblit5vwtdqn2zoc7ghzfmmf46bgm5g
==> No binary for scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw found: installing from source
==> Installing scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw [62/63]
==> No patches needed for scr
==> scr: Executing phase: 'cmake'
==> scr: Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    '/home/spack/spack/opt/spack/linux-x86_64_v3/gmake-4.4.1-4obn7cgqfwoxyvmnpeedud63bsjhoqma/bin/make' '-j4'

4 errors found in build log:
     147    cd /tmp/spack/spack-stage/spack-stage-scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw/
	    spack-build-xuzclql && /home/spack/spack/opt/spack/linux-x86_64_v3/cmake-3.31.9-i
	    vvor7vdsqvplv65yyfragoxppyfi33t/bin/cmake -E cmake_depends "Unix Makefiles" /home
	    /spack/devel-env/scr /home/spack/devel-env/scr/src /tmp/spack/spack-stage/spack-s
	    tage-scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw/spack-build-xuzclql /tmp/spack/sp
	    ack-stage/spack-stage-scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw/spack-build-xuzc
	    lql/src /tmp/spack/spack-stage/spack-stage-scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjo
	    xfyw/spack-build-xuzclql/src/CMakeFiles/scr-static.dir/DependInfo.cmake "--color=
	    "
     148    [ 81%] Built target scr_transfer
     149    make[2]: Leaving directory '/tmp/spack/spack-stage/spack-stage-scr-2.0.0-xuzclql7
	    mxpy3jegwg6po3ewjbjoxfyw/spack-build-xuzclql'
     150    make[2]: Leaving directory '/tmp/spack/spack-stage/spack-stage-scr-2.0.0-xuzclql7
	    mxpy3jegwg6po3ewjbjoxfyw/spack-build-xuzclql'
     151    /home/spack/spack/opt/spack/linux-x86_64_v3/gmake-4.4.1-4obn7cgqfwoxyvmnpeedud63b
	    sjhoqma/bin/make  -f src/CMakeFiles/scr.dir/build.make src/CMakeFiles/scr.dir/bui
	    ld
     152    /home/spack/spack/opt/spack/linux-x86_64_v3/gmake-4.4.1-4obn7cgqfwoxyvmnpeedud63b
	    sjhoqma/bin/make  -f src/CMakeFiles/scr-static.dir/build.make src/CMakeFiles/scr-
	    static.dir/build
  >> 153    /home/spack/devel-env/scr/src/scr_copy.c:44:1: error: expected ',' or ';' before
	    'int'
     154       44 | int print_usage()
     155	  | ^~~
     156    make[2]: Entering directory '/tmp/spack/spack-stage/spack-stage-scr-2.0.0-xuzclql
	    7mxpy3jegwg6po3ewjbjoxfyw/spack-build-xuzclql'
     157    make[2]: Nothing to be done for 'src/CMakeFiles/scr.dir/build'.
     158    make[2]: Leaving directory '/tmp/spack/spack-stage/spack-stage-scr-2.0.0-xuzclql7
	    mxpy3jegwg6po3ewjbjoxfyw/spack-build-xuzclql'
     159    make[2]: Entering directory '/tmp/spack/spack-stage/spack-stage-scr-2.0.0-xuzclql
	    7mxpy3jegwg6po3ewjbjoxfyw/spack-build-xuzclql'
     160    make[2]: Nothing to be done for 'src/CMakeFiles/scr-static.dir/build'.
     161    make[2]: Leaving directory '/tmp/spack/spack-stage/spack-stage-scr-2.0.0-xuzclql7
	    mxpy3jegwg6po3ewjbjoxfyw/spack-build-xuzclql'
     162    /home/spack/devel-env/scr/src/scr_copy.c: In function 'process_args':
     163    /home/spack/devel-env/scr/src/scr_copy.c:135:9: warning: implicit declaration of
	    function 'print_usage' [-Wimplicit-function-declaration]
     164      135 |	    print_usage();
     165	  |	    ^~~~~~~~~~~
  >> 166    make[2]: *** [src/CMakeFiles/scr_copy.dir/build.make:82: src/CMakeFiles/scr_copy.
	    dir/scr_copy.c.o] Error 1
     167    make[2]: Leaving directory '/tmp/spack/spack-stage/spack-stage-scr-2.0.0-xuzclql7
	    mxpy3jegwg6po3ewjbjoxfyw/spack-build-xuzclql'
  >> 168    make[1]: *** [CMakeFiles/Makefile2:1336: src/CMakeFiles/scr_copy.dir/all] Error 2
     169    make[1]: *** Waiting for unfinished jobs....
     170    [ 82%] Built target scr
     171    [ 83%] Built target scr-static
     172    make[1]: Leaving directory '/tmp/spack/spack-stage/spack-stage-scr-2.0.0-xuzclql7
	    mxpy3jegwg6po3ewjbjoxfyw/spack-build-xuzclql'
  >> 173    make: *** [Makefile:149: all] Error 2
     174    ==> scr: Executing phase: 'install'
     175    ==> [2025-11-17-08:53:40.517325] '/home/spack/spack/opt/spack/linux-x86_64_v3/gma
	    ke-4.4.1-4obn7cgqfwoxyvmnpeedud63bsjhoqma/bin/make' '-j4' 'install'
     176    /home/spack/spack/opt/spack/linux-x86_64_v3/cmake-3.31.9-ivvor7vdsqvplv65yyfragox
	    ppyfi33t/bin/cmake -S/home/spack/devel-env/scr -B/tmp/spack/spack-stage/spack-sta
	    ge-scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw/spack-build-xuzclql --check-build-s
	    ystem CMakeFiles/Makefile.cmake 0
     177    /home/spack/spack/opt/spack/linux-x86_64_v3/cmake-3.31.9-ivvor7vdsqvplv65yyfragox
	    ppyfi33t/bin/cmake -E cmake_progress_start /tmp/spack/spack-stage/spack-stage-scr
	    -2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw/spack-build-xuzclql/CMakeFiles /tmp/spack
	    /spack-stage/spack-stage-scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw/spack-build-x
	    uzclql//CMakeFiles/progress.marks
     178    /home/spack/spack/opt/spack/linux-x86_64_v3/gmake-4.4.1-4obn7cgqfwoxyvmnpeedud63b
	    sjhoqma/bin/make  -f CMakeFiles/Makefile2 all
     179    make[1]: Entering directory '/tmp/spack/spack-stage/spack-stage-scr-2.0.0-xuzclql
	    7mxpy3jegwg6po3ewjbjoxfyw/spack-build-xuzclql'

See build log for details:
  /tmp/spack/spack-stage/spack-stage-scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw/spack-build-out.txt

==> Warning: Skipping build of macsio-1.1-3uiweakny3apccoagw7fczwrjwoeqxwu since scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw failed
==> Error: macsio-1.1-3uiweakny3apccoagw7fczwrjwoeqxwu: Package was not installed
==> Error: Installation request failed.	 Refer to reported errors for failing package(s).

Here, the build failed as expected. We can look at the output for the build in the stage directory scr/build-linux-*/spack-build-out.txt to find out why. The build-linux-* directory inside the source tree is a symlink to the spec’s stage directory where all the logs are stored. The full name of this directory can be found with spack location --stage scr or quickly navigated to with spack cd --stage scr. We can also launch a shell directly with the appropriate environment variables to figure out what went wrong by using spack build-env scr -- bash. If that’s too much to remember, then sourcing scr/build-linux-*/spack-build-env.txt will also set all the appropriate environment variables so we can diagnose the build ourselves. Now let’s fix it and rebuild directly.

$ $EDITOR scr/src/scr_copy.c
$ spack install
[+] /usr (external glibc-2.35-qg7qyazcn2fwrck5vgr3mxq3i4uxkhlo)
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/compiler-wrapper-1.0-ntccuj2fi3y7asqifeq3i4iylrbxakw2
[+] /usr (external gcc-11.4.0-ml7cem5pfeoluzbhb7jsiisemvoauaz5)
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gcc-runtime-11.4.0-nokfxvaa4aklxfhoymvz6hlvbqw3kwnh
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pkgconf-2.5.1-f4qiprwjcd2q3fp63uzgp47f3kw66r5h
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pdsh-2.31-eicmuq5waz3gebhupmaxeo5bw5cl2gkm
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libxcrypt-4.4.38-yiij42powrfh2mwjebwiqxmkvinutofr
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gmake-4.4.1-4obn7cgqfwoxyvmnpeedud63bsjhoqma
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/ncurses-6.5-20250705-ncdxq3juvboefpavgmovg5rb5bx76ohz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libiconv-1.18-qtepnkrdvqazp3jmfnheapckbemejrhq
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/json-c-0.18-5viqezvbwekimujkx7dj3xc6qlwzmqes
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/berkeley-db-18.1.40-vdgigswy4fvttrivbedl4cp7bl72esma
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/xz-5.6.3-yzaocbs7geczi5d7qyvmqwrxi4r7dvph
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libmd-1.1.0-gth3ii5boey5akbsen5la4rgqpeaqkmx
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/util-linux-uuid-2.41-n45otd3wy4fv7iawv52hiogfs63hxfpz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/json-cwx-0.12-xb3rsbkkz6mb4b2rapjkeq5iginffvwi
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/nghttp2-1.48.0-ft5kpbdiz6kctcooz4ksam37pskskehg
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libpciaccess-0.17-txbwcin227323qmxmg4opnj4r4tvenz4
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/mbedtls-2.28.9-bz3ghzheol2ecbi3vpldgbb3yd7yblle
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/numactl-2.0.18-yzbm5q6nblzcjccy7kddijqdnkgkxvtp
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/lz4-1.10.0-7m7ijnovafqfvcxitqrbveal5qgz57cy
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libffi-3.5.2-zgro4twqg3qnq6hx6ohsjlejatth3rxa
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/zstd-1.5.7-foiizhdg2mc4jdjtuddksbzvr64roxea
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pcre2-10.44-z5vjj7qwopnby63vu35fgh5ojadzfc4x
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libgpg-error-1.55-jerwahojdgk4ga5b5vopwmu4oto356qx
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/zlib-ng-2.0.7-qlavhjbsgqyboovfvsultjdwzz5nvthw
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/bzip2-1.0.8-x7t4najic2jb46srjiebkrf55utda3nl
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libedit-3.1-20240808-hdzcfgipukcqxsw7hjkksstbbz5otx3x
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/readline-8.3-c6d2zljdklrjdya4xy236brmpdonc2xo
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libbsd-0.12.2-ucrhsywwalxwcqigisozcu2xvvhx3zwx
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libgcrypt-1.11.2-4j7zjzsf5hbmbypjyd4gqmpy6yp2esmn
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libssh2-1.11.1-txa2olxuoxfkv7jdjrdw27djcf6fa5sz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pigz-2.8-kaz756eya6nj3ardls4b6aiaclkaux7i
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libxml2-2.13.5-5trxrsws5dig3bf63uc4xzcbsywsqtjo
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/openssl-3.6.0-gv7wpik32unrnickoxbgm5iqza572w6s
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/hdf5-1.14.6-7mm6knnnjxquabuuhme4sjpe6d6vfawh
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gdbm-1.25-aq7qwy6yezwhtwdrsyqen65rmjmlth27
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/sqlite-3.50.4-elvodybm3egmq2kdk74ca3jdhzcnxu77
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/expat-2.7.3-7yr5v6w2xu5f4bohfxjcsnb2snnwwj5m
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/tar-1.35-lfgvgvawnpdxxz7prdfn4vsfgtwruxjc
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/hwloc-2.12.2-4hos3725nynk2bsskofvtcmk6pmtcjg4
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libevent-2.1.12-lprginh6npwjirdhmghjqjckaa7ukrri
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/munge-0.5.15-itsnhnf3jn5ual4k7intv62hnu54gplq
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/curl-8.15.0-isdtvvdziidtozwacfgtdcv3cukd7uuz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/perl-5.42.0-62kt5y4kl7uhwljviixr3m4dyf5vre7t
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/gettext-0.23.1-cakgj4ntlncc4zsgty5z6sa7f7mwvlt7
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/pmix-6.0.0-yqlblh6p4u6vwmfhtffyrnxyqrliw2jm
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/cmake-3.31.9-ivvor7vdsqvplv65yyfragoxppyfi33t
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/python-3.11.14-rbf2x4id5wuf7yihgm6fv5vtm2qoc2bz
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/krb5-1.21.3-hnmy4fwwly5s4xifp5roodj6opppwwiu
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/elfutils-0.193-brbwl6lkh323duhprbcxlmwwk7zz2cs2
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/prrte-4.0.0-buin62mintd2cczzj6vsvai45vhpgg6w
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/python-venv-1.0-essoyki3al5aam2wqamlejoclcd7jf6o
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/openssh-9.9p1-cxdcxo5pdxc2nk7domxigdqiygg4757e
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/glib-2.86.1-arsxi4dynj54sx2h3mcryrco7pml4yhf
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/openmpi-5.0.8-tqxbnvoiw3nzds4xypcualamx3cvisln
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/slurm-25-05-1-1-tc55ttmephfx5nx3s7o7lem6w2hbke34
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/silo-4.11.1-vsguusygbz767eclcacoq2hzu3ivoeez
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/lwgrp-1.0.6-bms4pwx7whtr74fbz6idyyk5lbgpv72z
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/libyogrt-1.35-ylblit5vwtdqn2zoc7ghzfmmf46bgm5g
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/dtcmp-1.1.5-2kcmd2x76so5ites4co66qnkqukjenot
==> No binary for scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw found: installing from source
==> Installing scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw [62/63]
==> No patches needed for scr
==> scr: Executing phase: 'cmake'
==> scr: Executing phase: 'build'
==> scr: Executing phase: 'install'
==> scr: Successfully installed scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw
  Stage: 0.00s.	 Cmake: 0.00s.	Build: 0.26s.  Install: 0.17s.	Post-install: 0.04s.  Total: 0.60s
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/scr-2.0.0-xuzclql7mxpy3jegwg6po3ewjbjoxfyw
==> No binary for macsio-1.1-3uiweakny3apccoagw7fczwrjwoeqxwu found: installing from source
==> Installing macsio-1.1-3uiweakny3apccoagw7fczwrjwoeqxwu [63/63]
==> Using cached archive: /home/spack/spack/var/spack/cache/_source-cache/archive/a8/a86249b0f10647c0b631773db69568388094605ec1a0af149d9e61e95e6961ec.tar.gz
==> Applied patch /home/spack/.spack/package_repos/fncqgg4/repos/spack_repo/builtin/packages/macsio/cast.patch
==> macsio: Executing phase: 'cmake'
==> macsio: Executing phase: 'build'
==> macsio: Executing phase: 'install'
==> macsio: Successfully installed macsio-1.1-3uiweakny3apccoagw7fczwrjwoeqxwu
  Stage: 0.04s.	 Cmake: 1.17s.	Build: 1.29s.  Install: 0.05s.	Post-install: 0.02s.  Total: 2.65s
[+] /home/spack/spack/opt/spack/linux-x86_64_v3/macsio-1.1-3uiweakny3apccoagw7fczwrjwoeqxwu

You’ll notice here that Spack rebuilt both scr and macsio, as expected.

Taking advantage of iterative builds with Spack requires cooperation from your build system. When Spack performs a rebuild on a development package, it reruns all the build stages for your package without cleaning the source and build directories to a pristine state. If your build system can take advantage of the previously compiled object files then you’ll end up with an iterative build.

  • If your package just uses make, you also should get iterative builds for free when running spack develop.

  • If your package uses CMake with the typical cmake / build / install build stages, you’ll get iterative builds for free with Spack because CMake doesn’t modify the filetime on the CMakeCache.txt file if your cmake flags haven’t changed.

  • If your package uses autoconf, then rerunning the typical autoreconf stage typically modifies the filetime of config.h, which can trigger a cascade of rebuilding.

Multi-package development

You may have noticed that macsio is restaged and fully rebuilt each time we call spack install. Usually developers do not want to fully rebuild the canonical source every time; either for performance or because they are co-developing the two packages. Spack does not limit how many packages can be developed so spack develop can be applied to any spec in our environment including macsio. The --recursive option provides a convenient way to mark all downstream dependencies as develop specs.

$ spack develop --recursive scr
==> Skipping developer download of scr because its path /home/spack/devel-env/scr already exists.
==> Cloning source code for macsio@=1.1
==> Using cached archive: /home/spack/spack/var/spack/cache/_source-cache/archive/a8/a86249b0f10647c0b631773db69568388094605ec1a0af149d9e61e95e6961ec.tar.gz
==> Updating view at /home/spack/devel-env/.spack-env/view
$ spack find -cv macsio
==> In environment /home/spack/devel-env
==> 1 root specs
-- no arch / no compilers ---------------------------------------
 -  macsio+scr

-- linux-ubuntu22.04-x86_64_v3 / %c,[email protected] --------------
 -   [email protected]~exodus~hdf5~ipo+mpi~pdb+scr+silo~szip~typhonio~zfp~zlib build_system=cmake build_type=Release dev_path=/home/spack/devel-env/macsio generator=make patches:=59479b9
==> 0 installed packages
==> 1 concretized package to be installed

spack develop --recursive can only be used with a concrete environment. When called Spack traces the graph from the supplied develop spec to every root in the graph that transitively depends on the develop package. Using --recursive can be very powerful when developing applications deep in a graph. In this case our development point is very close to the root spec so we could have called spack develop macsio and gotten the same result.

Pre-configuring development environments

So far all of our calls to spack develop have been on a concretized environment, and we have allowed Spack to automatically update the build specs for us. If we don’t want Spack to update the concrete environment’s specs we can pass the ---no-modify-concrete-spec. Using ---no-modify-concrete-spec will require you to force concretize an environment to have the develop specs take affect.

There is a limited set of use-cases where one might want to use this option. Some example cases include:

  • Updating a develop spec before updating the environment to change a variant or version

  • Adding a develop spec that is not yet in the environment

  • Debugging unexpected behavior

For illustrative purposes we will show an example of switching scr to a debug build via the build_type=Debug variant.

$ spack develop scr build_type=Debug
==> Defaulting to highest version: scr@develop
==> Skipping developer download of scr because its path /home/spack/devel-env/scr already exists.
==> Error: Develop spec 'scr@=develop build_type=Debug' conflicts with concrete specs in environment. Try again with 'spack develop --no-modify-concrete-specs' and run 'spack concretize --force' to apply your changes.
$ spack develop --no-modify-concrete-specs scr build_type=Debug
==> Defaulting to highest version: scr@develop
==> Skipping developer download of scr because its path /home/spack/devel-env/scr already exists.

We see that naively updating the develops spec, resulted first in an error and then an undesired version change. To preserve the version and get the new variant added we run the following commands:

$ # Not the verison we wanted. This time lets add a version
$ spack develop --no-modify-concrete-specs [email protected] build_type=Debug
==> Skipping developer download of scr because its path /home/spack/devel-env/scr already exists.
$ spack concretize --force
==> Concretized 1 spec:
 -   ltx53j7  [email protected]~exodus~hdf5~ipo+mpi~pdb+scr~silo~szip~typhonio~zfp~zlib build_system=cmake build_type=Release dev_path=/home/spack/devel-env/macsio generator=make patches:=59479b9 platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  ivvor7v	  ^[email protected]~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  isdtvvd	      ^[email protected]~gssapi~ldap~libidn2~librtmp~libssh+libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=mbedtls,openssl platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  txa2olx		  ^[email protected]+shared build_system=autotools crypto=mbedtls platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  yzaocbs		      ^[email protected]~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  bz3ghzh		  ^[email protected]+pic build_system=makefile build_type=Release libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  ft5kpbd		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  gv7wpik		  ^[email protected]~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  etqlnw5		      ^ca-certificates-mozilla@2025-08-12 build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  ncdxq3j	      ^[email protected]~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  qlavhjb	      ^[email protected] cflags=-O3 +compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  ntccuj2	  ^[email protected] build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[e]  ml7cem5	  ^[email protected]~binutils+bootstrap~graphite~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  nokfxva	  ^[email protected] build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[e]  qg7qyaz	  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  4obn7cg	  ^[email protected]~guile build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  xb3rsbk	  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  fgsf5ij	      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  pzmnwzm	      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  yt7ajy4	      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  irvryts		  ^[email protected] build_system=autotools patches:=440b954 platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  cakgj4n		      ^[email protected]+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  lfgvgva			  ^[email protected] build_system=autotools zip=pigz platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  kaz756e			      ^[email protected] build_system=makefile platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  foiizhd			      ^[email protected]+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  lxvpwti	      ^[email protected]+sigsegv build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  cih4xrz		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  qtepnkr		      ^[email protected] build_system=autotools libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  bs5ujst		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  tqxbnvo	  ^[email protected]+atomics~cuda~debug+fortran~gpfs~internal-hwloc~internal-libevent~internal-pmix~ipv6~java~lustre~memchecker~openshmem~rocm~romio+rsh~static~two_level_namespace+vt+wrapper-rpath build_system=autotools fabrics:=none romio-filesystem:=none schedulers:=none platform=linux os=ubuntu22.04 target=x86_64_v3 %c,cxx,[email protected]
[+]  4hos372	      ^[email protected]~cairo~cuda~gl~level_zero~libudev+libxml2~nvml~opencl+pci~rocm build_system=autotools libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  txbwcin		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  hwxnwvm		      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  5trxrsw		  ^[email protected]~http+pic~python+shared build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  lprginh	      ^[email protected]+openssl build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  yzbm5q6	      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  cxdcxo5	      ^[email protected]+gssapi build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  hnmy4fw		  ^[email protected]+shared build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  n7yzkyl		      ^[email protected]~color build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  hdzcfgi		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  yiij42p		  ^[email protected]~obsolete_api build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  62kt5y4	      ^[email protected]+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  vdgigsw		  ^[email protected]+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  x7t4naj		  ^[email protected]~debug~pic+shared build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  aq7qwy6		  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  c6d2zlj		      ^[email protected] build_system=autotools patches:=21f0a03 platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  f4qiprw	      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  yqlblh6	      ^[email protected]~munge~python build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  buin62m	      ^[email protected] build_system=autotools schedulers:=none platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  gmhq65u		  ^[email protected]+lex~nls build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
 -   wunzdqt	  ^[email protected]~bbapi~dw+examples~fortran~ipo+libyogrt+pdsh+pthreads+shared+tests build_system=cmake build_type=Debug cache_base=/dev/shm cntl_base=/dev/shm copy_config=none dev_path=/home/spack/devel-env/scr file_lock=FLOCK generator=make resource_manager=SLURM scr_config=scr.conf platform=linux os=ubuntu22.04 target=x86_64_v3 %c,cxx,[email protected]
 -   zxozdyw	      ^[email protected]~bbapi~bbapi_fallback~dw~ipo+mpi+pthreads+shared async_api:=daemon build_system=cmake build_type=Release generator=make platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  2kcmd2x	      ^[email protected]+shared build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  bms4pwx		  ^[email protected]+shared build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
 -   4qdy75u	      ^[email protected]~ipo+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
 -   vexogwp	      ^[email protected]~ipo+mpi+shared build_system=cmake build_type=Release file_lock=FLOCK generator=make platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  ylblit5	      ^[email protected]~static build_system=autotools scheduler=slurm platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  tc55ttm		  ^slurm@25-05-1-1~cgroup~gtk~hdf5~hwloc~mariadb~nvml~pam~pmix+readline~restd~rsmi build_system=autotools sysconfdir=PREFIX/etc platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  arsxi4d		      ^[email protected]+introspection~libmount~strip build_system=meson buildtype=release default_library:=shared tracing:=none platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  brbwl6l			  ^[email protected]~debuginfod+exeprefix+nls build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  zktpj7s			  ^[email protected]~strip build_system=meson buildtype=release default_library:=shared platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  dcark7n			      ^[email protected]~strip build_system=meson buildtype=release default_library:=shared platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  zgro4tw			  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  muzysky			  ^[email protected] build_system=python_pip patches:=0f0b1bd platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  hjc6yfd			      ^[email protected] build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  lnndrge			      ^[email protected] build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  hpfn7k7			      ^[email protected] build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  2stmhpc			  ^[email protected]+re2c build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  axmwics			      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  z5vjj7q			  ^[email protected]~jit+multibyte+pic build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  rbf2x4i			  ^[email protected]+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches:=13fa8bf,b0615b2,ebdca64,f2fd060 platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  7yr5v6w			      ^[email protected]+libbsd build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  ucrhsyw				  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  gth3ii5				      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  elvodyb			      ^[email protected]+column_metadata+fts+rtree build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  n45otd3			      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  essoyki			  ^[email protected] build_system=generic platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  5viqezv		      ^[email protected]~ipo build_system=cmake build_type=Release generator=make platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  7m7ijno		      ^[email protected]+pic build_system=makefile libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  itsnhnf		      ^[email protected] build_system=autotools localstatedir=PREFIX/var platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  4j7zjzs			  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  jerwaho			      ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  irhvzch				  ^[email protected]~nls build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  mj4re3l				      ^[email protected]+cxx build_system=autotools libs:=shared,static platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  m3nwalt				      ^[email protected] build_system=autotools libs:=shared,static patches:=3ec29a6 platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
[+]  7p7gq7u					  ^[email protected] build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3
[+]  k6ygirs					  ^[email protected]~xs build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
[+]  eicmuq5	      ^[email protected]+ssh+static_modules build_system=autotools platform=linux os=ubuntu22.04 target=x86_64_v3 %[email protected]
 -   quwelpn	      ^[email protected]~ipo+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
 -   27xkpnh	      ^[email protected]~cuda~ipo~openmp~pthreads+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
 -   k7tdoqr	      ^[email protected]~ipo+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]
 -   5usa3ye	      ^[email protected]~ipo+mpi+shared build_system=cmake build_type=Release generator=make platform=linux os=ubuntu22.04 target=x86_64_v3 %c,[email protected]

==> Updating view at /home/spack/devel-env/.spack-env/view

Some additional concerns to navigate for effective use of the spack develop command include:

  • spack add <package> with the matching version you want to develop is a way to ensure the develop spec is satisfied in the spack.yaml environments file.

  • If the spec is not already concrete in the environment, you need to provide Spack a spec version so it can supply the correct flags for the package’s build system.

  • If a version is not supplied or detectable in the environment, then Spack falls back to the maximum version defined in the package where infinity versions like develop and main have a higher value than the numeric versions.

  • The source code located at the spec’s dev_path is the users responsibility to manage. Spack does provide an initial clone of the source code, but it makes no guarantees or additional verification of the source beyond that. Users can manage the code locally via a version control system like git, or can trigger a re-stage by calling spack develop --force.

Sharing development environments

Using development workflows also lets us ship our whole development process to another developer on the team. They can simply take our spack.yaml, create a new environment, and use this to replicate our build process. For example, we’ll make another development environment here.

$ cd ~
$ mkdir devel-other
$ cd devel-other
$ cp ../devel-env/spack.yaml .
$ spacktivate .
$ spack develop
==> Cloning source code for scr@=3.1.0 build_type=Debug
==> Fetching https://mirror.spack.io/_source-cache/archive/ca/ca1f37c84e0ff7a307e68f213c8cc868974d7fb30f16826853a711c7c3a55ffa.tar.gz
    [100%]   61.27 MB @	 109.0 MB/s
==> Cloning source code for macsio@=1.1
==> Using cached archive: /home/spack/spack/var/spack/cache/_source-cache/archive/a8/a86249b0f10647c0b631773db69568388094605ec1a0af149d9e61e95e6961ec.tar.gz
$ ls
macsio	scr  spack.yaml

Here, spack develop with no arguments will check out or download the source code and place it in the appropriate places.

When we’re done developing, we simply tell Spack that it no longer needs to keep a development version of the package.

$ spack undevelop scr macsio
==> Undevelop: removing scr
==> Undevelop: removing macsio

Workflow Summary

Use the spack develop command with an environment to make a reproducible build environment for your development workflow. Spack will set up all the dependencies for you and link all your packages together. Within a development environment, spack install works similarly to make in that it will check file times to rebuild the minimum number of Spack packages necessary to reflect the changes to your build.

Optional: Tips and Tricks

This section will cover some additional features that are useful additions to the core tutorial above. Many of these items are very useful to specific projects and developers. A list of the options for the spack develop can be viewed below:

$ spack develop --help

Source Code Management

spack develop allows users to manipulate the source code locations The default behavior is to let Spack manage its location and cloning operations, but software developers often want more control over these.

The source directory can be set with the --path argument when calling spack develop. If this directory already exists then spack develop will not attempt to fetch the code for you. This allows developers to pre-clone the software or use preferred paths as they wish.

# pre-clone the source code and then point spack develop to it
# note that we can clone into any repo/branch combination desired
$ git clone https://github.com/llnl/scr.git $SPACK_ENV/code
# note that with `--path` the code directory and package name can be different
$ spack develop --path $SPACK_ENV/code [email protected]
$ spack concretize -f

Combinatorics

The final note we will look at in this tutorial will be the power of combinatoric development builds. There are many instances where developers want to see how a single set of changes affects multiple builds i.e. +cuda vs ~cuda, %gcc vs %clang, build_type=Release vs build_type=Debug, etc.

Developers can achieve builds of both cases from a single spack install as long as the develop spec is generic enough to cover the packages’ spec variations

# First we have to allow repeat specs in the environment
$ spack config add concretizer:unify:false
# Next we need to specify the specs we want ('==' propagates the variant to deps)
$ spack change macsio build_type==Release
$ spack add macsio+scr build_type==Debug
# Inspect the graph for multiple dev_path=
$ spack concretize -f

While we won’t build out this example it illustrates how the dev_path for build_type=Release and build_type=Debug points to the same source code.

Now if we want to do most of our incremental builds using the Release build and periodically check the results using the Debug build we can combine the workflow from the previous example: dive into the Release versions build environment using spack build-env scr build_type=Release -- bash and navigate with spack cd -b scr build_type=Release. Note that since there are two scr specs in the environment we must distinguish which one we want for these commands. When we are ready to check our changes for the debug build, we can exit out of the build environment subshell, rerun spack install to rebuild everything, and then inspect the debug build through our method of choice.