splicing breaks .cmake and other text files: files contain paths invalidated by splice #50560
Description
Steps to reproduce
Text files installed from build cache, like *.cmake files, contain invalid dependency paths when we splice in a local, externally-installed mpich.
Reproducer is quick and simple using an existing binary cache (shown below):
$> docker run -it ecpe4s/e4s-release-base-cuda-amd64:latest
# Spack setup
root@e0ce78b40c1b:/# git clone https://github.com/eugeneswalker/spack
root@e0ce78b40c1b:/# git -C spack checkout a9c879d53e
root@e0ce78b40c1b:/# . spack/share/spack/setup-env.sh
root@e0ce78b40c1b:/# spack mirror add e4s https://cache.e4s.io/25.05rc0
root@e0ce78b40c1b:/# spack buildcache keys -it
# Reproducer files
root@e0ce78b40c1b:/# wget -q https://cache.e4s.io/eugenewalker-splicing-issue-environments.tgz
root@e0ce78b40c1b:/# tar xzf eugenewalker-splicing-issue-environments.tgz
x envs/
x envs/0-mpich/
x envs/0-mpich/spack.yaml
x envs/0-mpich/concretize.log
...
$> tree envs
envs
├── 0-mpich
│ ├── concretize.log
│ ├── spack.lock
│ └── spack.yaml
└── 1-axom
├── concretize.log
├── spack.lock
├── spack.yaml
└── spack.yaml.in
# Register MPICH external with Spack
root@e0ce78b40c1b:/# cd envs/0-mpich
root@e0ce78b40c1b:/envs/0-mpich# cat spack.yaml
spack:
...
packages:
...
mpich:
buildable: false
externals:
- spec: '[email protected] ~argobots~cuda+fortran~hwloc+hydra+libxml2+pci~rocm+romio~slurm~vci~verbs~wrapperrpath~xpmem build_system=autotools datatype-engine=auto device=ch4 netmod=ofi pmi=default target=x86_64_v3'
prefix: /usr/local/mpich/install/mpich
specs:
- mpich
root@e0ce78b40c1b:/envs/0-mpich# spack -e . install
[+] /usr/local/mpich/install/mpich (external mpich-4.2.3-47excoypwhfmhx57rfs6reouvninugcf)
root@e0ce78b40c1b:/envs/0-mpich# spack find -l mpich
-- linux-ubuntu22.04-x86_64_v3 / [email protected] ---------------------
47excoy [email protected]
==> 1 installed package
root@e0ce78b40c1b:/envs/0-mpich# spack location -i /47ex
/usr/local/mpich/install/mpich
# Install Axom from binary cache
root@e0ce78b40c1b:/# cd /envs/1-axom
root@e0ce78b40c1b:/envs/1-axom# export HASH=$(spack find --format '{hash}' mpich)
root@e0ce78b40c1b:/envs/1-axom# envsubst <spack.yaml.in >spack.yaml
root@e0ce78b40c1b:/envs/1-axom# time spack -e . concretize -f | tee concretize.log
root@e0ce78b40c1b:/envs/1-axom# time spack -e . install --cache-only
...
==> Installing axom-0.10.1-35r7ashoylcwjr53sry3twgqgvpplc4l [21/21]
==> Fetching https://cache.e4s.io/25.05rc0/build_cache/linux-ubuntu22.04-x86_64_v3-gcc-11.4.0-axom-0.10.1-gmgytaeoo2plus5oqu5omawzsvpplc4l.spec.json.sig
gpg: Signature made Wed Mar 26 02:46:01 2025 America
gpg: using RSA key 25645FA2B218FE55B4EF649E4345F04B40005581
gpg: Good signature from "University of Oregon - E4S" [ultimate]
==> Fetching https://cache.e4s.io/25.05rc0/build_cache/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/axom-0.10.1/linux-ubuntu22.04-x86_64_v3-gcc-11.4.0-axom-0.10.1-gmgytaeoo2plus5oqu5omawzsvpplc4l.spack
==> Extracting axom-0.10.1-35r7ashoylcwjr53sry3twgqgvpplc4l from binary cache
[+] /spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/axom-0.10.1-35r7ashoylcwjr53sry3twgqgvpplc4l
real 0m29.486s
user 0m5.281s
sys 0m1.753s
At this point we've installed axom from cache and spliced in a local, externally-installed mpich. The axom from the cache was originally built against a Spack-installed mpich. The hash of this Spack-installed
The problem is apparent when we look at the files installed with axom and see that the old, original mpich path is still contained in them. These paths should have been updated to the prefix for our spliced in target mpich.
See:
# See that the original mpich path is still present in numerous files in axom install tree
root@e0ce78b40c1b:/envs/1-axom# grep -Ilnr mpich-4.2.3-jtqgr $(spack location -i axom)
...
/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/axom-0.10.1-35r7ashoylcwjr53sry3twgqgvpplc4l/lib/cmake/axom-targets.cmake
...
root@e0ce78b40c1b:/envs/1-axom# grep mpich $(spack location -i axom)/lib/cmake/axom-targets.cmake
INTERFACE_LINK_LIBRARIES "hdf5-shared;m;dl;/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/mpich-4.2.3-jtqgrprucu7cyascfsa3msuvq4eoproq/lib/libmpi.so;/spack/opt/spack/linux-ubuntu22.04-x86_64_v3/gcc-11.4.0/zlib-ng-2.2.3-y3inxsiezsxb5eetx5xfp2syclcn7b2u/lib/libz.so"
Notice the mpich path is wrong!
Our splicing config is:
spack:
...
splice:
explicit:
- target: mpi
replacement: mpich/47excoypwhfmhx57rfs6reouvninugcf
transitive: false
...
Here is the root spec.json from reproducer (axom's spec.json): spec.json
Error message
No response
Information on your system
- Spack: 1.0.0.dev0 (a9c879d)
- Python: 3.10.12
- Platform: linux-ubuntu22.04-zen4
General information
- I have run
spack debug reportand reported the version of Spack/Python/Platform - I have searched the issues of this repo and believe this is not a duplicate
- I have run the failing commands in debug mode and reported the output