-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Checksum and Tar Errors from Several Packages from mantevo.github.io #15291
Copy link
Copy link
Closed
Labels
Description
This issue is essentially an expanded follow-up to #14340.
I've found the following packages from Mantevo that produce checksum failures and cannot be extracted from their tarballs during install:
cloverleafminimdminighostminismac2dminixycehpccgpathfindertealeaf
There are several things that these packages have in common:
- All come from this GitHub repo.
- The URL that Spack is using to get the tarball points to a Git LFS pointer file.
- The actual file being obtained with
curlseems to be the Git LFS pointer file. However, I'm still not positive that this is the case. - The pointer file's
oidfield matches the SHA256 checksum that is in the Spack package file. - The checksum that causes the mismatch can be reproduced by running
sha256sumon this pointer file.
Additionally, when I tried manually extracting the actual tarball for cloverleaf using curl on the command line (following these instructions), I got the following:
$ curl -X POST -d '{"operation":"download", "objects":[{"oid": "sha256:de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f ", "size": 1756656}]}' https://github.com/Mantevo/mantevo.github.io.git/info/lfs/objects/batch
{
"message": "This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.",
"documentation_url": "https://help.github.com/articles/purchasing-additional-storage-and-bandwidth-for-an-organization/"
}To keep this issue from getting too lengthy, I'm only going to include the output from trying to install cloverleaf below, but I have gotten the same errors for all of the above packages.
Spack version
$ spack --version
# On my personal laptop (Ubuntu 18.04 x86_64)
0.14.0-24-ae8782852
# On UTK's Power9 Cluster
0.14.0-21-676eb56Steps to reproduce the issue
# For the checksum error
$ spack install cloverleaf
# For the tarball extraction error
$ spack install -n cloverleafErrors
On my personal laptop:
$ spack install cloverleaf
==> 13573: Installing cloverleaf
==> Fetching from /home/ilumsden/spack/var/spack/cache/_source-cache/archive/de/de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f.tar.gz failed.
==> Fetching https://github.com/Mantevo/mantevo.github.io/raw/master/download_files/CloverLeaf-1.1.tar.gz
################################################################################################################################## 100.0%################################################################################################################################## 100.0%
==> Error: ChecksumError: sha256 checksum failed for /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz
Expected de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f but got 5a69bc3d1b4ca0d071609ecd844ba46636d5239a0acfa56e3c93fa606e112ab9
/home/ilumsden/spack/lib/spack/spack/package.py:1131, in do_fetch:
1128 self.stage.fetch(mirror_only)
1129 self._fetch_time = time.time() - start_time
1130
>> 1131 if checksum and self.version in self.versions:
1132 self.stage.check()
1133
1134 self.stage.cache_local()
==> Error: Failed to install cloverleaf due to ChildError: ChecksumError: sha256 checksum failed for /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz
Expected de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f but got 5a69bc3d1b4ca0d071609ecd844ba46636d5239a0acfa56e3c93fa606e112ab9
/home/ilumsden/spack/lib/spack/spack/package.py:1131, in do_fetch:
1128 self.stage.fetch(mirror_only)
1129 self._fetch_time = time.time() - start_time
1130
>> 1131 if checksum and self.version in self.versions:
1132 self.stage.check()
1133
1134 self.stage.cache_local()
Traceback (most recent call last):
File "/home/ilumsden/spack/lib/spack/spack/build_environment.py", line 801, in child_process
return_value = function()
File "/home/ilumsden/spack/lib/spack/spack/installer.py", line 984, in build_process
pkg.do_patch()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1167, in do_patch
self.do_stage()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1152, in do_stage
self.do_fetch(mirror_only)
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1132, in do_fetch
self.stage.check()
File "/home/ilumsden/spack/lib/spack/spack/util/pattern.py", line 68, in getter
getattr(item, self.name)(*args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/stage.py", line 490, in check
self.fetcher.check()
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 73, in wrapper
return fun(self, *args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 492, in check
"Expected %s but got %s" % (self.digest, checker.sum))
spack.fetch_strategy.ChecksumError: sha256 checksum failed for /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz
Expected de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f but got 5a69bc3d1b4ca0d071609ecd844ba46636d5239a0acfa56e3c93fa606e112ab9
$ spack install -n cloverleaf
==> 13599: Installing cloverleaf
==> Fetching from /home/ilumsden/spack/var/spack/cache/_source-cache/archive/de/de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f.tar.gz failed.
==> Fetching https://github.com/Mantevo/mantevo.github.io/raw/master/download_files/CloverLeaf-1.1.tar.gz
################################################################################################################################## 100.0%################################################################################################################################## 100.0%
==> Staging archive: /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz
/bin/tar: This does not look like a tar archive
gzip: stdin: not in gzip format
/bin/tar: Child returned status 1
/bin/tar: Error is not recoverable: exiting now
==> Error: ProcessError: Command exited with status 2:
'/bin/tar' '-xf' '/tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz'
==> Error: Failed to install cloverleaf due to ChildError: ProcessError: Command exited with status 2:
'/bin/tar' '-xf' '/tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz'Traceback (most recent call last):
File "/home/ilumsden/spack/lib/spack/spack/build_environment.py", line 801, in child_process
return_value = function()
File "/home/ilumsden/spack/lib/spack/spack/installer.py", line 984, in build_process
pkg.do_patch()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1167, in do_patch
self.do_stage()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1153, in do_stage
self.stage.expand_archive()
File "/home/ilumsden/spack/lib/spack/spack/util/pattern.py", line 68, in getter
getattr(item, self.name)(*args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/stage.py", line 528, in expand_archive
self.fetcher.expand()
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 73, in wrapper
return fun(self, *args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 434, in expand
decompress(self.archive_file)
File "/home/ilumsden/spack/lib/spack/spack/util/executable.py", line 189, in __call__
proc.returncode, long_msg)
spack.util.executable.ProcessError: Command exited with status 2:
'/bin/tar' '-xf' '/tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz'On UTK's Power9 Cluster:
$ spack install cloverleaf
==> 94167: Installing cloverleaf
==> Fetching https://github.com/Mantevo/mantevo.github.io/raw/master/download_files/CloverLeaf-1.1.tar.gz
################################################################################################################################## 100.0%################################################################################################################################## 100.0%
==> Error: ChecksumError: sha256 checksum failed for /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz
Expected de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f but got 5a69bc3d1b4ca0d071609ecd844ba46636d5239a0acfa56e3c93fa606e112ab9
/home/ilumsden/spack/lib/spack/spack/package.py:1131, in do_fetch:
1128 self.stage.fetch(mirror_only)
1129 self._fetch_time = time.time() - start_time
1130
>> 1131 if checksum and self.version in self.versions:
1132 self.stage.check()
1133
1134 self.stage.cache_local()
==> Error: Failed to install cloverleaf due to ChildError: ChecksumError: sha256 checksum failed for /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz
Expected de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f but got 5a69bc3d1b4ca0d071609ecd844ba46636d5239a0acfa56e3c93fa606e112ab9
/home/ilumsden/spack/lib/spack/spack/package.py:1131, in do_fetch:
1128 self.stage.fetch(mirror_only)
1129 self._fetch_time = time.time() - start_time
1130
>> 1131 if checksum and self.version in self.versions:
1132 self.stage.check()
1133
1134 self.stage.cache_local()
Traceback (most recent call last):
File "/home/ilumsden/spack/lib/spack/spack/build_environment.py", line 801, in child_process
return_value = function()
File "/home/ilumsden/spack/lib/spack/spack/installer.py", line 984, in build_process
pkg.do_patch()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1167, in do_patch
self.do_stage()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1152, in do_stage
self.do_fetch(mirror_only)
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1132, in do_fetch
self.stage.check()
File "/home/ilumsden/spack/lib/spack/spack/util/pattern.py", line 68, in getter
getattr(item, self.name)(*args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/stage.py", line 490, in check
self.fetcher.check()
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 73, in wrapper
return fun(self, *args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 492, in check
"Expected %s but got %s" % (self.digest, checker.sum))
spack.fetch_strategy.ChecksumError: sha256 checksum failed for /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz
Expected de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f but got 5a69bc3d1b4ca0d071609ecd844ba46636d5239a0acfa56e3c93fa606e112ab9
$ spack install -n cloverleaf
==> 98004: Installing cloverleaf
==> Fetching https://github.com/Mantevo/mantevo.github.io/raw/master/download_files/CloverLeaf-1.1.tar.gz
################################################################################################################################## 100.0%################################################################################################################################## 100.0%
==> Staging archive: /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz
/usr/bin/tar: This does not look like a tar archive
gzip: stdin: not in gzip format
/usr/bin/tar: Child returned status 1
/usr/bin/tar: Error is not recoverable: exiting now
==> Error: ProcessError: Command exited with status 2:
'/usr/bin/tar' '-xf' '/tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz'
==> Error: Failed to install cloverleaf due to ChildError: ProcessError: Command exited with status 2:
'/usr/bin/tar' '-xf' '/tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz'Traceback (most recent call last):
File "/home/ilumsden/spack/lib/spack/spack/build_environment.py", line 801, in child_process
return_value = function()
File "/home/ilumsden/spack/lib/spack/spack/installer.py", line 984, in build_process
pkg.do_patch()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1167, in do_patch
self.do_stage()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1153, in do_stage
self.stage.expand_archive()
File "/home/ilumsden/spack/lib/spack/spack/util/pattern.py", line 68, in getter
getattr(item, self.name)(*args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/stage.py", line 528, in expand_archive
self.fetcher.expand()
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 73, in wrapper
return fun(self, *args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 434, in expand
decompress(self.archive_file)
File "/home/ilumsden/spack/lib/spack/spack/util/executable.py", line 189, in __call__
proc.returncode, long_msg)
spack.util.executable.ProcessError: Command exited with status 2:
'/usr/bin/tar' '-xf' '/tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz'Platform and user environment
On my personal laptop:
$ uname -a
Linux ilumsden-XPS-15-9560 5.0.0-37-generic #40~18.04.1-Ubuntu SMP Thu Nov 14 12:06:39 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -d
Description: Ubuntu 18.04.3 LTS
On UTK's Power9 Cluster:
$ uname -a
Linux tellico-master0 4.14.0-115.10.1.el7a.ppc64le #1 SMP Wed Jun 26 09:32:17 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux
$ lsb_release -d
Description: Red Hat Enterprise Linux Server release 7.6 (Maipo)
Additional information
General information
- I have run
spack --versionand reported the version of Spack - I have searched the issues of this repo and believe this is not a duplicate
Reactions are currently unavailable