package version auto-location exceptions and mis-identification #282
Description
Working with gnu package directories, which are pretty regularly structured and I expected to work well, it seems that some oddities come up with the current spider approach. This actually turned up two separate but related issues.
The first is that loading a directory from an FTP url appears to cause issues, second is that .sig files are being recognized as potential archives and causing failures when spack attempts to treat them as tar archives.
See errors below:
rzalastor1@scogland:EDITOR='emacs -nw' spack create ftp://ftp.gnu.org/pub/gnu/emacs/emacs-24.5.tar.xz
==> This looks like a URL for emacs version 24.5.
==> Creating template for package emacs
Traceback (most recent call last):
File "/g/g12/scogland/spack/bin/spack", line 168, in <module>
main()
File "/g/g12/scogland/spack/bin/spack", line 146, in main
return_val = command(parser, args)
File "/g/g12/scogland/spack/lib/spack/spack/cmd/create.py", line 169, in create
versions = spack.util.web.find_versions_of_archive(url)
File "/g/g12/scogland/spack/lib/spack/spack/util/web.py", line 214, in find_versions_of_archive
p, l = spider(lurl, depth=list_depth)
File "/g/g12/scogland/spack/lib/spack/spack/util/web.py", line 176, in spider
pages, links = _spider((root_url, set(), root_url, None, 1, max_depth, False))
ValueError: need more than 0 values to unpack
rzalastor1@scogland:EDITOR='emacs -nw' spack create http://ftp.gnu.org/pub/gnu/emacs/emacs-24.5.tar.xz
==> This looks like a URL for emacs version 24.5.
==> Creating template for package emacs
==> Found 8 versions of emacs:
24.5.tar.xz.sighttp://ftp.gnu.org/pub/gnu/emacs/emacs-24.5.tar.xz.sig
24.5 http://ftp.gnu.org/pub/gnu/emacs/emacs-24.5.tar.xz
24.4.tar.xz.sighttp://ftp.gnu.org/pub/gnu/emacs/emacs-24.4.tar.xz.sig
24.4 http://ftp.gnu.org/pub/gnu/emacs/emacs-24.4.tar.xz
24.3.tar.xz.sighttp://ftp.gnu.org/pub/gnu/emacs/emacs-24.3.tar.xz.sig
24.3 http://ftp.gnu.org/pub/gnu/emacs/emacs-24.3.tar.xz
24.2.tar.xz.sighttp://ftp.gnu.org/pub/gnu/emacs/emacs-24.2.tar.xz.sig
24.2 http://ftp.gnu.org/pub/gnu/emacs/emacs-24.2.tar.xz
Include how many checksums in the package file? (default is 5, q to abort)
==> Downloading...
==> Trying to fetch from http://ftp.gnu.org/pub/gnu/emacs/emacs-24.5.tar.xz.sig
######################################################################## 100.0%
/bin/tar: This does not look like a tar archive
/bin/tar: Skipping to next header
/bin/tar: Exiting with failure status due to previous errors
==> Error: Command exited with status 2:
/bin/tar --exclude=*/*/* -tf /g/g12/scogland/spack/var/spack/stage/spack-stage-aC986I/emacs-24.5.tar.xz.sig