Skip to content

cdo: fix URLs#10025

Merged
adamjstewart merged 1 commit intospack:developfrom
michaelkuhn:cdo-url
Dec 7, 2018
Merged

cdo: fix URLs#10025
adamjstewart merged 1 commit intospack:developfrom
michaelkuhn:cdo-url

Conversation

@michaelkuhn
Copy link
Copy Markdown
Member

Not sure how this ever worked but something seems to have changed so we now have to use the correct URLs.

@adamjstewart
Copy link
Copy Markdown
Member

Are you sure something changed? For me, it correctly falls back on list_url and finds the correct URL:

$ spack fetch cdo
==> Fetching https://code.mpimet.mpg.de/attachments/download/12760/cdo-1.9.5.tar.gz

curl: (22) The requested URL returned error: 404 Not Found
==> Fetching from https://code.mpimet.mpg.de/attachments/download/12760/cdo-1.9.5.tar.gz failed.
==> Fetching https://code.mpimet.mpg.de/attachments/download/18264/cdo-1.9.5.tar.gz
######################################################################## 100.0%

Of course, I'm not opposed to explicitly specifying the URL since it saves the time of looking at the wrong location.

@michaelkuhn
Copy link
Copy Markdown
Member Author

Huh, strange. This is what happens for me:

$ spack fetch cdo
==> Fetching https://code.mpimet.mpg.de/attachments/download/12760/cdo-1.9.5.tar.gz

curl: (22) The requested URL returned error: 404 Not Found
==> Fetching from https://code.mpimet.mpg.de/attachments/download/12760/cdo-1.9.5.tar.gz failed.
==> Fetching https://code.mpimet.mpg.de/attachments/18264/cdo-1.9.5.tar.gz

==> Warning: The contents of the archive look like HTML. Either the URL you are trying to use does not exist or you have an internet gateway issue. You can remove the bad archive using 'spack clean <package>', then try again using the correct URL.
==> Error: md5 checksum failed for .../cdo-1.9.5-ujq54kfqekw5pcjybm557fxa6sr6x2qw/cdo-1.9.5.tar.gz
Expected 0c60f2c94dc5c76421ecf363153a5043 but got 19f72e8ea96b05f7f1c97563f3e83bca

(Note the missing /download.)

@adamjstewart
Copy link
Copy Markdown
Member

@michaelkuhn I uncovered the source of the discrepancy! Things work fine for me with Python 3, but I get the exact same error as you for Python 2. Let me try to track this down. We should still probably merge this PR, as it doesn't hurt, but I hate when Spack behaves differently for Python 2 and 3.

@adamjstewart
Copy link
Copy Markdown
Member

Hmm, maybe this is less of a Python 2 bug and more of a non-deterministic bug:

$ python3 $(which spack) checksum cdo
==> Found 10 versions of cdo:
  
  1.9.6rc4  https://code.mpimet.mpg.de/attachments/18911/cdo-1.9.6rc4.tar.gz
  1.9.5     https://code.mpimet.mpg.de/attachments/download/18264/cdo-1.9.5.tar.gz
  1.9.4     https://code.mpimet.mpg.de/attachments/download/17374/cdo-1.9.4.tar.gz
  1.9.3     https://code.mpimet.mpg.de/attachments/download/16435/cdo-1.9.3.tar.gz
  1.9.2     https://code.mpimet.mpg.de/attachments/download/16035/cdo-1.9.2.tar.gz
  1.9.1     https://code.mpimet.mpg.de/attachments/15653/cdo-1.9.1.tar.gz
  1.9.0     https://code.mpimet.mpg.de/attachments/15187/cdo-1.9.0.tar.gz
  1.8.2     https://code.mpimet.mpg.de/attachments/download/14686/cdo-1.8.2.tar.gz
  1.7.2     https://code.mpimet.mpg.de/attachments/download/12760/cdo-1.7.2.tar.gz
  1.6.4     https://code.mpimet.mpg.de/attachments/8376/cdo-1.6.4.tar.gz

==> How many would you like to checksum? (default is 1, q to abort) q
==> Error: Aborted.
$ python2 $(which spack) checksum cdo
==> Found 10 versions of cdo:
  
  1.9.6rc4  https://code.mpimet.mpg.de/attachments/18911/cdo-1.9.6rc4.tar.gz
  1.9.5     https://code.mpimet.mpg.de/attachments/18264/cdo-1.9.5.tar.gz
  1.9.4     https://code.mpimet.mpg.de/attachments/download/17374/cdo-1.9.4.tar.gz
  1.9.3     https://code.mpimet.mpg.de/attachments/download/16435/cdo-1.9.3.tar.gz
  1.9.2     https://code.mpimet.mpg.de/attachments/16035/cdo-1.9.2.tar.gz
  1.9.1     https://code.mpimet.mpg.de/attachments/download/15653/cdo-1.9.1.tar.gz
  1.9.0     https://code.mpimet.mpg.de/attachments/15187/cdo-1.9.0.tar.gz
  1.8.2     https://code.mpimet.mpg.de/attachments/14686/cdo-1.8.2.tar.gz
  1.7.2     https://code.mpimet.mpg.de/attachments/12760/cdo-1.7.2.tar.gz
  1.6.4     https://code.mpimet.mpg.de/attachments/8376/cdo-1.6.4.tar.gz

==> How many would you like to checksum? (default is 1, q to abort) q
==> Error: Aborted.

The problem is that the download page contains links with and without the /download part of the path.

@adamjstewart
Copy link
Copy Markdown
Member

Fixed in #10047. I'll still merge this so that we don't have to rely on the list_url.

@adamjstewart adamjstewart merged commit 6ec8e41 into spack:develop Dec 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants