mozdownload sometimes fails on Taskcluster and Travis #13274
Description
At least twice some of the Firefox tasks for pushes to master have failed in mozdownload:
https://tools.taskcluster.net/groups/AGupAeh6TrSdyNlDxcqZXw (for commit 2df7f9f, now only discoverable via API)
https://tools.taskcluster.net/groups/eRtqwYxjTfeob0g4hHcOlw (for commit 91491de)
The most recent failure was:
Traceback (most recent call last):
File "./wpt", line 5, in <module>
wpt.main()
File "/home/test/web-platform-tests/tools/wpt/wpt.py", line 129, in main
rv = script(*args, **kwargs)
File "/home/test/web-platform-tests/tools/wpt/run.py", line 510, in run
**kwargs)
File "/home/test/web-platform-tests/tools/wpt/run.py", line 488, in setup_wptrunner
kwargs["binary"] = setup_cls.install(venv, channel=channel)
File "/home/test/web-platform-tests/tools/wpt/run.py", line 165, in install
return self.browser.install(venv.path, channel)
File "/home/test/web-platform-tests/tools/wpt/browser.py", line 134, in install
destination=dest).download()
File "/home/test/web-platform-tests/_venv/lib/python2.7/site-packages/mozdownload/factory.py", line 121, in __init__
scraper_types[scraper_type].__init__(self, **kwargs)
File "/home/test/web-platform-tests/_venv/lib/python2.7/site-packages/mozdownload/scraper.py", line 346, in __init__
Scraper.__init__(self, *args, **kwargs)
File "/home/test/web-platform-tests/_venv/lib/python2.7/site-packages/mozdownload/scraper.py", line 135, in __init__
self._retry_check_404(self.get_build_info)
File "/home/test/web-platform-tests/_venv/lib/python2.7/site-packages/mozdownload/scraper.py", line 150, in _retry_check_404
self._retry(func, **retry_kwargs)
File "/home/test/web-platform-tests/_venv/lib/python2.7/site-packages/mozdownload/scraper.py", line 141, in _retry
return redo.retry(func, **retry_kwargs)
File "/home/test/web-platform-tests/_venv/lib/python2.7/site-packages/redo/__init__.py", line 162, in retry
return action(*args, **kwargs)
File "/home/test/web-platform-tests/_venv/lib/python2.7/site-packages/mozdownload/scraper.py", line 403, in get_build_info
self.date, self.build_index)
File "/home/test/web-platform-tests/_venv/lib/python2.7/site-packages/mozdownload/scraper.py", line 484, in get_build_info_for_date
raise errors.NotFoundError(message, url)
mozdownload.errors.NotFoundError: Folder for builds on 2018-09-28-22-04-33 has not been found: https://archive.mozilla.org/pub/firefox/nightly/2018/09/
The previous was very similar except for the date: "mozdownload.errors.NotFoundError: Folder for builds on 2018-09-24-10-03-54 has not been found: https://archive.mozilla.org/pub/firefox/nightly/2018/09/"
That it's happened 4 days apart suggests that it wasn't just a transient problem with archive.mozilla.org.
Here's where the error is thrown:
https://github.com/mozilla/mozdownload/blob/866cfebe9b8137bfe7ba8411efbe9d0e9d24093a/mozdownload/scraper.py
@jgraham, can you take a look?