Add a graceful retry logic for intermittent network failure when downloading packages

In certain cases, it is possible that the network might not be reliable and be prone to outages. In these cases, it would be great if poetry's artifact download logic would recover gracefully. Ideally, with both an exponential back-off as well as a retry attempts.

A sample output when this happens.
```console
  • Installing html5lib (1.1)

ConnectionError

('Connection aborted.', error(54, 'Connection reset by peer'))

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 183, in _execute_operation
    result = self._do_execute_operation(operation)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 257, in _do_execute_operation
    result = getattr(self, '_execute_{}'.format(method))(operation)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 394, in _execute_install
    return self._install(operation)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 420, in _install
    archive = self._download(operation)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 574, in _download
    return self._download_link(operation, link)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 583, in _download_link
    archive = self._download_archive(operation, link)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 609, in _download_archive
    response = self._authenticator.request('get', link.url, stream=True)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/authenticator.py", line 66, in request
    resp = session.send(prepared_request, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)


  • Installing httpretty (0.9.7)

ERROR: InvocationError for command /usr/local/bin/poetry install -vv (exited with code 1)
```

The relevant logic could be implemented here.

https://github.com/python-poetry/poetry/blob/cae154c26485e3177f994f828b73d342aec12810/poetry/installation/authenticator.py#L37-L70

	def request(self, method, url, **kwargs): # type: (str, str, Any) -> Response
	from requests import Request # noqa
	from requests.auth import HTTPBasicAuth

	request = Request(method, url)

	username, password = self._get_credentials_for_url(url)

	if username is not None and password is not None:
	request = HTTPBasicAuth(username, password)(request)

	session = self.session
	prepared_request = session.prepare_request(request)

	proxies = kwargs.get("proxies", {})
	stream = kwargs.get("stream")
	verify = kwargs.get("verify")
	cert = kwargs.get("cert")

	settings = session.merge_environment_settings(
	prepared_request.url, proxies, stream, verify, cert
	)

	# Send the request.
	send_kwargs = {
	"timeout": kwargs.get("timeout"),
	"allow_redirects": kwargs.get("allow_redirects", True),
	}
	send_kwargs.update(settings)
	resp = session.send(prepared_request, **send_kwargs)

	resp.raise_for_status()

	return resp

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a graceful retry logic for intermittent network failure when downloading packages #2736

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add a graceful retry logic for intermittent network failure when downloading packages #2736

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions