Skip to content

Add ChunkEncodingError to list of errors on which to retry requests #1118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
PGijsbers merged 1 commit into from
Oct 27, 2021
Merged

Add ChunkEncodingError to list of errors on which to retry requests #1118

PGijsbers merged 1 commit into from
Oct 27, 2021

Conversation

@PGijsbers
Copy link
Collaborator

@PGijsbers PGijsbers commented Oct 21, 2021

Encountered a ChunkEncodingError the other day which was reraised without retrying first. The error stems from a connectivity issue (as far as I can tell) and either way seemed to have disappeared when I tried again - so I think it's reasonable to include this in the set of errors which lead to retries.

  File "/repo/venv/lib/python3.7/site-packages/openml/tasks/functions.py", line 361, in get_task
    dataset = get_dataset(task.dataset_id, download_data, download_qualities=download_qualities)
  File "/repo/venv/lib/python3.7/site-packages/openml/datasets/functions.py", line 429, in get_dataset
    arff_file = _get_dataset_arff(description) if download_data else None
  File "/repo/venv/lib/python3.7/site-packages/openml/datasets/functions.py", line 1051, in _get_dataset_arff
    source=url, output_path=output_file_path, md5_checksum=md5_checksum_fixture
  File "/repo/venv/lib/python3.7/site-packages/openml/_api_calls.py", line 154, in _download_text_file
    response = __read_url(source, request_method="get", md5_checksum=md5_checksum)
  File "/repo/venv/lib/python3.7/site-packages/openml/_api_calls.py", line 206, in __read_url
    request_method=request_method, url=url, data=data, md5_checksum=md5_checksum
  File "/repo/venv/lib/python3.7/site-packages/openml/_api_calls.py", line 230, in _send_request
    response = session.get(url, params=data)
  File "/repo/venv/lib/python3.7/site-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/repo/venv/lib/python3.7/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/repo/venv/lib/python3.7/site-packages/requests/sessions.py", line 697, in send
    r.content
  File "/repo/venv/lib/python3.7/site-packages/requests/models.py", line 831, in content
    self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
  File "/repo/venv/lib/python3.7/site-packages/requests/models.py", line 756, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", Invalid
ChunkLength(got length b'', 0 bytes read))

@PGijsbers PGijsbers changed the title Improv/retry Add ChunkEncodingError to list of errors on which to retry requests Oct 21, 2021
@PGijsbers PGijsbers requested a review from mfeurer October 21, 2021 08:10
@PGijsbers
Add ChunkedError to list of retry exception
d9384c1 
Since it can stem from connectivity issues and it might not occur on a
retry.
mfeurer
mfeurer approved these changes Oct 27, 2021
View reviewed changes
Copy link
Collaborator

@mfeurer mfeurer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds very reasonable to me.

@PGijsbers PGijsbers merged commit 2984403 into develop Oct 27, 2021
@PGijsbers PGijsbers deleted the improv/retry branch October 27, 2021 12:38
github-actions bot pushed a commit that referenced this pull request Oct 27, 2021
Pieter Gijsbers: Add ChunkedError to list of retry exception (#1118)
ea70dda
PGijsbers added a commit to Mirkazemi/openml-python that referenced this pull request Feb 23, 2023
@PGijsbers
Add ChunkedError to list of retry exception (openml#1118)
7eed926 
Since it can stem from connectivity issues and it might not occur on a
retry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mfeurer mfeurer mfeurer approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@PGijsbers @mfeurer