Skip to content

stream() "contract" changed in 1.10.3, can now raise IncompleteRead #673

@jmoldow

Description

@jmoldow

In 1.10.2, HTTPResponse.stream() always called HTTPResponse.read(). This method catches all HTTPException (including IncompleteRead), and re-raises them as ProtocolError.

In 1.10.3 and 1.10.4, HTTPResponse.stream() may call HTTPResponse.read_chunked() instead. This method may (via the _update_chunk_length() helper method) raise IncompleteRead. This exception will not be caught, and will be raised out of the call to HTTPResponse.stream().

A project I work on recently upgraded to requests 2.7.0 (which has vendored urllib3 1.10.4), and has started to see crashes because of IncompleteRead being raised from HTTPResponse.stream(). We previously would catch RequestException, as requests would catch ProtocolError (but not IncompleteRead) and re-raise a ChunkedEncodingError in Response.iter_content().

I'm not familiar with urllib3.util.retry, but at a glance it also seems like this change would negatively affect the _is_read_error() helper method.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions