Skip to content

requests[security] / PyOpenSSL does not make requests more secure but more brittle #5267

@tiran

Description

@tiran

Requests added the requests[security] dependency many, many years ago because the ssl module in Python's stdlib was lacking some features. Since PEP 466 has been implemented in Python 2.7.9, the ssl module supports hostname verification and SNI. 2.7.9 was released in 2014. Python 3.7.0 improved the situation even further by delegating hostname verification to OpenSSL. The PyOpenSSL compatibility layer in urllib3 uses the deprecated ssl.match_hostname function or a backport.

Unconditional monkey-patching of urllib3 with urllib3.contrib.pyopenssl.inject_into_urllib3 is causing issues, too.

PyOpenSSL uses dynamic libffi callbacks (also known as cffi old-style callbacks). The callbacks are implemented with trampolines and dynamic creation of native machine code. This either uses executable and writeable memory pages or some hacks with shared mmap() regions. Dynamic callbacks are a security risk and blocked by security frameworks like SELinux for good reasons (deny_execmem). Armin Rigo and I explored various ways to work around these problems, but there is simply solution. Eventually Armin implemented a new callback system for cffi. Some of the issues with old callbacks are documented at https://cffi.readthedocs.io/en/latest/using.html#callbacks-old-style .

PyOpenSSL currently pulls in a versions of cryptography, which itself depends on asn1crypto. A problem with asn1crypto and ctypes causes Python to segfault on recent macOS, pyca/pyopenssl#874 and wbond/asn1crypto#158 .

Due to bugs like https://bugzilla.redhat.com/show_bug.cgi?id=1535689 I convinced the Fedora and RHEL maintainers to patch requests and drop inject_into_urllib3 a while ago. The distros have been running with patch https://src.fedoraproject.org/rpms/python-requests/blob/master/f/Don-t-inject-pyopenssl-into-urllib3.patch for over a year without any reported issues.

I propose to:

  1. Remove requests[security] extra requires.
  2. Remove unconditionally monkey-patch of urllib3 from requests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions