Skip to content

Commit 6a0ecc6

Browse files
pquentinpquentin
authored
Update v2 migration guide to 2.4.0 (#3621)
* Update v2 migration guide to 2.4.0 * Fix version number deprecation * Update ssl_version string in tests * Adjust more test deprecation matches * Bump 2.5.0 removals to 2.6.0
1 parent 8e32e60 commit 6a0ecc6

5 files changed

Lines changed: 137 additions & 174 deletions

File tree

‎docs/v2-migration-guide.rst‎

Lines changed: 130 additions & 167 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
v2.0 Migration Guide
22
====================
33

4-
**urllib3 v2.0 is now available!** Read below for how to get started and what is contained in the new major release.
4+
**urllib3 2.x is now available!** Read below for how to get started and what is contained in the new major release.
55

6-
**🚀 Migrating from 1.x to 2.0**
6+
**🚀 Migrating from 1.x to 2.x**
77
--------------------------------
88

99
We're maintaining **functional API compatibility for most users** to make the
@@ -13,44 +13,25 @@ So unless you're in a specific situation you should notice no changes! 🎉
1313

1414
.. note::
1515

16-
If you have difficulty migrating to v2.0 or following this guide
16+
If you have difficulty migrating to 2.x or following this guide
1717
you can `open an issue on GitHub <https://github.com/urllib3/urllib3/issues>`_
1818
or reach out in `our community Discord channel <https://discord.gg/urllib3>`_.
1919

2020

21-
Timeline for deprecations and breaking changes
22-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
23-
24-
The 2.x initial release schedule will look like this:
25-
26-
* urllib3 ``v2.0.0-alpha1`` will be released in November 2022. This release
27-
contains **minor breaking changes and deprecation warnings for other breaking changes**.
28-
There may be other pre-releases to address fixes before v2.0.0 is released.
29-
* urllib3 ``v2.0.0`` will be released in early 2023 after some initial integration testing
30-
against dependent packages and fixing of bug reports.
31-
* urllib3 ``v2.1.0`` will be released in the summer of 2023 with **all breaking changes
32-
being warned about in v2.0.0**.
33-
34-
.. warning::
35-
36-
Please take the ``DeprecationWarnings`` you receive when migrating from v1.x to v2.0 seriously
37-
as they will become errors after 2.1.0 is released.
38-
39-
4021
What are the important changes?
4122
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4223

43-
Here's a short summary of which changes in urllib3 v2.0 are most important:
24+
Here's a short summary of which changes in urllib3 2.x are most important:
4425

45-
- Python version must be **3.7 or later** (previously supported Python 2.7, 3.5, and 3.6).
26+
- Python version must be **3.9 or later** (previously supported Python 2.7, and 3.5 to 3.8).
4627
- Removed support for non-OpenSSL TLS libraries (like LibreSSL and wolfSSL).
4728
- Removed support for OpenSSL versions older than 1.1.1.
4829
- Removed support for Python implementations that aren't CPython or PyPy3 (previously supported Google App Engine, Jython).
4930
- Removed the ``urllib3.contrib.ntlmpool`` module.
50-
- Deprecated the ``urllib3.contrib.pyopenssl``, ``urllib3.contrib.securetransport`` modules, will be removed in v2.1.0.
51-
- Deprecated the ``urllib3[secure]`` extra, will be removed in v2.1.0.
52-
- Deprecated the ``HTTPResponse.getheaders()`` method in favor of ``HTTPResponse.headers``, will be removed in v2.1.0.
53-
- Deprecated the ``HTTPResponse.getheader(name, default)`` method in favor of ``HTTPResponse.headers.get(name, default)``, will be removed in v2.1.0.
31+
- Removed the ``urllib3.contrib.securetransport`` module.
32+
- Removed the ``urllib3[secure]`` extra.
33+
- Deprecated the ``HTTPResponse.getheaders()`` method in favor of ``HTTPResponse.headers``, will be removed in 2.6.0.
34+
- Deprecated the ``HTTPResponse.getheader(name, default)`` method in favor of ``HTTPResponse.headers.get(name, default)``, will be removed in 2.6.0.
5435
- Deprecated URLs without a scheme (ie 'https://') and will be raising an error in a future version of urllib3.
5536
- Changed the default minimum TLS version to TLS 1.2 (previously was TLS 1.0).
5637
- Changed the default request body encoding from 'ISO-8859-1' to 'UTF-8'.
@@ -60,129 +41,22 @@ Here's a short summary of which changes in urllib3 v2.0 are most important:
6041
For a full list of changes you can look at `the changelog <https://github.com/urllib3/urllib3/blob/main/CHANGES.rst>`_.
6142

6243

63-
Migrating as a package maintainer?
64-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
65-
66-
If you're a maintainer of a package that uses urllib3 under the hood then this section is for you.
67-
You may have already seen an issue opened from someone on our team about the upcoming release.
68-
69-
The primary goal for migrating to urllib3 v2.x should be to ensure your package supports **both urllib3 v1.26.x and v2.0 for some time**.
70-
This is to reduce the chance that diamond dependencies are introduced into your users' dependencies which will then cause issues
71-
with them upgrading to the latest version of **your package**.
72-
73-
The first step to supporting urllib3 v2.0 is to make sure the version v2.x not being excluded by ``install_requires``. You should
74-
ensure your package allows for both urllib3 1.26.x and 2.0 to be used:
75-
76-
.. code-block:: python
77-
78-
# setup.py (setuptools)
79-
setup(
80-
...
81-
install_requires=["urllib3>=1.26,<3"]
82-
)
83-
84-
# pyproject.toml (hatch)
85-
[project]
86-
dependencies = [
87-
"urllib3>=1.26,<3"
88-
]
89-
90-
Next you should try installing urllib3 v2.0 locally and run your test suite.
91-
92-
.. code-block:: bash
93-
94-
$ python -m pip install -U --pre 'urllib3>=2.0.0a1'
95-
96-
97-
Because there are many ``DeprecationWarnings`` you should ensure that you're
98-
able to see those warnings when running your test suite. To do so you can add
99-
the following to your test setup to ensure even ``DeprecationWarnings`` are
100-
output to the terminal:
101-
102-
.. code-block:: bash
103-
104-
# Set PYTHONWARNING=default to show all warnings.
105-
$ export PYTHONWARNINGS="default"
106-
107-
# Run your test suite and look for failures.
108-
# Pytest automatically prints all warnings.
109-
$ pytest tests/
110-
111-
or you can opt-in within your Python code:
112-
113-
.. code-block:: python
114-
115-
# You can change warning filters according to the filter rules:
116-
# https://docs.python.org/3/library/warnings.html#warning-filter
117-
import warnings
118-
warnings.filterwarnings("default", category=DeprecationWarning)
119-
120-
Any failures or deprecation warnings you receive should be fixed as urllib3 v2.1.0 will remove all
121-
deprecated features. Many deprecation warnings will make suggestions about what to do to avoid the deprecated feature.
122-
123-
Warnings will look something like this:
124-
125-
.. code-block:: bash
126-
127-
DeprecationWarning: 'ssl_version' option is deprecated and will be removed
128-
in urllib3 v2.1.0. Instead use 'ssl_minimum_version'
129-
130-
Continue removing deprecation warnings until there are no more. After this you can publish a new release of your package
131-
that supports both urllib3 v1.26.x and v2.x.
132-
133-
.. note::
134-
135-
If you're not able to support both 1.26.x and v2.0 of urllib3 at the same time with your package please
136-
`open an issue on GitHub <https://github.com/urllib3/urllib3/issues>`_ or reach out in
137-
`our community Discord channel <https://discord.gg/urllib3>`_.
138-
139-
140-
Migrating as an application developer?
141-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
142-
143-
If you're someone who writes Python but doesn't ship as a package (things like web services, data science, tools, and more) this section is for you.
144-
145-
Python environments only allow for one version of a dependency to be installed per environment which means
146-
that **all of your dependencies using urllib3 need to support v2.0 for you to upgrade**.
147-
148-
The best way to visualize relationships between your dependencies is using `pipdeptree <https://pypi.org/project/pipdeptree>`_ and ``$ pipdeptree --reverse``:
149-
150-
.. code-block:: bash
151-
152-
# From inside your Python environment:
153-
$ python -m pip install pipdeptree
154-
# We only care about packages requiring urllib3
155-
$ pipdeptree --reverse | grep "requires: urllib3"
156-
157-
- botocore==1.29.8 [requires: urllib3>=1.25.4,<2]
158-
- requests==2.28.1 [requires: urllib3>=1.21.1,<2]
159-
160-
Reading the output from above, there are two packages which depend on urllib3: ``botocore`` and ``requests``.
161-
The versions of these two packages both require urllib3 that is less than v2.0 (ie ``<2``).
162-
163-
Because both of these packages require urllib3 before v2.0 the new version of urllib3 can't be installed
164-
by default. There are ways to force installing the newer version of urllib3 v2.0 (ie pinning to ``urllib3==2.0.0``)
165-
which you can do to test your application.
166-
167-
It's important to know that even if you don't upgrade all of your services to 2.x
168-
immediately you will `receive security fixes on the 1.26.x release stream <#security-fixes-for-urllib3-v1-26-x>` for some time.
169-
17044

171-
Security fixes for urllib3 v1.26.x
172-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
45+
Sunsetting urllib3 1.26.x
46+
~~~~~~~~~~~~~~~~~~~~~~~~~
17347

174-
Thanks to support from `Tidelift <https://tidelift.com/subscription/pkg/pypi-urllib3>`_
175-
we're able to continue supporting the v1.26.x release stream with
176-
security fixes for the foreseeable future 💖
48+
urllib3 1.26.x is not currently maintained. urllib3 2.x is the best version of urllib3
49+
and is widely supported by the larger Python ecosystem. That said, urllib3 1.26.x still
50+
sees significant download numbers, mainly because the botocore package still requires
51+
urllib3 1.26.x for Python 3.9 and earlier. If your organization would benefit from the
52+
continued support of urllib3 1.26.x, please contact [email protected] to
53+
discuss sponsorship or contribution opportunities.
17754

17855
However, upgrading is still recommended as **no new feature developments or non-critical
17956
bug fixes will be shipped to the 1.26.x release stream**.
18057

181-
If your organization relies on urllib3 and is interested in continuing support you can learn
182-
more about the `Tidelift Subscription for Enterprise <https://tidelift.com/subscription/pkg/pypi-urllib3?utm_source=pypi-urllib3&utm_medium=referral&utm_campaign=docs>`_.
183-
18458
**🤔 Common upgrading issues**
185-
-------------------------------
59+
------------------------------
18660

18761
ssl module is compiled with OpenSSL 1.0.2.k-fips
18862
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -194,15 +68,15 @@ ssl module is compiled with OpenSSL 1.0.2.k-fips
19468
19569
Remediation depends on your system:
19670

197-
- **AWS Lambda**: Upgrade to the Python3.10 runtime as it uses OpenSSL 1.1.1. Alternatively, you can
71+
- **AWS Lambda**: Upgrade to the Python 3.10 (or later) runtime as it uses OpenSSL 1.1.1. Alternatively, you can
19872
use a `custom Docker image
19973
<https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/>`_ and ensure you
20074
use a Python build that uses OpenSSL 1.1.1 or later.
20175
- **Amazon Linux 2**: Upgrade to `Amazon Linux 2023
20276
<https://aws.amazon.com/linux/amazon-linux-2023/>`_. Alternatively, you can install OpenSSL 1.1.1
20377
on Amazon Linux 2 using ``yum install openssl11 openssl11-devel`` and then install Python with a
20478
tool like pyenv.
205-
- **Red Hat Enterpritse Linux 7 (RHEL 7)**: Upgrade to RHEL 8 or RHEL 9.
79+
- **Red Hat Enterpritse Linux 7 (RHEL 7)**: Upgrade to RHEL 8 or later.
20680
- **Read the Docs**: Upgrade your `configuration file to use Ubuntu 22.04
20781
<https://docs.readthedocs.io/en/stable/config-file/v2.html>`_ by using ``os: ubuntu-22.04`` in the
20882
``build`` section. Feel free to use the `urllib3 configuration
@@ -228,12 +102,12 @@ library is unmaintained, but `replacements exist
228102
This likely happens because you're using botocore which `does not support urllib3 2.0 yet
229103
<https://github.com/boto/botocore/issues/2921>`_. The good news is that botocore explicitly declares
230104
in its dependencies that it only supports ``urllib3<2``. Make sure to use a recent pip. That way, pip
231-
will install urllib3 1.26.x until botocore starts supporting urllib3 2.0.
105+
will install urllib3 1.26.x for versions of botocore that do not support urllib3 2.0.
232106

233-
If you're deploying to an AWS environment such as Lambda or a host using Amazon Linux 2,
234-
you'll need to explicitly pin to ``urllib3<2`` in your project to ensure urllib3 2.0 isn't
235-
brought into your environment. Otherwise, this may result in unintended side effects with
236-
the default boto3 installation.
107+
If you're deploying to an AWS environment such as Lambda with the Python 3.9 runtime or a host
108+
using Amazon Linux 2, you'll need to explicitly pin to ``urllib3<2`` in your project to ensure
109+
urllib3 2.0 isn't brought into your environment. Otherwise, this may result in unintended side
110+
effects with the default boto3 installation.
237111

238112
AttributeError: module 'urllib3.connectionpool' has no attribute 'VerifiedHTTPSConnection'
239113
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -251,18 +125,108 @@ AttributeError: 'HTTPResponse' object has no attribute 'strict'
251125

252126
The ``strict`` parameter is unneeded with Python 3 and should be removed.
253127

254-
Pinning urllib3<2
255-
~~~~~~~~~~~~~~~~~
256128

257-
If the advice from the above sections did not help, you can pin urllib3 to 1.26.x by installing
258-
``urllib3<2``. Please do **not** specify ``urllib3==1.26.15`` to make sure you continue getting
259-
1.26.x updates!
260129

261-
While urllib3 1.26.x is still supported, it won't get new features or bug fixes, just security
262-
updates. Consider opening a tracking issue to unpin urllib3 in the future to not stay on 1.26.x
263-
indefinitely. For more details on the recommended way to handle your dependencies in general, see
264-
`Semantic Versioning Will Not Save You <https://hynek.me/articles/semver-will-not-save-you/>`_. The
265-
second half even uses urllib3 2.0 as an example!
130+
Migrating as an application developer?
131+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
132+
133+
If you're someone who writes Python but doesn't ship as a package (things like web services, data science, tools, and more) this section is for you.
134+
135+
Python environments only allow for one version of a dependency to be installed per environment which means
136+
that **all of your dependencies using urllib3 need to support 2.x for you to upgrade**.
137+
138+
The best way to visualize relationships between your dependencies is using `pipdeptree <https://pypi.org/project/pipdeptree>`_ and ``$ pipdeptree --reverse``:
139+
140+
.. code-block:: bash
141+
142+
# From inside your Python environment:
143+
$ python -m pip install pipdeptree
144+
# We only care about packages requiring urllib3
145+
$ pipdeptree --reverse | grep "requires: urllib3"
146+
├── botocore==1.38.36 [requires: urllib3>=1.25.4,<1.27]
147+
└── requests==2.32.4 [requires: urllib3>=1.21.1,<3]
148+
149+
Reading the output from above, there are two packages which depend on urllib3: ``botocore`` and ``requests``.
150+
While requests supports urllib3 2.x (with the ``<3`` specified), botocore requires urllib3 1.26.x (with the ``<1.27`` version specifier).
151+
Note that botocore does support urllib3 2.x, but it only supports it on Python 3.10 and later, which mandates OpenSSL 1.1.1+.
152+
153+
It's important to know `urllib3 does not receive security fixes at the moment <#sunsetting-urllib3-1-26-x>`.
154+
155+
156+
Migrating as a package maintainer?
157+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
158+
159+
If you're a maintainer of a package that uses urllib3 under the hood then this section is for you.
160+
You may have already seen an issue opened from someone on our team about the upcoming release.
161+
162+
The primary goal for migrating to urllib3 2.x should be to ensure your package supports **both urllib3 v1.26.x and 2.x for some time**.
163+
This is to reduce the chance that diamond dependencies are introduced into your users' dependencies which will then cause issues
164+
with them upgrading to the latest version of **your package**.
165+
166+
The first step to supporting urllib3 2.x is to make sure the version 2.x not being excluded by ``install_requires``. You should
167+
ensure your package allows for both urllib3 1.26.x and 2.x to be used:
168+
169+
.. code-block:: python
170+
171+
# setup.py (setuptools)
172+
setup(
173+
...
174+
install_requires=["urllib3>=1.26,<3"]
175+
)
176+
177+
# pyproject.toml (hatch)
178+
[project]
179+
dependencies = [
180+
"urllib3>=1.26,<3"
181+
]
182+
183+
Next you should try installing urllib3 v2.0 locally and run your test suite.
184+
185+
.. code-block:: bash
186+
187+
$ python -m pip install -U 'urllib3>=2'
188+
189+
Because there are new ``DeprecationWarnings`` you should ensure that you're
190+
able to see those warnings when running your test suite. To do so you can add
191+
the following to your test setup to ensure even ``DeprecationWarnings`` are
192+
output to the terminal:
193+
194+
.. code-block:: bash
195+
196+
# Set PYTHONWARNING=default to show all warnings.
197+
$ export PYTHONWARNINGS="default"
198+
199+
# Run your test suite and look for failures.
200+
# Pytest automatically prints all warnings.
201+
$ pytest tests/
202+
203+
or you can opt-in within your Python code:
204+
205+
.. code-block:: python
206+
207+
# You can change warning filters according to the filter rules:
208+
# https://docs.python.org/3/library/warnings.html#warning-filter
209+
import warnings
210+
warnings.filterwarnings("default", category=DeprecationWarning)
211+
212+
Any failures or deprecation warnings you receive should be fixed as urllib3 v2.1.0 will remove all
213+
deprecated features. Many deprecation warnings will make suggestions about what to do to avoid the deprecated feature.
214+
215+
Warnings will look something like this:
216+
217+
.. code-block:: bash
218+
219+
DeprecationWarning: 'ssl_version' option is deprecated and will be removed
220+
in urllib3 v2.6.0. Instead use 'ssl_minimum_version'
221+
222+
Continue removing deprecation warnings until there are no more. After this you can publish a new release of your package
223+
that supports both urllib3 1.26.x and 2.x.
224+
225+
.. note::
226+
227+
If you're not able to support both 1.26.x and 2.x of urllib3 at the same time with your package please
228+
`open an issue on GitHub <https://github.com/urllib3/urllib3/issues>`_ or reach out in
229+
`our community Discord channel <https://discord.gg/urllib3>`_.
266230

267231

268232
**💪 User-friendly features**
@@ -321,15 +285,14 @@ for requests and ``HTTPResponse.json()`` method on responses:
321285
}
322286
323287
324-
**✨ Optimized for Python 3.7+**
288+
**✨ Optimized for Python 3.9+**
325289
--------------------------------
326290

327-
In v2.0 we'll be specifically targeting
328-
CPython 3.7+ and PyPy 7.0+ (compatible with CPython 3.7)
329-
and dropping support for Python versions 2.7, 3.5, and 3.6.
330-
291+
urllib3 2.x specifically targets CPython 3.9+ and PyPy 7.3.17+ (compatible with CPython 3.10)
292+
and dropping support for Python versions 2.7, and 3.5 to 3.8.
293+
331294
By dropping end-of-life Python versions we're able to optimize
332-
the codebase for Python 3.7+ by using new features to improve
295+
the codebase for Python 3.9+ by using new features to improve
333296
performance and reduce the amount of code that needs to be executed
334297
in order to support legacy versions.
335298

‎src/urllib3/response.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -543,7 +543,7 @@ def readinto(self, b: bytearray) -> int:
543543
def getheaders(self) -> HTTPHeaderDict:
544544
warnings.warn(
545545
"HTTPResponse.getheaders() is deprecated and will be removed "
546-
"in urllib3 v2.1.0. Instead access HTTPResponse.headers directly.",
546+
"in urllib3 v2.6.0. Instead access HTTPResponse.headers directly.",
547547
category=DeprecationWarning,
548548
stacklevel=2,
549549
)
@@ -552,7 +552,7 @@ def getheaders(self) -> HTTPHeaderDict:
552552
def getheader(self, name: str, default: str | None = None) -> str | None:
553553
warnings.warn(
554554
"HTTPResponse.getheader() is deprecated and will be removed "
555-
"in urllib3 v2.1.0. Instead use HTTPResponse.headers.get(name, default).",
555+
"in urllib3 v2.6.0. Instead use HTTPResponse.headers.get(name, default).",
556556
category=DeprecationWarning,
557557
stacklevel=2,
558558
)

0 commit comments

Comments
 (0)