****************************

What's New In Python 3.11

****************************

.. Rules for maintenance:

* Anyone can add text to this document. Do not spend very much time

on the wording of your changes, because your text will probably

get rewritten to some degree.

* The maintainer will go through Misc/NEWS periodically and add

changes; it's therefore more important to add your changes to

Misc/NEWS than to this file.

* This is not a complete list of every single change; completeness

is the purpose of Misc/NEWS. Some changes I consider too small

or esoteric to include. If such a change is added to the text,

I'll just remove it. (This is another reason you shouldn't spend

too much time on writing your addition.)

* If you want to draw your new text to the attention of the

maintainer, add 'XXX' to the beginning of the paragraph or

section.

* It's OK to just add a fragmentary note about a change. For

example: "XXX Describe the transmogrify() function added to the

socket module." The maintainer will research the change and

write the necessary text.

* You can comment out your additions if you like, but it's not

necessary (especially when a final release is some months away).

* Credit the author of a patch or bugfix. Just the name is

sufficient; the e-mail address isn't necessary.

* It's helpful to add the bug/patch number as a comment:

XXX Describe the transmogrify() function added to the socket

module.

(Contributed by P.Y. Developer in :issue:`12345`.)

This saves the maintainer the effort of going through the Mercurial log

when researching a change.

This article explains the new features in Python 3.11, compared to 3.10.

.. _whatsnew311-summary:

Summary -- Release highlights

=============================

.. This section singles out the most important changes in Python 3.11.

Brevity is key.

* Python 3.11 is between 10-60% faster than Python 3.10.

On average, we measured a 1.25x speedup on the standard benchmark suite.

See :ref:`whatsnew311-faster-cpython` for details.

.. PEP-sized items next.

New syntax features:

New built-in features:

New standard library modules:

* :pep:`680`: :mod:`tomllib` —

Support for parsing `TOML <https://toml.io/>`_ in the Standard Library

Interpreter improvements:

variable to :ref:`disable automatically prepending potentially unsafe paths

<whatsnew311-pythonsafepath>` to :data:`sys.path`

New typing features:

* :ref:`whatsnew311-pep646`

* :ref:`whatsnew311-pep655`

* :ref:`whatsnew311-pep673`

* :ref:`whatsnew311-pep675`

* :ref:`whatsnew311-pep681`

* :pep:`594`:

:ref:`Many legacy standard library modules have been deprecated

<whatsnew311-pep594>` and will be removed in Python 3.13

* :pep:`624`:

:ref:`Py_UNICODE encoder APIs have been removed <whatsnew311-pep624>`

* :pep:`670`:

:ref:`Macros converted to static inline functions <whatsnew311-pep670>`

.. _whatsnew311-features:

New Features

============

.. _whatsnew311-pep657:

PEP 657: Fine-grained error locations in tracebacks

---------------------------------------------------

When printing tracebacks, the interpreter will now point to the exact expression

.. code-block:: python

Traceback (most recent call last):

File "distance.py", line 11, in <module>

print(manhattan_distance(p1, p2))

^^^^^^^^^^^^^^^^^^^^^^^^^^

File "distance.py", line 6, in manhattan_distance

return abs(point_1.x - point_2.x) + abs(point_1.y - point_2.y)

^^^^^^^^^

AttributeError: 'NoneType' object has no attribute 'x'

.. code-block:: python

Traceback (most recent call last):

File "query.py", line 37, in <module>

magic_arithmetic('foo')

File "query.py", line 18, in magic_arithmetic

return add_counts(x) / 25

^^^^^^^^^^^^^

File "query.py", line 24, in add_counts

return 25 + query_user(user1) + query_user(user2)

^^^^^^^^^^^^^^^^^

File "query.py", line 32, in query_user

return 1 + query_count(db, response['a']['b']['c']['user'], retry=True)

~~~~~~~~~~~~~~~~~~^^^^^

TypeError: 'NoneType' object is not subscriptable

.. code-block:: python

Traceback (most recent call last):

File "calculation.py", line 54, in <module>

result = (x / y / z) * (a / b / c)

~~~~~~^~~

ZeroDivisionError: division by zero

Additionally, the information used by the enhanced traceback feature

is made available via a general API, that can be used to correlate

:term:`bytecode` :ref:`instructions <bytecodes>` with source code location.

This information can be retrieved using:

- The :meth:`codeobject.co_positions` method in Python.

See :pep:`657` for more details. (Contributed by Pablo Galindo, Batuhan Taskaya

and Ammar Askar in :issue:`43950`.)

.. note::

This feature requires storing column positions in :ref:`codeobjects`,

which may result in a small increase in interpreter memory usage

and disk usage for compiled Python files.

To avoid storing the extra information

and deactivate printing the extra traceback information,

use the :option:`-X no_debug_ranges <-X>` command line option

or the :envvar:`PYTHONNODEBUGRANGES` environment variable.

.. _whatsnew311-pep654:

PEP 654: Exception Groups and ``except*``

-----------------------------------------

:pep:`654` introduces language features that enable a program

to raise and handle multiple unrelated exceptions simultaneously.

The builtin types :exc:`ExceptionGroup` and :exc:`BaseExceptionGroup`

make it possible to group exceptions and raise them together,

and the new :keyword:`except* <except_star>` syntax generalizes

:keyword:`except` to match subgroups of exception groups.

See :pep:`654` for more details.

(Contributed by Irit Katriel in :issue:`45292`. PEP written by

Irit Katriel, Yury Selivanov and Guido van Rossum.)

----------------------------------------------

The :meth:`~BaseException.add_note` method is added to :exc:`BaseException`.

It can be used to enrich exceptions with context information

that is not available at the time when the exception is raised.

The added notes appear in the default traceback.

See :pep:`678` for more details.

(Contributed by Irit Katriel in :issue:`45607`.

PEP written by Zac Hatfield-Dodds.)

.. _whatsnew311-windows-launcher:

Windows ``py.exe`` launcher improvements

----------------------------------------

The copy of the :ref:`launcher` included with Python 3.11 has been significantly

updated. It now supports company/tag syntax as defined in :pep:`514` using the

When using ``-V:`` selectors, either company or tag can be omitted, but all

installs will be searched. For example, ``-V:OtherPython/`` will select the

"best" tag registered for ``OtherPython``, while ``-V:3.11`` or ``-V:/3.11``

will select the "best" distribution with tag ``3.11``.

When using the legacy :samp:`-{<major>}`, :samp:`-{<major>}.{<minor>}`,

:samp:`-{<major>}-{<bitness>}` or :samp:`-{<major>}.{<minor>}-{<bitness>}` arguments,

all existing behaviour should be preserved from past versions,

and only releases from ``PythonCore`` will be selected.

However, the ``-64`` suffix now implies "not 32-bit" (not necessarily x86-64),

as there are multiple supported 64-bit platforms.

32-bit runtimes are detected by checking the runtime's tag for a ``-32`` suffix.

All releases of Python since 3.5 have included this in their 32-bit builds.

New Features Related to Type Hints

==================================

This section covers major changes affecting :pep:`484` type hints and

the :mod:`typing` module.

.. _whatsnew311-pep646:

PEP 646: Variadic generics

--------------------------

:pep:`484` previously introduced :data:`~typing.TypeVar`, enabling creation

of generics parameterised with a single type. :pep:`646` adds

:data:`~typing.TypeVarTuple`, enabling parameterisation

with an *arbitrary* number of types. In other words,

a :data:`~typing.TypeVarTuple` is a *variadic* type variable,

enabling *variadic* generics.

This enables a wide variety of use cases.

In particular, it allows the type of array-like structures

in numerical computing libraries such as NumPy and TensorFlow to be

parameterised with the array *shape*. Static type checkers will now

be able to catch shape-related bugs in code that uses these libraries.

See :pep:`646` for more details.

(Contributed by Matthew Rahtz in :issue:`43224`, with contributions by

Serhiy Storchaka and Jelle Zijlstra. PEP written by Mark Mendoza, Matthew

Rahtz, Pradeep Kumar Srinivasan, and Vincent Siles.)

.. _whatsnew311-pep655:

PEP 655: Marking individual ``TypedDict`` items as required or not-required

---------------------------------------------------------------------------

:data:`~typing.Required` and :data:`~typing.NotRequired` provide a

straightforward way to mark whether individual items in a

using inheritance.

All fields are still required by default,

unless the *total* parameter is set to ``False``,

in which case all fields are still not-required by default.

For example, the following specifies a :class:`!TypedDict`

with one required and one not-required key::

class Movie(TypedDict):

title: str

year: NotRequired[int]

m1: Movie = {"title": "Black Panther", "year": 2018} # OK

m2: Movie = {"title": "Star Wars"} # OK (year is not required)

m3: Movie = {"year": 2022} # ERROR (missing required field title)

The following definition is equivalent::

class Movie(TypedDict, total=False):

title: Required[str]

year: int

See :pep:`655` for more details.

(Contributed by David Foster and Jelle Zijlstra in :issue:`47087`. PEP

written by David Foster.)

.. _whatsnew311-pep673:

PEP 673: ``Self`` type

----------------------

The new :data:`~typing.Self` annotation provides a simple and intuitive

way to annotate methods that return an instance of their class. This

behaves the same as the :class:`~typing.TypeVar`-based approach

:pep:`specified in PEP 484 <484#annotating-instance-and-class-methods>`,

but is more concise and easier to follow.

Common use cases include alternative constructors provided as

:func:`classmethod <classmethod>`\s,

and :meth:`~object.__enter__` methods that return ``self``::

class MyLock:

def __enter__(self) -> Self:

self.lock()

return self

...

class MyInt:

@classmethod

def fromhex(cls, s: str) -> Self:

return cls(int(s, 16))

...

:data:`~typing.Self` can also be used to annotate method parameters

or attributes of the same type as their enclosing class.

See :pep:`673` for more details.

(Contributed by James Hilton-Balfe in :issue:`46534`. PEP written by

Pradeep Kumar Srinivasan and James Hilton-Balfe.)

.. _whatsnew311-pep675:

PEP 675: Arbitrary literal string type

--------------------------------------

The new :data:`~typing.LiteralString` annotation may be used to indicate

that a function parameter can be of any literal string type. This allows

a function to accept arbitrary literal string types, as well as strings

created from other literal strings. Type checkers can then

enforce that sensitive functions, such as those that execute SQL

statements or shell commands, are called only with static arguments,

providing protection against injection attacks.

For example, a SQL query function could be annotated as follows::

def run_query(sql: LiteralString) -> ...

...

def caller(

arbitrary_string: str,

query_string: LiteralString,

table_name: LiteralString,

) -> None:

run_query("SELECT * FROM students") # ok

run_query(query_string) # ok

run_query("SELECT * FROM " + table_name) # ok

run_query(arbitrary_string) # type checker error

run_query( # type checker error

f"SELECT * FROM students WHERE name = {arbitrary_string}"

)

See :pep:`675` for more details.

(Contributed by Jelle Zijlstra in :issue:`47088`. PEP written by Pradeep

Kumar Srinivasan and Graham Bleaney.)

.. _whatsnew311-pep681:

------------------------------

:data:`~typing.dataclass_transform` may be used to

decorate a class, metaclass, or a function that is itself a decorator.

decorated object performs runtime "magic" that transforms a class,

giving it :func:`dataclass <dataclasses.dataclass>`-like behaviors.

For example::

cls.__init__ = ...

cls.__eq__ = ...

cls.__ne__ = ...

return cls

@create_model

class CustomerModel:

id: int

name: str

See :pep:`681` for more details.

(Contributed by Jelle Zijlstra in :gh:`91860`. PEP written by

Erik De Bonte and Eric Traut.)

.. _whatsnew311-pep563-deferred:

PEP 563 may not be the future

-----------------------------

:pep:`563` Postponed Evaluation of Annotations

(the ``from __future__ import annotations`` :ref:`future statement <future>`)

that was originally planned for release in Python 3.10

has been put on hold indefinitely.

See `this message from the Steering Council <https://mail.python.org/archives/list/[email protected]/message/VIZEBX5EYMSYIJNDBF6DMUMZOCWHARSO/>`__

for more information.

.. _whatsnew311-other-lang-changes:

Other Language Changes

======================

* Starred unpacking expressions can now be used in :keyword:`for` statements.

(See :issue:`46725` for more details.)

* Asynchronous :ref:`comprehensions <comprehensions>` are now allowed

inside comprehensions in :ref:`asynchronous functions <async def>`.

Outer comprehensions implicitly become asynchronous in this case.

(Contributed by Serhiy Storchaka in :issue:`33346`.)

:keyword:`with` statements and :meth:`contextlib.ExitStack.enter_context`

for objects that do not support the :term:`context manager` protocol,

and in :keyword:`async with` statements and

:meth:`contextlib.AsyncExitStack.enter_async_context`

for objects not supporting the :term:`asynchronous context manager` protocol.

(Contributed by Serhiy Storchaka in :issue:`12022` and :issue:`44471`.)

* Added :meth:`object.__getstate__`, which provides the default

implementation of the :meth:`!__getstate__` method. :mod:`copy`\ing

and :mod:`pickle`\ing instances of subclasses of builtin types

:class:`bytearray`, :class:`set`, :class:`frozenset`,

:class:`collections.OrderedDict`, :class:`collections.deque`,

:class:`weakref.WeakSet`, and :class:`datetime.tzinfo` now copies and

pickles instance attributes implemented as :term:`slots <__slots__>`.

This change has an unintended side effect: It trips up a small minority

of existing Python projects not expecting :meth:`object.__getstate__` to

exist. See the later comments on :gh:`70766` for discussions of what

workarounds such code may need.

(Contributed by Serhiy Storchaka in :issue:`26579`.)

.. _whatsnew311-pythonsafepath:

* Added a :option:`-P` command line option

and a :envvar:`PYTHONSAFEPATH` environment variable,

which disable the automatic prepending to :data:`sys.path`

of the script's directory when running a script,

or the current directory when using :option:`-c` and :option:`-m`.

This ensures only stdlib and installed modules

are picked up by :keyword:`import`,

and avoids unintentionally or maliciously shadowing modules

with those in a local (and typically user-writable) directory.

(Contributed by Victor Stinner in :gh:`57684`.)

* A ``"z"`` option was added to the :ref:`formatspec` that

coerces negative to positive zero after rounding to the format precision.

See :pep:`682` for more details.

(Contributed by John Belmonte in :gh:`90153`.)

* Bytes are no longer accepted on :data:`sys.path`. Support broke sometime

between Python 3.2 and 3.6, with no one noticing until after Python 3.10.0

was released. In addition, bringing back support would be problematic due to

interactions between :option:`-b` and :data:`sys.path_importer_cache` when

there is a mixture of :class:`str` and :class:`bytes` keys.

(Contributed by Thomas Grainger in :gh:`91181`.)

.. _whatsnew311-other-implementation-changes:

Other CPython Implementation Changes

====================================

* The special methods :meth:`~object.__complex__` for :class:`complex`

and :meth:`~object.__bytes__` for :class:`bytes` are implemented to support

the :class:`typing.SupportsComplex` and :class:`typing.SupportsBytes` protocols.

* ``siphash13`` is added as a new internal hashing algorithm.

It has similar security properties as ``siphash24``,

but it is slightly faster for long inputs.

:class:`str`, :class:`bytes`, and some other types

now use it as the default algorithm for :func:`hash`.

:pep:`552` :ref:`hash-based .pyc files <pyc-invalidation>`

now use ``siphash13`` too.

* When an active exception is re-raised by a :keyword:`raise` statement with no parameters,

the traceback attached to this exception is now always ``sys.exc_info()[1].__traceback__``.

This means that changes made to the traceback in the current :keyword:`except` clause are

reflected in the re-raised exception.

(Contributed by Irit Katriel in :issue:`45711`.)

* The interpreter state's representation of handled exceptions

(aka ``exc_info`` or ``_PyErr_StackItem``)

now only has the ``exc_value`` field; ``exc_type`` and ``exc_traceback``

have been removed, as they can be derived from ``exc_value``.

(Contributed by Irit Katriel in :issue:`45711`.)

* A new :ref:`command line option <install-quiet-option>`, ``AppendPath``,

has been added for the Windows installer.

It behaves similarly to ``PrependPath``,

but appends the install and scripts directories instead of prepending them.

(Contributed by Bastian Neuburger in :issue:`44934`.)

initialization to use :c:member:`PyConfig.module_search_paths` to initialize

:data:`sys.path`. Otherwise, initialization will recalculate the path and replace

any values added to ``module_search_paths``.

* The output of the :option:`--help` option now fits in 50 lines/80 columns.

Information about :ref:`Python environment variables <using-on-envvars>`

and :option:`-X` options is now available using the respective

:option:`--help-env` and :option:`--help-xoptions` flags,

and with the new :option:`--help-all`.

(Contributed by Éric Araujo in :issue:`46142`.)

* Converting between :class:`int` and :class:`str` in bases other than 2

(binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal)

now raises a :exc:`ValueError` if the number of digits in string form is

above a limit to avoid potential denial of service attacks due to the

This limit can be configured or disabled by environment variable, command

line flag, or :mod:`sys` APIs. See the :ref:`integer string conversion

length limitation <int_max_str_digits>` documentation. The default limit

is 4300 digits in string form.

.. _whatsnew311-new-modules:

New Modules

===========

* :mod:`tomllib`: For parsing `TOML <https://toml.io/>`_.

See :pep:`680` for more details.

* :mod:`wsgiref.types`:

:pep:`WSGI <3333>`-specific types for static type checking.

(Contributed by Sebastian Rittau in :issue:`42012`.)

.. _whatsnew311-improved-modules:

Improved Modules

================

.. _whatsnew311-asyncio:

asyncio

-------

an :ref:`asynchronous context manager <async-context-managers>`

holding a group of tasks that will wait for all of them upon exit.

For new code this is recommended over using

:func:`~asyncio.create_task` and :func:`~asyncio.gather` directly.

(Contributed by Yury Selivanov and others in :gh:`90908`.)

* Added :func:`~asyncio.timeout`, an asynchronous context manager for

setting a timeout on asynchronous operations. For new code this is

recommended over using :func:`~asyncio.wait_for` directly.

(Contributed by Andrew Svetlov in :gh:`90927`.)

* Added the :class:`~asyncio.Runner` class, which exposes the machinery