Changelog

Python 3.12.4 final

Release date: 2024-06-06

Security

• gh-118486: os.mkdir() on Windows now accepts mode of 0o700 to restrict the new directory to the current user. This fixes CVE-2024-4030 affecting tempfile.mkdtemp() in scenarios where the base temporary directory is more permissive than the default.

• gh-116741: Update bundled libexpat to 2.6.2

• gh-117233: Detect BLAKE2, SHA3, Shake, & truncated SHA512 support in the OpenSSL-ish libcrypto library at build time. This allows hashlib to be used with libraries that do not to support every algorithm that upstream OpenSSL does.

Core and Builtins

• gh-119821: Fix execution of annotation scopes within classes when globals is set to a non-dict. Patch by Jelle Zijlstra.

• gh-118263: Speed up os.path.normpath() with a direct C call.

• gh-119311: Fix bug where names are unexpectedly mangled in the bases of generic classes.

• gh-119395: Fix bug where names appearing after a generic class are mangled as if they are in the generic class.

• gh-118507: Fix os.path.isfile() on Windows for pipes.

• gh-119213: Non-builtin modules built with argument clinic were crashing if used in a subinterpreter before the main interpreter. The objects that were causing the problem by leaking between interpreters carelessly have been fixed.

• gh-119011: Fixes type.__type_params__ to return an empty tuple instead of a descriptor.

• gh-118997: Fix _Py_ClearImmortal() assertion: use _Py_IsImmortal() to tolerate reference count lower than _Py_IMMORTAL_REFCNT. Fix the assertion for the stable ABI, when a C extension is built with Python 3.11 or lower. Patch by Victor Stinner.

• gh-118513: Fix incorrect UnboundLocalError when two comprehensions in the same function both reference the same name, and in one comprehension the name is bound while in the other it’s an implicit global.

• gh-118164: Break a loop between the Python implementation of the decimal module and the Python code for integer to string conversion. Also optimize integer to string conversion for values in the range from 9_000 to 135_000 decimal digits.

• gh-118272: Fix bug where generator.close does not free the generator frame’s locals.

• gh-116767: Fix crash in compiler on ‘async with’ that has many context managers.

• gh-117894: Prevent agen.aclose() objects being re-used after .throw().

• gh-117881: prevent concurrent access to an async generator via athrow().throw() or asend().throw()

• gh-115874: Fixed a possible segfault during garbage collection of _asyncio.FutureIter objects

Library

• gh-119819: Fix regression to allow logging configuration with multiprocessing queue types.

• gh-89727: Fix issue with shutil.rmtree() where a RecursionError is raised on deep directory trees.

• gh-89727: Partially fix issue with shutil.rmtree() where a RecursionError is raised on deep directory trees. A recursion error is no longer raised when rmtree.avoids_symlink_attacks is false.

• gh-119118: Fix performance regression in the tokenize module by caching the line token attribute and calculating the column offset more efficiently.

• gh-89727: Fix issue with os.fwalk() where a RecursionError was raised on deep directory trees by adjusting the implementation to be iterative instead of recursive.

• gh-113892: Now, the method sock_connect of asyncio.ProactorEventLoop raises a ValueError if given socket is not in non-blocking mode, as well as in other loop implementations.

• gh-119174: Fix high DPI causes turtledemo(turtle-graphics examples) windows blurry Patch by Wulian233 and Terry Jan Reedy

• gh-118643: Fix an AttributeError in the email module when re-fold a long address list. Also fix more cases of incorrect encoding of the address separator in the address list.

• gh-58933: Make pdb return to caller frame correctly when f_trace of the caller frame is not set

• gh-118868: Fixed issue where kwargs were no longer passed to the logging handler QueueHandler

• gh-118164: The Python implementation of the decimal module could appear to hang in relatively small power cases (like 2**117) if context precision was set to a very high value. A different method to check for exactly representable results is used now that doesn’t rely on computing 10**precision (which could be effectively too large to compute).

• gh-118404: Fix inspect.signature() for non-comparable callables.

• gh-118314: Fix an edge case in binascii.a2b_base64() strict mode, where excessive padding is not detected when no padding is necessary.

• gh-118042: Fix an unraisable exception in telnetlib.Telnet.__del__() when the __init__() method was not called.

• gh-118221: Fix a bug where sqlite3.iterdump() could fail if a custom row factory was used. Patch by Erlend Aasland.

• gh-118013: Fix regression introduced in gh-103193 that meant that calling inspect.getattr_static() on an instance would cause a strong reference to that instance’s class to persist in an internal cache in the inspect module. This caused unexpected memory consumption if the class was dynamically created, the class held strong references to other objects which took up a significant amount of memory, and the cache contained the sole strong reference to the class. The fix for the regression leads to a slowdown in getattr_static(), but the function should still be significantly faster than it was in Python 3.11. Patch by Alex Waygood.

• gh-90848: Fixed unittest.mock.create_autospec() to configure parent mock with keyword arguments.

• gh-118168: Fix incorrect argument substitution when typing.Unpack is used with the builtin tuple. typing.Unpack now raises TypeError when used with certain invalid types. Patch by Jelle Zijlstra.

• gh-118033: Fix dataclasses.dataclass() not creating a __weakref__ slot when subclassing typing.Generic.

• gh-117535: Do not try to get the source line for made up file name “sys” in warnings.

• gh-114053: Fix erroneous NameError when calling typing.get_type_hints() on a class that made use of PEP 695 type parameters in a module that had from __future__ import annotations at the top of the file. Patch by Alex Waygood.

• gh-117995: Don’t raise DeprecationWarning when a sequence of parameters is used to bind indexed, nameless placeholders. See also gh-100668.

• gh-80361: Fix TypeError in email.message.Message.get_payload() when the charset is RFC 2231 encoded.

• gh-86650: Fix IndexError when parse some emails with invalid Message-ID (including one-off addresses generated by Microsoft Outlook).

• gh-117691: Improve the error messages emitted by tarfile deprecation warnings relating to PEP 706. If a filter argument is not provided to extract() or extractall, the deprecation warning now points to the line in the user’s code where the relevant function was called. Patch by Alex Waygood.

• gh-77102: site module now parses .pth file with UTF-8 first, and locale encoding if UnicodeDecodeError happened. It supported only locale encoding before.

• gh-117692: Fixes a bug when doctest.DocTestFinder was failing on wrapped builtin_function_or_method.

• gh-117566: ipaddress.IPv6Address.is_loopback() will now return True for IPv4-mapped loopback addresses, i.e. addresses in the ::ffff:127.0.0.0/104 address space.

• gh-117503: Fix support of non-ASCII user names in bytes paths in os.path.expanduser() on Posix.

• gh-117313: Only treat '\n', '\r' and '\r\n' as line separators in re-folding the email messages. Preserve control characters '\v', '\f', '\x1c', '\x1d' and '\x1e' and Unicode line separators '\x85', '\u2028' and '\u2029' as is.

• gh-113171: Fixed various false positives and false negatives in

  • ipaddress.IPv4Address.is_private (see these docs for details)

  • ipaddress.IPv4Address.is_global

  • ipaddress.IPv6Address.is_private

  • ipaddress.IPv6Address.is_global

  Also in the corresponding ipaddress.IPv4Network and ipaddress.IPv6Network attributes.

• gh-103956: Fix lack of newline characters in trace module output when line tracing is enabled but source code line for current frame is not available.

• gh-92081: Fix missing spaces in email headers when the spaces are mixed with encoded 8-bit characters.

• gh-103194: Prepare Tkinter for C API changes in Tcl 8.7/9.0 to avoid _tkinter.Tcl_Obj being unexpectedly returned instead of bool, str, bytearray, or int.

• gh-87106: Fixed handling in inspect.Signature.bind() of keyword arguments having the same name as positional-only arguments when a variadic keyword argument (e.g. **kwargs) is present.

• bpo-45767: Fix integer conversion in os.major(), os.minor(), and os.makedev(). Support device numbers larger than 2**63-1. Support non-existent device number (NODEV).

• bpo-40943: Fix several IndexError when parse emails with truncated Message-ID, address, routes, etc, e.g. example@.

• bpo-30988: Fix parsing of emails with invalid address headers having a leading or trailing dot. Patch by tsufeki.

• gh-67693: Fix urllib.parse.urlunparse() and urllib.parse.urlunsplit() for URIs with path starting with multiple slashes and no authority. Based on patch by Ashwin Ramaswami.

• bpo-15010: unittest.TestLoader.discover() now saves the original value of unittest.TestLoader._top_level_dir and restores it at the end of the call.

Documentation

• gh-117928: The minimum Sphinx version required for the documentation is now 6.2.1.

• gh-91565: Changes to documentation files and config outputs to reflect the new location for reporting bugs - i.e. GitHub rather than bugs.python.org.

Tests

• gh-119050: regrtest test runner: Add XML support to the refleak checker (-R option). Patch by Victor Stinner.

Windows

• gh-119690: Adds Unicode support and fixes audit events for _winapi.CreateNamedPipe.

• gh-119070: Fixes py.exe handling of shebangs like /usr/bin/env python3.12, which were previously interpreted as python3.exe instead of python3.12.exe.

• gh-118347: Fixes launcher updates not being installed.

• gh-115009: Update Windows installer to use SQLite 3.45.3.

• gh-90329: Suppress the warning displayed on virtual environment creation when the requested and created paths differ only by a short (8.3 style) name. Warnings will continue to be shown if a junction or symlink in the path caused the venv to be created in a different location than originally requested.

macOS

• gh-115009: Update macOS installer to use SQLite 3.45.3.

• gh-116145: Update macOS installer to Tcl/Tk 8.6.14.

IDLE

• bpo-34774: Use user-selected color theme for Help => IDLE Doc.

C API

• gh-119585: Fix crash when a thread state that was created by PyGILState_Ensure() calls a destructor that during PyThreadState_Clear() that calls back into PyGILState_Ensure() and PyGILState_Release(). This might occur when in the free-threaded build or when using thread-local variables whose destructors call PyGILState_Ensure().

• gh-117534: Improve validation logic in the C implementation of datetime.fromisoformat() to better handle invalid years. Patch by Vlad Efanov.

Python 3.12.3 final

Release date: 2024-04-09

Security

• gh-115398: Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:

  • xml.etree.ElementTree.XMLParser.flush()

  • xml.etree.ElementTree.XMLPullParser.flush()

  • xml.parsers.expat.xmlparser.GetReparseDeferralEnabled()

  • xml.parsers.expat.xmlparser.SetReparseDeferralEnabled()

  • xml.sax.expatreader.ExpatParser.flush()

• gh-115399: Update bundled libexpat to 2.6.0

• gh-115243: Fix possible crashes in collections.deque.index() when the deque is concurrently modified.

• gh-114572: ssl.SSLContext.cert_store_stats() and ssl.SSLContext.get_ca_certs() now correctly lock access to the certificate store, when the ssl.SSLContext is shared across multiple threads.

Core and Builtins

• gh-109120: Added handle of incorrect star expressions, e.g f(3, *). Patch by Grigoryev Semyon

• gh-99108: Updated the hashlib built-in HACL* project C code from upstream that we use for many implementations when they are not present via OpenSSL in a given build. This also avoids the rare potential for a C symbol name one definition rule linking issue.

• gh-116735: For INSTRUMENTED_CALL_FUNCTION_EX, set arg0 to sys.monitoring.MISSING instead of None for CALL event.

• gh-113964: Starting new threads and process creation through os.fork() are now only prevented once all non-daemon threads exit.

• gh-116604: Respect the status of the garbage collector when indirect calls are made via PyErr_CheckSignals() and the evaluation breaker. Patch by Pablo Galindo

• gh-116626: Ensure INSTRUMENTED_CALL_FUNCTION_EX always emits CALL

• gh-116296: Fix possible refleak in object.__reduce__() internal error handling.

• gh-116034: Fix location of the error on a failed assertion.

• gh-115823: Properly calculate error ranges in the parser when raising SyntaxError exceptions caused by invalid byte sequences. Patch by Pablo Galindo

• gh-112087: For an empty reverse iterator for list will be reduced to reversed(). Patch by Donghee Na.

• gh-115154: Fix a bug that was causing the tokenize.untokenize() function to handle unicode named literals incorrectly. Patch by Pablo Galindo

• gh-114828: Fix compilation crashes in uncommon code examples using super() inside a comprehension in a class body.

• gh-115011: Setters for members with an unsigned integer type now support the same range of valid values for objects that has a __index__() method as for int.

• gh-112215: Change the C recursion limits to more closely reflect the underlying platform limits.

• gh-96497: Fix incorrect resolution of mangled class variables used in assignment expressions in comprehensions.

Library

• gh-117467: Preserve mailbox ownership when rewriting in mailbox.mbox.flush(). Patch by Tony Mountifield.

• gh-117310: Fixed an unlikely early & extra Py_DECREF triggered crash in ssl when creating a new _ssl._SSLContext if CPython was built implausibly such that the default cipher list is empty or the SSL library it was linked against reports a failure from its C SSL_CTX_set_cipher_list() API.

• gh-117178: Fix regression in lazy loading of self-referential modules, introduced in gh-114781.

• gh-117084: Fix zipfile extraction for directory entries with the name containing backslashes on Windows.

• gh-117110: Fix a bug that prevents subclasses of typing.Any to be instantiated with arguments. Patch by Chris Fu.

• gh-90872: On Windows, subprocess.Popen.wait() no longer calls WaitForSingleObject() with a negative timeout: pass 0 ms if the timeout is negative. Patch by Victor Stinner.

• gh-116957: configparser: Don’t leave ConfigParser values in an invalid state (stored as a list instead of a str) after an earlier read raised DuplicateSectionError or DuplicateOptionError.

• gh-90095: Ignore empty lines and comments in .pdbrc

• gh-116764: Restore support of None and other false values in urllib.parse functions parse_qs() and parse_qsl(). Also, they now raise a TypeError for non-zero integers and non-empty sequences.

• gh-116811: In PathFinder.invalidate_caches, delegate to MetadataPathFinder.invalidate_caches.

• gh-116600: Fix repr() for global Flag members.

• gh-116484: Change automatically generated tkinter.Checkbutton widget names to avoid collisions with automatically generated tkinter.ttk.Checkbutton widget names within the same parent widget.

• gh-116401: Fix blocking os.fwalk() and shutil.rmtree() on opening named pipe.

• gh-116143: Fix a race in pydoc _start_server, eliminating a window in which _start_server can return a thread that is “serving” but without a docserver set.

• gh-116325: typing: raise SyntaxError instead of AttributeError on forward references as empty strings.

• gh-90535: Fix support of interval values > 1 in logging.TimedRotatingFileHandler for when='MIDNIGHT' and when='Wx'.

• gh-115978: Disable preadv(), readv(), pwritev(), and writev() on WASI.

  Under wasmtime for WASI 0.2, these functions don’t pass test_posix (https://github.com/bytecodealliance/wasmtime/issues/7830).

• gh-88352: Fix the computation of the next rollover time in the logging.TimedRotatingFileHandler handler. computeRollover() now always returns a timestamp larger than the specified time and works correctly during the DST change. doRollover() no longer overwrite the already rolled over file, saving from data loss when run at midnight or during repeated time at the DST change.

• gh-87115: Set __main__.__spec__ to None when running a script with pdb

• gh-76511: Fix UnicodeEncodeError in email.Message.as_string() that results when a message that claims to be in the ascii character set actually has non-ascii characters. Non-ascii characters are now replaced with the U+FFFD replacement character, like in the replace error handler.

• gh-116040: [Enum] fix by-value calls when second value is falsey; e.g. Cardinal(1, 0)

• gh-75988: Fixed unittest.mock.create_autospec() to pass the call through to the wrapped object to return the real result.

• gh-115881: Fix issue where ast.parse() would incorrectly flag conditional context managers (such as with (x() if y else z()): ...) as invalid syntax if feature_version=(3, 8) was passed. This reverts changes to the grammar made as part of gh-94949.

• gh-115886: Fix silent truncation of the name with an embedded null character in multiprocessing.shared_memory.SharedMemory.

• gh-115809: Improve algorithm for computing which rolled-over log files to delete in logging.TimedRotatingFileHandler. It is now reliable for handlers without namer and with arbitrary deterministic namer that leaves the datetime part in the file name unmodified.

• gh-74668: urllib.parse functions parse_qs() and parse_qsl() now support bytes arguments containing raw and percent-encoded non-ASCII data.

• gh-67044: csv.writer() now always quotes or escapes '\r' and '\n', regardless of lineterminator value.

• gh-115712: csv.writer() now quotes empty fields if delimiter is a space and skipinitialspace is true and raises exception if quoting is not possible.

• gh-112364: Fixed ast.unparse() to handle format_spec with ", ' or \\. Patched by Frank Hoffmann.

• gh-111358: Fix a bug in asyncio.BaseEventLoop.shutdown_default_executor() to ensure the timeout passed to the coroutine behaves as expected.

• gh-115618: Fix improper decreasing the reference count for None argument in property methods getter(), setter() and deleter().

• gh-115570: A DeprecationWarning is no longer omitted on access to the __doc__ attributes of the deprecated typing.io and typing.re pseudo-modules.

• gh-112006: Fix inspect.unwrap() for types with the __wrapper__ data descriptor.

• gh-101293: Support callables with the __call__() method and types with __new__() and __init__() methods set to class methods, static methods, bound methods, partial functions, and other types of methods and descriptors in inspect.Signature.from_callable().

• gh-115392: Fix a bug in doctest where incorrect line numbers would be reported for decorated functions.

• gh-114563: Fix several format() bugs when using the C implementation of Decimal: * memory leak in some rare cases when using the z format option (coerce negative 0) * incorrect output when applying the z format option to type F (fixed-point with capital NAN / INF) * incorrect output when applying the # format option (alternate form)

• gh-115197: urllib.request no longer resolves the hostname before checking it against the system’s proxy bypass list on macOS and Windows.

• gh-115165: Most exceptions are now ignored when attempting to set the __orig_class__ attribute on objects returned when calling typing generic aliases (including generic aliases created using typing.Annotated). Previously only AttributeError was ignored. Patch by Dave Shawley.

• gh-115133: Fix tests for XMLPullParser with Expat 2.6.0.

• gh-115059: io.BufferedRandom.read1() now flushes the underlying write buffer.

• gh-79382: Trailing ** no longer allows to match files and non-existing paths in recursive glob().

• gh-114071: Support tuple subclasses using auto() for enum member value.

• gh-114763: Protect modules loaded with importlib.util.LazyLoader from race conditions when multiple threads try to access attributes before the loading is complete.

• gh-97959: Fix rendering class methods, bound methods, method and function aliases in pydoc. Class methods no longer have “method of builtins.type instance” note. Corresponding notes are now added for class and unbound methods. Method and function aliases now have references to the module or the class where the origin was defined if it differs from the current. Bound methods are now listed in the static methods section. Methods of builtin classes are now supported as well as methods of Python classes.

• gh-112281: Allow creating union of types for typing.Annotated with unhashable metadata.

• gh-111775: Fix importlib.resources.simple.ResourceHandle.open() for text mode, added missed stream argument.

• gh-90095: Make .pdbrc and -c work with any valid pdb commands.

• gh-107155: Fix incorrect output of help(x) where x is a lambda function, which has an __annotations__ dictionary attribute with a "return" key.

• gh-105866: Fixed _get_slots bug which caused error when defining dataclasses with slots and a weakref_slot.

• gh-60346: Fix ArgumentParser inconsistent with parse_known_args.

• gh-100985: Update HTTPSConnection to consistently wrap IPv6 Addresses when using a proxy.

• gh-100884: email: fix misfolding of comma in address-lists over multiple lines in combination with unicode encoding.

• gh-95782: Fix io.BufferedReader.tell(), io.BufferedReader.seek(), _pyio.BufferedReader.tell(), io.BufferedRandom.tell(), io.BufferedRandom.seek() and _pyio.BufferedRandom.tell() being able to return negative offsets.

• gh-96310: Fix a traceback in argparse when all options in a mutually exclusive group are suppressed.

• gh-93205: Fixed a bug in logging.handlers.TimedRotatingFileHandler where multiple rotating handler instances pointing to files with the same name but different extensions would conflict and not delete the correct files.

• bpo-44865: Add missing call to localization function in argparse.

• bpo-43952: Fix multiprocessing.connection.Listener.accept() to accept empty bytes as authkey. Not accepting empty bytes as key causes it to hang indefinitely.

• bpo-42125: linecache: get module name from __spec__ if available. This allows getting source code for the __main__ module when a custom loader is used.

• gh-66543: Make mimetypes.guess_type() properly parsing of URLs with only a host name, URLs containing fragment or query, and filenames with only a UNC sharepoint on Windows. Based on patch by Dong-hee Na.

• bpo-33775: Add ‘default’ and ‘version’ help text for localization in argparse.

Documentation

• gh-115399: Document CVE-2023-52425 of Expat <2.6.0 under “XML vulnerabilities”.

• gh-115233: Fix an example for LoggerAdapter in the Logging Cookbook.

Tests

• gh-83434: Disable JUnit XML output (--junit-xml=FILE command line option) in regrtest when hunting for reference leaks (-R option). Patch by Victor Stinner.

• gh-117187: Fix XML tests for vanilla Expat <2.6.0.

• gh-116333: Tests of TLS related things (error codes, etc) were updated to be more lenient about specific error message strings and behaviors as seen in the BoringSSL and AWS-LC forks of OpenSSL.

• gh-115979: Update test_importlib so that it passes under WASI SDK 21.

• gh-112536: Add –tsan to test.regrtest for running TSAN tests in reasonable execution times. Patch by Donghee Na.

• gh-116307: Added import helper isolated_modules as CleanImport does not remove modules imported during the context. Use it in importlib.resources tests to avoid leaving mod around to impede importlib.metadata tests.

• gh-115720: Leak tests (-R, --huntrleaks) now show a summary of the number of leaks found in each iteration.

• gh-115122: Add --bisect option to regrtest test runner: run failed tests with test.bisect_cmd to identify failing tests. Patch by Victor Stinner.

• gh-115596: Fix ProgramPriorityTests in test_os permanently changing the process priority.

Build

• gh-116313: Get WASI builds to work under wasmtime 18 w/ WASI 0.2/preview2 primitives.

• gh-116117: Backport libb2’s PR #42 to fix compiling CPython on 32-bit Windows with clang-cl.

• gh-115167: Avoid vendoring vcruntime140_threads.dll when building with Visual Studio 2022 version 17.8.

• gh-112536: Add support for thread sanitizer (TSAN)

Windows

• gh-117267: Ensure DirEntry.stat().st_ctime behaves consistently with os.stat() during the deprecation period of st_ctime by containing the same value as st_birthtime. After the deprecation period, st_ctime will be the metadata change time (or unavailable through DirEntry), and only st_birthtime will contain the creation time.

• gh-116773: Fix instances of <_overlapped.Overlapped object at 0xXXX> still has pending operation at deallocation, the process may crash.

• gh-91227: Fix the asyncio ProactorEventLoop implementation so that sending a datagram to an address that is not listening does not prevent receiving any more datagrams.

• gh-115554: The installer now has more strict rules about updating the Python Launcher for Windows. In general, most users only have a single launcher installed and will see no difference. When multiple launchers have been installed, the option to install the launcher is disabled until all but one have been removed. Downgrading the launcher (which was never allowed) is now more obviously blocked.

• gh-115543: Python Launcher for Windows can now detect Python 3.13 when installed from the Microsoft Store, and will install Python 3.12 by default when PYLAUNCHER_ALLOW_INSTALL is set.

• gh-115049: Fixes py.exe launcher failing when run as users without user profiles.

• gh-115009: Update Windows installer to use SQLite 3.45.1.

IDLE

• gh-88516: On macOS show a proxy icon in the title bar of editor windows to match platform behaviour.

Tools/Demos

• gh-113516: Don’t set LDSHARED when building for WASI.

C API

• gh-117021: Fix integer overflow in PyLong_AsPid() on non-Windows 64-bit platforms.

• gh-116869: Make the C API compatible with -Werror=declaration-after-statement compiler flag again. Patch by Victor Stinner.

Python 3.12.2 final

Release date: 2024-02-06

Security

• gh-113659: Skip .pth files with names starting with a dot or hidden file attribute.

Core and Builtins

• gh-114887: Changed socket type validation in create_datagram_endpoint() to accept all non-stream sockets. This fixes a regression in compatibility with raw sockets.

• gh-114388: Fix a RuntimeWarning emitted when assign an integer-like value that is not an instance of int to an attribute that corresponds to a C struct member of type T_UINT and T_ULONG. Fix a double RuntimeWarning emitted when assign a negative integer value to an attribute that corresponds to a C struct member of type T_UINT.

• gh-113703: Fix a regression in the codeop module that was causing it to incorrectly identify incomplete f-strings. Patch by Pablo Galindo

• gh-89811: Check for a valid tp_version_tag before performing bytecode specializations that rely on this value being usable.

• gh-113602: Fix an error that was causing the parser to try to overwrite existing errors and crashing in the process. Patch by Pablo Galindo

• gh-113297: Fix segfault in the compiler on with statement with 19 context managers.

• gh-106905: Use per AST-parser state rather than global state to track recursion depth within the AST parser to prevent potential race condition due to simultaneous parsing.

  The issue primarily showed up in 3.11 by multithreaded users of ast.parse(). In 3.12 a change to when garbage collection can be triggered prevented the race condition from occurring.

• gh-112943: Correctly compute end column offsets for multiline tokens in the tokenize module. Patch by Pablo Galindo

• gh-112716: Fix SystemError in the import statement and in __reduce__() methods of builtin types when __builtins__ is not a dict.

• gh-94606: Fix UnicodeEncodeError when email.message.get_payload() reads a message with a Unicode surrogate character and the message content is not well-formed for surrogateescape encoding. Patch by Sidney Markowitz.

Library

• gh-114965: Update bundled pip to 24.0

• gh-114959: tarfile no longer ignores errors when trying to extract a directory on top of a file.

• gh-109475: Fix support of explicit option value “–” in argparse (e.g. --option=--).

• gh-110190: Fix ctypes structs with array on Windows ARM64 platform by setting MAX_STRUCT_SIZE to 32 in stgdict. Patch by Diego Russo

• gh-113280: Fix a leak of open socket in rare cases when error occurred in ssl.SSLSocket creation.

• gh-77749: email.policy.EmailPolicy.fold() now always encodes non-ASCII characters in headers if utf8 is false.

• gh-114492: Make the result of termios.tcgetattr() reproducible on Alpine Linux. Previously it could leave a random garbage in some fields.

• gh-113267: Revert changes in gh-106584 which made calls of TestResult methods startTest() and stopTest() unbalanced.

• gh-75128: Ignore an OSError in asyncio.BaseEventLoop.create_server() when IPv6 is available but the interface cannot actually support it.

• gh-114257: Dismiss the FileNotFound error in ctypes.util.find_library() and just return None on Linux.

• gh-114328: The tty.setcbreak() and new tty.cfmakecbreak() no longer clears the terminal input ICRLF flag. This fixes a regression introduced in 3.12 that no longer matched how OSes define cbreak mode in their stty(1) manual pages.

• gh-101438: Avoid reference cycle in ElementTree.iterparse. The iterator returned by ElementTree.iterparse may hold on to a file descriptor. The reference cycle prevented prompt clean-up of the file descriptor if the returned iterator was not exhausted.

• gh-104522: OSError raised when run a subprocess now only has filename attribute set to cwd if the error was caused by a failed attempt to change the current directory.

• gh-114149: Enum: correctly handle tuple subclasses in custom __new__.

• gh-109534: Fix a reference leak in asyncio.selector_events.BaseSelectorEventLoop when SSL handshakes fail. Patch contributed by Jamie Phan.

• gh-114077: Fix possible OverflowError in socket.socket.sendfile() when pass count larger than 2 GiB on 32-bit platform.

• gh-114014: Fixed a bug in fractions.Fraction where an invalid string using d in the decimals part creates a different error compared to other invalid letters/characters. Patch by Jeremiah Gabriel Pascual.

• gh-113951: Fix the behavior of tag_unbind() methods of tkinter.Text and tkinter.Canvas classes with three arguments. Previously, widget.tag_unbind(tag, sequence, funcid) destroyed the current binding for sequence, leaving sequence unbound, and deleted the funcid command. Now it removes only funcid from the binding for sequence, keeping other commands, and deletes the funcid command. It leaves sequence unbound only if funcid was the last bound command.

• gh-113877: Fix tkinter method winfo_pathname() on 64-bit Windows.

• gh-113661: unittest runner: Don’t exit 5 if tests were skipped. The intention of exiting 5 was to detect issues where the test suite wasn’t discovered at all. If we skipped tests, it was correctly discovered.

• gh-113781: Silence unraisable AttributeError when warnings are emitted during Python finalization.

• gh-112932: Restore the ability for zipfile to extractall from zip files with a “/” directory entry in them as is commonly added to zips by some wiki or bug tracker data exporters.

• gh-113594: Fix UnicodeEncodeError in email when re-fold lines that contain unknown-8bit encoded part followed by non-unknown-8bit encoded part.

• gh-113538: In asyncio.StreamReaderProtocol.connection_made(), there is callback that logs an error if the task wrapping the “connected callback” fails. This callback would itself fail if the task was cancelled. Prevent this by checking whether the task was cancelled first. If so, close the transport but don’t log an error.

• gh-85567: Fix resource warnings for unclosed files in pickle and pickletools command line interfaces.

• gh-101225: Increase the backlog for multiprocessing.connection.Listener objects created by multiprocessing.manager and multiprocessing.resource_sharer to significantly reduce the risk of getting a connection refused error when creating a multiprocessing.connection.Connection to them.

• gh-113543: Make sure that webbrowser.MacOSXOSAScript sends webbrowser.open audit event.

• gh-113028: When a second reference to a string appears in the input to pickle, and the Python implementation is in use, we are guaranteed that a single copy gets pickled and a single object is shared when reloaded. Previously, in protocol 0, when a string contained certain characters (e.g. newline) it resulted in duplicate objects.

• gh-113421: Fix multiprocessing logger for %(filename)s.

• gh-111784: Fix segfaults in the _elementtree module. Fix first segfault during deallocation of _elementtree.XMLParser instances by keeping strong reference to pyexpat module in module state for capsule lifetime. Fix second segfault which happens in the same deallocation process by keeping strong reference to _elementtree module in XMLParser structure for _elementtree module lifetime.

• gh-113407: Fix import of unittest.mock when CPython is built without docstrings.

• gh-113320: Fix regression in Python 3.12 where Protocol classes that were not marked as runtime-checkable would be unnecessarily introspected, potentially causing exceptions to be raised if the protocol had problematic members. Patch by Alex Waygood.

• gh-113358: Fix rendering tracebacks for exceptions with a broken __getattr__.

• gh-113214: Fix an AttributeError during asyncio SSL protocol aborts in SSL-over-SSL scenarios.

• gh-113246: Update bundled pip to 23.3.2.

• gh-113199: Make http.client.HTTPResponse.read1 and http.client.HTTPResponse.readline close IO after reading all data when content length is known. Patch by Illia Volochii.

• gh-113188: Fix shutil.copymode() and shutil.copystat() on Windows. Previously they worked differenly if dst is a symbolic link: they modified the permission bits of dst itself rather than the file it points to if follow_symlinks is true or src is not a symbolic link, and did not modify the permission bits if follow_symlinks is false and src is a symbolic link.

• gh-61648: Detect line numbers of properties in doctests.

• gh-112559: signal.signal() and signal.getsignal() no longer call repr on callable handlers. asyncio.run() and asyncio.Runner.run() no longer call repr on the task results. Patch by Yilei Yang.

• gh-110190: Fix ctypes structs with array on PPC64LE platform by setting MAX_STRUCT_SIZE to 64 in stgdict. Patch by Diego Russo.

• gh-79429: Ignore FileNotFoundError when remove a temporary directory in the multiprocessing finalizer.

• gh-81194: Fix a crash in socket.if_indextoname() with specific value (UINT_MAX). Fix an integer overflow in socket.if_indextoname() on 64-bit non-Windows platforms.

• gh-112343: Improve handling of pdb convenience variables to avoid replacing string contents.

• gh-111615: Fix a regression caused by a fix to gh-93162 whereby you couldn’t configure a QueueHandler without specifying handlers.

• gh-111049: Fix crash during garbage collection of the io.BytesIO buffer object.

• gh-110345: Show the Tcl/Tk patchlevel (rather than version) in tkinter._test().

• gh-109858: Protect zipfile from “quoted-overlap” zipbomb. It now raises BadZipFile when try to read an entry that overlaps with other entry or central directory.

• gh-114440: On Windows, closing the connection writer when cleaning up a broken multiprocessing.Queue queue is now done for all queues, rather than only in concurrent.futures manager thread. This can prevent a deadlock when a multiprocessing worker process terminates without cleaning up. This completes the backport of patches by Victor Stinner and Serhiy Storchaka.

• gh-38807: Fix race condition in trace. Instead of checking if a directory exists and creating it, directly call os.makedirs() with the kwarg exist_ok=True.

• gh-75705: Set unixfrom envelope in mailbox.mbox and mailbox.MMDF.

• gh-106233: Fix stacklevel in InvalidTZPathWarning during zoneinfo module import.

• gh-105102: Allow ctypes.Union to be nested in ctypes.Structure when the system endianness is the opposite of the classes.

• gh-104282: Fix null pointer dereference in lzma._decode_filter_properties() due to improper handling of BCJ filters with properties of zero length. Patch by Radislav Chugunov.

• gh-102512: When os.fork() is called from a foreign thread (aka _DummyThread), the type of the thread in a child process is changed to _MainThread. Also changed its name and daemonic status, it can be now joined.

• bpo-35928: io.TextIOWrapper now correctly handles the decoding buffer after read() and write().

• bpo-26791: shutil.move() now moves a symlink into a directory when that directory is the target of the symlink. This provides the same behavior as the mv shell command. The previous behavior raised an exception. Patch by Jeffrey Kintscher.

• bpo-36959: Fix some error messages for invalid ISO format string combinations in strptime() that referred to directives not contained in the format string. Patch by Gordon P. Hemsley.

• bpo-18060: Fixed a class inheritance issue that can cause segfaults when deriving two or more levels of subclasses from a base class of Structure or Union.

Documentation

• gh-110746: Improved markup for valid options/values for methods ttk.treeview.column and ttk.treeview.heading, and for Layouts.

• gh-95649: Document that the asyncio module contains code taken from v0.16.0 of the uvloop project, as well as the required MIT licensing information.

Tests

• gh-109980: Fix test_tarfile_vs_tar in test_shutil for macOS, where system tar can include more information in the archive than shutil.make_archive.

• gh-105089: Fix test.test_zipfile.test_core.TestWithDirectory.test_create_directory_with_write test in AIX by doing a bitwise AND of 0xFFFF on mode , so that it will be in sync with zinfo.external_attr

• bpo-40648: Test modes that file can get with chmod() on Windows.

Build

• gh-112305: Fixed the check-clean-src step performed on out of tree builds to detect errant $(srcdir)/Python/frozen_modules/*.h files and recommend appropriate source tree cleanup steps to get a working build again.

• gh-112867: Fix the build for the case that WITH_PYMALLOC_RADIX_TREE=0 set.

• bpo-11102: The os.major(), os.makedev(), and os.minor() functions are now available on HP-UX v3.

• bpo-36351: Do not set ipv6type when cross-compiling.

Windows

• gh-109991: Update Windows build to use OpenSSL 3.0.13.

• gh-111239: Update Windows builds to use zlib v1.3.1.

• gh-100107: The py.exe launcher will no longer attempt to run the Microsoft Store redirector when launching a script containing a /usr/bin/env shebang

• gh-114096: Process privileges that are activated for creating directory junctions are now restored afterwards, avoiding behaviour changes in other parts of the program.

• gh-111877: os.stat() calls were returning incorrect time values for files that could not be accessed directly.

• gh-113009: multiprocessing: On Windows, fix a race condition in Process.terminate(): no longer set the returncode attribute to always call WaitForSingleObject() in Process.wait(). Previously, sometimes the process was still running after TerminateProcess() even if GetExitCodeProcess() is not STILL_ACTIVE. Patch by Victor Stinner.

• gh-87868: Correctly sort and remove duplicate environment variables in _winapi.CreateProcess().

• bpo-37308: Fix mojibake in mmap.mmap when using a non-ASCII tagname argument on Windows.

macOS

• gh-115009: Update macOS installer to use SQLite 3.45.1.

• gh-109991: Update macOS installer to use OpenSSL 3.0.13.

• gh-110459: Running configure ... --with-openssl-rpath=X/Y/Z no longer fails to detect OpenSSL on macOS.

• gh-74573: Document that dbm.ndbm can silently corrupt DBM files on updates when exceeding undocumented platform limits, and can crash (segmentation fault) when reading such a corrupted file. (FB8919203)

• gh-65701: The freeze tool doesn’t work with framework builds of Python. Document this and bail out early when running the tool with such a build.

• gh-108269: Set CFBundleAllowMixedLocalizations to true in the Info.plist for the framework, embedded Python.app and IDLE.app with framework installs on macOS. This allows applications to pick up the user’s preferred locale when that’s different from english.

• gh-102362: Make sure the result of sysconfig.get_plaform() includes at least a major and minor versions, even if MACOSX_DEPLOYMENT_TARGET is set to only a major version during build to match the format expected by pip.

• gh-110017: Disable a signal handling stress test on macOS due to a bug in macOS (FB13453490).

• gh-110820: Make sure the preprocessor definitions for ALIGNOF_MAX_ALIGN_T, SIZEOF_LONG_DOUBLE and HAVE_GCC_ASM_FOR_X64 are correct for Universal 2 builds on macOS.

IDLE

• gh-96905: In idlelib code, stop redefining built-ins ‘dict’ and ‘object’.

• gh-72284: Improve the lists of features, editor key bindings, and shell key bingings in the IDLE doc.

• gh-113903: Fix rare failure of test.test_idle, in test_configdialog.

• gh-113729: Fix the “Help -> IDLE Doc” menu bug in 3.11.7 and 3.12.1.

• gh-113269: Fix test_editor hang on macOS Catalina.

• gh-112898: Fix processing unsaved files when quitting IDLE on macOS.

• gh-103820: Revise IDLE bindings so that events from mouse button 4/5 on non-X11 windowing systems (i.e. Win32 and Aqua) are not mistaken for scrolling.

• bpo-13586: Enter the selected text when opening the “Replace” dialog.

Tools/Demos

• gh-109991: Update GitHub CI workflows to use OpenSSL 3.0.13 and multissltests to use 1.1.1w, 3.0.13, 3.1.5, and 3.2.1.

• gh-115015: Fix a bug in Argument Clinic that generated incorrect code for methods with no parameters that use the METH_METHOD | METH_FASTCALL | METH_KEYWORDS calling convention. Only the positional parameter count was checked; any keyword argument passed would be silently accepted.

Python 3.12.1 final

Release date: 2023-12-07

Core and Builtins

• gh-112125: Fix None.__ne__(None) returning NotImplemented instead of False

• gh-112625: Fixes a bug where a bytearray object could be cleared while iterating over an argument in the bytearray.join() method that could result in reading memory after it was freed.

• gh-105967: Workaround a bug in Apple’s macOS platform zlib library where zlib.crc32() and binascii.crc32() could produce incorrect results on multi-gigabyte inputs. Including when using zipfile on zips containing large data.

• gh-112356: Stopped erroneously deleting a LOAD_NULL bytecode instruction when optimized twice.

• gh-111058: Change coro.cr_frame/gen.gi_frame to return None after the coroutine/generator has been closed. This fixes a bug where getcoroutinestate() and getgeneratorstate() return the wrong state for a closed coroutine/generator.

• gh-112388: Fix an error that was causing the parser to try to overwrite tokenizer errors. Patch by pablo Galindo

• gh-112387: Fix error positions for decoded strings with backwards tokenize errors. Patch by Pablo Galindo

• gh-112367: Avoid undefined behaviour when using the perf trampolines by not freeing the code arenas until shutdown. Patch by Pablo Galindo

• gh-112243: Don’t include comments in f-string debug expressions. Patch by Pablo Galindo

• gh-112266: Change docstrings of __dict__ and __weakref__.

• gh-111654: Fix runtime crash when some error happens in opcode LOAD_FROM_DICT_OR_DEREF.

• gh-109181: Speed up Traceback object creation by lazily compute the line number. Patch by Pablo Galindo

• gh-102388: Fix a bug where iso2022_jp_3 and iso2022_jp_2004 codecs read out of bounds

• gh-111366: Fix an issue in the codeop that was causing SyntaxError exceptions raised in the presence of invalid syntax to not contain precise error messages. Patch by Pablo Galindo

• gh-111380: Fix a bug that was causing SyntaxWarning to appear twice when parsing if invalid syntax is encountered later. Patch by Pablo galindo

• gh-94438: Fix a regression that prevented jumping across is None and is not None when debugging. Patch by Savannah Ostrowski.

• gh-110938: Fix error messages for indented blocks with functions and classes with generic type parameters. Patch by Pablo Galindo

• gh-109894: Fixed crash due to improperly initialized static MemoryError in subinterpreter.

• gh-110782: Fix crash when typing.TypeVar is constructed with a keyword argument. Patch by Jelle Zijlstra.

• gh-110696: Fix incorrect error message for invalid argument unpacking. Patch by Pablo Galindo

• gh-110543: Fix regression in Python 3.12 where types.CodeType.replace() would produce a broken code object if called on a module or class code object that contains a comprehension. Patch by Jelle Zijlstra.

• gh-110514: Add PY_THROW to sys.setprofile() events

• gh-110455: Guard assert(tstate->thread_id > 0) with #ifndef HAVE_PTHREAD_STUBS. This allows for for pydebug builds to work under WASI which (currently) lacks thread support.

• gh-110259: Correctly identify the format spec in f-strings (with single or triple quotes) that have multiple lines in the expression part and include a formatting spec. Patch by Pablo Galindo

• gh-110237: Fix missing error checks for calls to PyList_Append in _PyEval_MatchClass.

• gh-109889: Fix the compiler’s redundant NOP detection algorithm to skip over NOPs with no line number when looking for the next instruction’s lineno.

• gh-109853: sys.path[0] is now set correctly for subinterpreters.

• gh-105716: Subinterpreters now correctly handle the case where they have threads running in the background. Before, such threads would interfere with cleaning up and destroying them, as well as prevent running another script.

• gh-109793: The main thread no longer exits prematurely when a subinterpreter is cleaned up during runtime finalization. The bug was a problem particularly because, when triggered, the Python process would always return with a 0 exitcode, even if it failed.

• gh-109596: Fix some tokens in the grammar that were incorrectly marked as soft keywords. Also fix some repeated rule names and ensure that repeated rules are not allowed. Patch by Pablo Galindo

• gh-109351: Fix crash when compiling an invalid AST involving a named (walrus) expression.

• gh-109216: Fix possible memory leak in BUILD_MAP.

• gh-109207: Fix a SystemError in __repr__ of symtable entry object.

• gh-109179: Fix bug where the C traceback display drops notes from SyntaxError.

• gh-109052: Use the base opcode when comparing code objects to avoid interference from instrumentation

• gh-88943: Improve syntax error for non-ASCII character that follows a numerical literal. It now points on the invalid non-ASCII character, not on the valid numerical literal.

• gh-106931: Statically allocated string objects are now interned globally instead of per-interpreter. This fixes a situation where such a string would only be interned in a single interpreter. Normal string objects are unaffected.

Library

• gh-79325: Fix an infinite recursion error in tempfile.TemporaryDirectory() cleanup on Windows.

• gh-112645: Remove deprecation error on passing onerror to shutil.rmtree().

• gh-112618: Fix a caching bug relating to typing.Annotated. Annotated[str, True] is no longer identical to Annotated[str, 1].

• gh-112334: Fixed a performance regression in 3.12’s subprocess on Linux where it would no longer use the fast-path vfork() system call when it should have due to a logic bug, instead always falling back to the safe but slower fork().

  Also fixed a related 3.12 security regression: If a value of extra_groups=[] was passed to subprocess.Popen or related APIs, the underlying setgroups(0, NULL) system call to clear the groups list would not be made in the child process prior to exec(). This has been assigned CVE-2023-6507.

  This was identified via code inspection in the process of fixing the first bug.

• gh-110190: Fix ctypes structs with array on Arm platform by setting MAX_STRUCT_SIZE to 32 in stgdict. Patch by Diego Russo.

• gh-112578: Fix a spurious RuntimeWarning when executing the zipfile module.

• gh-112509: Fix edge cases that could cause a key to be present in both the __required_keys__ and __optional_keys__ attributes of a typing.TypedDict. Patch by Jelle Zijlstra.

• gh-112414: Fix regression in Python 3.12 where calling repr() on a module that had been imported using a custom loader could fail with AttributeError. Patch by Alex Waygood.

• gh-112358: Revert change to struct.Struct initialization that broke some cases of subclassing.

• gh-94722: Fix bug where comparison between instances of DocTest fails if one of them has None as its lineno.

• gh-112105: Make readline.set_completer_delims() work with libedit

• gh-111942: Fix SystemError in the TextIOWrapper constructor with non-encodable “errors” argument in non-debug mode.

• gh-109538: Issue warning message instead of having RuntimeError be displayed when event loop has already been closed at StreamWriter.__del__().

• gh-111942: Fix crashes in io.TextIOWrapper.reconfigure() when pass invalid arguments, e.g. non-string encoding.

• gh-111460: curses: restore wide character support (including curses.unget_wch() and get_wch()) on macOS, which was unavailable due to a regression in Python 3.12.

• gh-103791: contextlib.suppress now supports suppressing exceptions raised as part of a BaseExceptionGroup, in addition to the recent support for ExceptionGroup.

• gh-111804: Remove posix.fallocate() under WASI as the underlying posix_fallocate() is not available in WASI preview2.

• gh-111841: Fix truncating arguments on an embedded null character in os.putenv() and os.unsetenv() on Windows.

• gh-111541: Fix doctest for SyntaxError not-builtin subclasses.

• gh-110894: Call loop exception handler for exceptions in client_connected_cb of asyncio.start_server() so that applications can handle it. Patch by Kumar Aditya.

• gh-111531: Fix reference leaks in bind_class() and bind_all() methods of tkinter widgets.

• gh-111356: Added io.text_encoding(), io.DEFAULT_BUFFER_SIZE, and io.IncrementalNewlineDecoder to io.__all__.

• gh-111342: Fixed typo in math.sumprod().

• gh-68166: Remove mention of not supported “vsapi” element type in tkinter.ttk.Style.element_create(). Add tests for element_create() and other ttk.Style methods. Add examples for element_create() in the documentation.

• gh-75666: Fix the behavior of tkinter widget’s unbind() method with two arguments. Previously, widget.unbind(sequence, funcid) destroyed the current binding for sequence, leaving sequence unbound, and deleted the funcid command. Now it removes only funcid from the binding for sequence, keeping other commands, and deletes the funcid command. It leaves sequence unbound only if funcid was the last bound command.

• gh-79033: Another attempt at fixing asyncio.Server.wait_closed(). It now blocks until both conditions are true: the server is closed, and there are no more active connections. (This means that in some cases where in 3.12.0 this function would incorrectly have returned immediately, it will now block; in particular, when there are no active connections but the server hasn’t been closed yet.)

• gh-111295: Fix time not checking for errors when initializing.

• gh-111253: Add error checking during _socket module init.

• gh-111251: Fix _blake2 not checking for errors when initializing.

• gh-111174: Fix crash in io.BytesIO.getbuffer() called repeatedly for empty BytesIO.

• gh-111187: Postpone removal version for locale.getdefaultlocale() to Python 3.15.

• gh-111159: Fix doctest output comparison for exceptions with notes.

• gh-110910: Fix invalid state handling in asyncio.TaskGroup and asyncio.Timeout. They now raise proper RuntimeError if they are improperly used and are left in consistent state after this.

• gh-111092: Make turtledemo run without default root enabled.

• gh-110488: Fix a couple of issues in pathlib.PurePath.with_name(): a single dot was incorrectly considered a valid name, and in PureWindowsPath, a name with an NTFS alternate data stream, like a:b, was incorrectly considered invalid.

• gh-110392: Fix tty.setraw() and tty.setcbreak(): previously they returned partially modified list of the original tty attributes. tty.cfmakeraw() and tty.cfmakecbreak() now make a copy of the list of special characters before modifying it.

• gh-110590: Fix a bug in _sre.compile() where TypeError would be overwritten by OverflowError when the code argument was a list of non-ints.

• gh-65052: Prevent pdb from crashing when trying to display undisplayable objects

• gh-110519: Deprecation warning about non-integer number in gettext now alwais refers to the line in the user code where gettext function or method is used. Previously it could refer to a line in gettext code.

• gh-110395: Ensure that select.kqueue() objects correctly appear as closed in forked children, to prevent operations on an invalid file descriptor.

• gh-110378: contextmanager() and asynccontextmanager() context managers now close an invalid underlying generator object that yields more then one value.

• gh-110365: Fix termios.tcsetattr() bug that was overwritting existing errors during parsing integers from term list.

• gh-109653: Fix a Python 3.12 regression in the import time of random. Patch by Alex Waygood.

• gh-110196: Add __reduce__ method to IPv6Address in order to keep scope_id

• gh-110036: On Windows, multiprocessing Popen.terminate() now catchs PermissionError and get the process exit code. If the process is still running, raise again the PermissionError. Otherwise, the process terminated as expected: store its exit code. Patch by Victor Stinner.

• gh-110038: Fixed an issue that caused KqueueSelector.select() to not return all the ready events in some cases when a file descriptor is registered for both read and write.

• gh-109631: re functions such as re.findall(), re.split(), re.search() and re.sub() which perform short repeated matches can now be interrupted by user.

• gh-109747: Improve errors for unsupported look-behind patterns. Now re.error is raised instead of OverflowError or RuntimeError for too large width of look-behind pattern.

• gh-109818: Fix reprlib.recursive_repr() not copying __type_params__ from decorated function.

• gh-109047: concurrent.futures: The executor manager thread now catches exceptions when adding an item to the call queue. During Python finalization, creating a new thread can now raise RuntimeError. Catch the exception and call terminate_broken() in this case. Patch by Victor Stinner.

• gh-109782: Ensure the signature of os.path.isdir() is identical on all platforms. Patch by Amin Alaee.

• gh-109590: shutil.which() will prefer files with an extension in PATHEXT if the given mode includes os.X_OK on win32. If no PATHEXT match is found, a file without an extension in PATHEXT can be returned. This change will have shutil.which() act more similarly to previous behavior in Python 3.11.

• gh-109786: Fix possible reference leaks and crash when re-enter the __next__() method of itertools.pairwise.

• gh-109593: Avoid deadlocking on a reentrant call to the multiprocessing resource tracker. Such a reentrant call, though unlikely, can happen if a GC pass invokes the finalizer for a multiprocessing object such as SemLock.

• gh-109613: Fix os.stat() and os.DirEntry.stat(): check for exceptions. Previously, on Python built in debug mode, these functions could trigger a fatal Python error (and abort the process) when a function succeeded with an exception set. Patch by Victor Stinner.

• gh-109375: The pdb alias command now prevents registering aliases without arguments.

• gh-107219: Fix a race condition in concurrent.futures. When a process in the process pool was terminated abruptly (while the future was running or pending), close the connection write end. If the call queue is blocked on sending bytes to a worker process, closing the connection write end interrupts the send, so the queue can be closed. Patch by Victor Stinner.

• gh-50644: Attempts to pickle or create a shallow or deep copy of codecs streams now raise a TypeError. Previously, copying failed with a RecursionError, while pickling produced wrong results that eventually caused unpickling to fail with a RecursionError.

• gh-108987: Fix _thread.start_new_thread() race condition. If a thread is created during Python finalization, the newly spawned thread now exits immediately instead of trying to access freed memory and lead to a crash. Patch by Victor Stinner.

• gh-108791: Improved error handling in pdb command line interface, making it produce more concise error messages.

• gh-105829: Fix concurrent.futures.ProcessPoolExecutor deadlock

• gh-106584: Fix exit code for unittest if all tests are skipped. Patch by Egor Eliseev.

• gh-102956: Fix returning of empty byte strings after seek in zipfile module

• gh-84867: unittest.TestLoader no longer loads test cases from exact unittest.TestCase and unittest.FunctionTestCase classes.

• gh-91133: Fix a bug in tempfile.TemporaryDirectory cleanup, which now no longer dereferences symlinks when working around file system permission errors.

• gh-73561: Omit the interface scope from an IPv6 address when used as Host header by http.client.

• gh-86826: zipinfo now supports the full range of values in the TZ string determined by RFC 8536 and detects all invalid formats. Both Python and C implementations now raise exceptions of the same type on invalid data.

• bpo-43153: On Windows, tempfile.TemporaryDirectory previously masked a PermissionError with NotADirectoryError during directory cleanup. It now correctly raises PermissionError if errors are not ignored. Patch by Andrei Kulakov and Ken Jin.

• bpo-35332: The shutil.rmtree() function now ignores errors when calling os.close() when ignore_errors is True, and os.close() no longer retried after error.

• bpo-41422: Fixed memory leaks of pickle.Pickler and pickle.Unpickler involving cyclic references via the internal memo mapping.

• bpo-40262: The ssl.SSLSocket.recv_into() method no longer requires the buffer argument to implement __len__ and supports buffers with arbitrary item size.

Documentation

• gh-111699: Relocate smtpd deprecation notice to its own section rather than under locale in What’s New in Python 3.12 document

• gh-108826: dis module command-line interface is now mentioned in documentation.

Tests

• gh-112769: The tests now correctly compare zlib version when zlib.ZLIB_RUNTIME_VERSION contains non-integer suffixes. For example zlib-ng defines the version as 1.3.0.zlib-ng.

• gh-110367: Make regrtest --verbose3 option compatible with --huntrleaks -jN options. The ./python -m test -j1 -R 3:3 --verbose3 command now works as expected. Patch by Victor Stinner.

• gh-111165: Remove no longer used functions run_unittest() and run_doctest() from the test.support module.

• gh-110932: Fix regrtest if the SOURCE_DATE_EPOCH environment variable is defined: use the variable value as the random seed. Patch by Victor Stinner.

• gh-110995: test_gdb: Fix detection of gdb built without Python scripting support. Patch by Victor Stinner.

• gh-110918: Test case matching patterns specified by options --match, --ignore, --matchfile and --ignorefile are now tested in the order of specification, and the last match determines whether the test case be run or ignored.

• gh-110647: Fix test_stress_modifying_handlers() of test_signal. Patch by Victor Stinner.

• gh-103053: Fix test_tools.test_freeze on FreeBSD: run “make distclean” instead of “make clean” in the copied source directory to remove also the “python” program. Patch by Victor Stinner.

• gh-110167: Fix a deadlock in test_socket when server fails with a timeout but the client is still running in its thread. Don’t hold a lock to call cleanup functions in doCleanups(). One of the cleanup function waits until the client completes, whereas the client could deadlock if it called addCleanup() in such situation. Patch by Victor Stinner.

• gh-110388: Add tests for tty.

• gh-81002: Add tests for termios.

• gh-110267: Add tests for pickling and copying PyStructSequence objects. Patched by Xuehai Pan.

• gh-110031: Skip test_threading tests using thread+fork if Python is built with Address Sanitizer (ASAN). Patch by Victor Stinner.

• gh-110088: Fix test_asyncio timeouts: don’t measure the maximum duration, a test should not measure a CI performance. Only measure the minimum duration when a task has a timeout or delay. Add CLOCK_RES to test_asyncio.utils. Patch by Victor Stinner.

• gh-109974: Fix race conditions in test_threading lock tests. Wait until a condition is met rather than using time.sleep() with a hardcoded number of seconds. Patch by Victor Stinner.

• gh-110033: Fix test_interprocess_signal() of test_signal. Make sure that the subprocess.Popen object is deleted before the test raising an exception in a signal handler. Otherwise, Popen.__del__() can get the exception which is logged as Exception ignored in: ... and the test fails. Patch by Victor Stinner.

• gh-109594: Fix test_timeout() of test_concurrent_futures.test_wait. Remove the future which may or may not complete depending if it takes longer than the timeout ot not. Keep the second future which does not complete before wait() timeout. Patch by Victor Stinner.

• gh-109972: Split test_gdb.py file into a test_gdb package made of multiple tests, so tests can now be run in parallel. Patch by Victor Stinner.

• gh-103053: Skip test_freeze_simple_script() of test_tools.test_freeze if Python is built with ./configure --enable-optimizations, which means with Profile Guided Optimization (PGO): it just makes the test too slow. The freeze tool is tested by many other CIs with other (faster) compiler flags. Patch by Victor Stinner.

• gh-109580: Skip test_perf_profiler if Python is built with ASAN, MSAN or UBSAN sanitizer. Python does crash randomly in this test on such build. Patch by Victor Stinner.

• gh-104736: Fix test_gdb on Python built with LLVM clang 16 on Linux ppc64le (ex: Fedora 38). Search patterns in gdb “bt” command output to detect when gdb fails to retrieve the traceback. For example, skip a test if Backtrace stopped: frame did not save the PC is found. Patch by Victor Stinner.

• gh-108927: Fixed order dependence in running tests in the same process when a test that has submodules (e.g. test_importlib) follows a test that imports its submodule (e.g. test_importlib.util) and precedes a test (e.g. test_unittest or test_compileall) that uses that submodule.

Build

• gh-112088: Add Tools/build/regen-configure.sh script to regenerate the configure with an Ubuntu container image. The quay.io/tiran/cpython_autoconf:271 container image (tiran/cpython_autoconf) is no longer used. Patch by Victor Stinner.

• gh-111046: For wasi-threads, memory is now exported to fix compatibility issues with some wasm runtimes.

• gh-103053: “make check-clean-src” now also checks if the “python” program is found in the source directory: fail with an error if it does exist. Patch by Victor Stinner.

• gh-109191: Fix compile error when building with recent versions of libedit.

Windows

• gh-111856: Fixes fstat() on file systems that do not support file ID requests. This includes FAT32 and exFAT.

• gh-111293: Fix os.DirEntry.inode dropping higher 64 bits of a file id on some filesystems on Windows.

• gh-110913: WindowsConsoleIO now correctly chunks large buffers without splitting up UTF-8 sequences.

• gh-110437: Allows overriding the source of VC redistributables so that releases can be guaranteed to never downgrade between updates.

• gh-109286: Update Windows installer to use SQLite 3.43.1.

macOS

• gh-109981: Use /dev/fd on macOS to determine the number of open files in test.support.os_helper.fd_count to avoid a crash with “guarded” file descriptors when probing for open files.

• gh-110950: Update macOS installer to include an upstream Tcl/Tk fix for the Secure coding is not enabled for restorable state! warning encountered in Tkinter on macOS 14 Sonoma.

• gh-111015: Ensure that IDLE.app and Python Launcher.app are installed with appropriate permissions on macOS builds.

• gh-109286: Update macOS installer to use SQLite 3.43.1.

• gh-71383: Update macOS installer to include an upstream Tcl/Tk fix for the ttk::ThemeChanged error encountered in Tkinter.

• gh-92603: Update macOS installer to include a fix accepted by upstream Tcl/Tk for a crash encountered after the first tkinter.Tk() instance is destroyed.

IDLE

• bpo-35668: Add docstrings to the IDLE debugger module. Fix two bugs: initialize Idb.botframe (should be in Bdb); in Idb.in_rpc_code, check whether prev_frame is None before trying to use it. Greatly expand test_debugger.

C API

• gh-106560: Fix redundant declarations in the public C API. Declare PyBool_Type and PyLong_Type only once. Patch by Victor Stinner.

• gh-112438: Fix support of format units “es”, “et”, “es#”, and “et#” in nested tuples in PyArg_ParseTuple()-like functions.

• gh-109521: PyImport_GetImporter() now sets RuntimeError if it fails to get sys.path_hooks or sys.path_importer_cache or they are not list and dict correspondingly. Previously it could return NULL without setting error in obscure cases, crash or raise SystemError if these attributes have wrong type.

Python 3.12.0 final

Release date: 2023-10-02

Core and Builtins

• gh-109823: Fix bug where compiler does not adjust labels when removing an empty basic block which is a jump target.

• gh-109719: Fix missing jump target labels when compiler reorders cold/warm blocks.

• gh-109627: Fix bug where the compiler does not assign a new jump target label to a duplicated small exit block.

Library

• gh-110045: Update the symtable module to support the new scopes introduced by PEP 695.

Documentation

• gh-109209: The minimum Sphinx version required for the documentation is now 4.2.

Windows

• gh-109991: Update Windows build to use OpenSSL 3.0.11.

macOS

• gh-109991: Update macOS installer to use OpenSSL 3.0.11.

Tools/Demos

• gh-109991: Update GitHub CI workflows to use OpenSSL 3.0.11 and multissltests to use 1.1.1w, 3.0.11, and 3.1.3.

Python 3.12.0 release candidate 3

Release date: 2023-09-18

Core and Builtins

• gh-109496: On a Python built in debug mode, Py_DECREF() now calls _Py_NegativeRefcount() if the object is a dangling pointer to deallocated memory: memory filled with 0xDD “dead byte” by the debug hook on memory allocators. The fix is to check the reference count before checking for _Py_IsImmortal(). Patch by Victor Stinner.

• gh-109371: Deopted instructions correctly for tool initialization and modified the incorrect assertion in instrumentation, when a previous tool already sets INSTRUCTION events

• gh-105658: Fix bug where the line trace of an except block ending with a conditional includes an excess event with the line of the conditional expression.

• gh-109219: Fix compiling type param scopes that use a name which is also free in an inner scope.

• gh-109341: Fix crash when compiling an invalid AST involving a ast.TypeAlias.

• gh-109195: Fix source location for the LOAD_* instruction preceding a LOAD_SUPER_ATTR to load the super global (or shadowing variable) so that it encompasses only the name super and not the following parentheses.

• gh-109118: Disallow nested scopes (lambdas, generator expressions, and comprehensions) within PEP 695 annotation scopes that are nested within classes.

• gh-109114: Relax the detection of the error message for invalid lambdas inside f-strings to not search for arbitrary replacement fields to avoid false positives. Patch by Pablo Galindo

• gh-109118: Fix interpreter crash when a NameError is raised inside the type parameters of a generic class.

• gh-108976: Fix crash that occurs after de-instrumenting a code object in a monitoring callback.

• gh-108732: Make iteration variables of module- and class-scoped comprehensions visible to pdb and other tools that use frame.f_locals again.

• gh-108959: Fix caret placement for error locations for subscript and binary operations that involve non-semantic parentheses and spaces. Patch by Pablo Galindo

Library

• gh-108682: Enum: require names=() or type=... to create an empty enum using the functional syntax.

• gh-108843: Fix an issue in ast.unparse() when unparsing f-strings containing many quote types.

Documentation

• gh-102823: Document the return type of x // y when x and y have type float.

Tests

• gh-109396: Fix test_socket.test_hmac_sha1() in FIPS mode. Use a longer key: FIPS mode requires at least of at least 112 bits. The previous key was only 32 bits. Patch by Victor Stinner.

• gh-104736: Fix test_gdb on Python built with LLVM clang 16 on Linux ppc64le (ex: Fedora 38). Search patterns in gdb “bt” command output to detect when gdb fails to retrieve the traceback. For example, skip a test if Backtrace stopped: frame did not save the PC is found. Patch by Victor Stinner.

• gh-109237: Fix test_site.test_underpth_basic() when the working directory contains at least one non-ASCII character: encode the ._pth file to UTF-8 and enable the UTF-8 Mode to use UTF-8 for the child process stdout. Patch by Victor Stinner.

• gh-109230: Fix test_pyexpat.test_exception(): it can now be run from a directory different than Python source code directory. Before, the test failed in this case. Skip the test if Modules/pyexpat.c source is not available. Skip also the test on Python implementations other than CPython. Patch by Victor Stinner.

• gh-109015: Fix test_asyncio, test_imaplib and test_socket tests on FreeBSD if the TCP blackhole is enabled (sysctl net.inet.tcp.blackhole). Skip the few tests which failed with ETIMEDOUT which such non standard configuration. Currently, the FreeBSD GCP image enables TCP and UDP blackhole (sysctl net.inet.tcp.blackhole=2 and sysctl net.inet.udp.blackhole=1). Patch by Victor Stinner.

• gh-91960: Skip test_gdb if gdb is unable to retrieve Python frame objects: if a frame is <optimized out>. When Python is built with “clang -Og”, gdb can fail to retrive the frame parameter of _PyEval_EvalFrameDefault(). In this case, tests like py_bt() are likely to fail. Without getting access to Python frames, python-gdb.py is mostly clueless on retrieving the Python traceback. Moreover, test_gdb is no longer skipped on macOS if Python is built with Clang. Patch by Victor Stinner.

• gh-108962: Skip test_tempfile.test_flags() if chflags() fails with “OSError: [Errno 45] Operation not supported” (ex: on FreeBSD 13). Patch by Victor Stinner.

• gh-108851: Fix test_tomllib recursion tests for WASI buildbots: reduce the recursion limit and compute the maximum nested array/dict depending on the current available recursion limit. Patch by Victor Stinner.

• gh-108851: Add get_recursion_available() and get_recursion_depth() functions to the test.support module. Patch by Victor Stinner.

• gh-108834: Add --fail-rerun option option to regrtest: if a test failed when then passed when rerun in verbose mode, exit the process with exit code 2 (error), instead of exit code 0 (success). Patch by Victor Stinner.

• gh-108834: Rename regrtest --verbose2 option (-w) to --rerun. Keep --verbose2 as a deprecated alias. Patch by Victor Stinner.

• gh-108834: When regrtest reruns failed tests in verbose mode (./python -m test --rerun), tests are now rerun in fresh worker processes rather than being executed in the main process. If a test does crash or is killed by a timeout, the main process can detect and handle the killed worker process. Tests are rerun in parallel if the -jN option is used to run tests in parallel. Patch by Victor Stinner.

• gh-103186: Suppress and assert expected RuntimeWarnings in test_sys_settrace.py

Build

• gh-108740: Fix a race condition in make regen-all. The deepfreeze.c source and files generated by Argument Clinic are now generated or updated before generating “global objects”. Previously, some identifiers may miss depending on the order in which these files were generated. Patch by Victor Stinner.

Python 3.12.0 release candidate 2

Release date: 2023-09-05

Security

• gh-108310: Fixed an issue where instances of ssl.SSLSocket were vulnerable to a bypass of the TLS handshake and included protections (like certificate verification) and treating sent unencrypted data as if it were post-handshake TLS encrypted data. Security issue reported as CVE-2023-40217 by Aapo Oksman. Patch by Gregory P. Smith.

• gh-107774: PEP 669 specifies that sys.monitoring.register_callback will generate an audit event. Pre-releases of Python 3.12 did not generate the audit event. This is now fixed.

Core and Builtins

• gh-108520: Fix multiprocessing.synchronize.SemLock.__setstate__() to properly initialize multiprocessing.synchronize.SemLock._is_fork_ctx. This fixes a regression when passing a SemLock accross nested processes.

  Rename multiprocessing.synchronize.SemLock.is_fork_ctx to multiprocessing.synchronize.SemLock._is_fork_ctx to avoid exposing it as public API.

• gh-108654: Restore locals shadowed by an inlined comprehension if the comprehension raises an exception.

• gh-108487: Change an assert that would cause a spurious crash in a devious case that should only trigger deoptimization.

• gh-106176: Use a WeakValueDictionary to track the lists containing the modules each thread is currently importing. This helps avoid a reference leak from keeping the list around longer than necessary. Weakrefs are used as GC can’t interrupt the cleanup.

• gh-107901: Fix missing line number on JUMP_BACKWARD at the end of a for loop.

• gh-108390: Raise an exception when setting a non-local event (RAISE, EXCEPTION_HANDLED, etc.) in sys.monitoring.set_local_events.

  Fixes crash when tracing in recursive calls to Python classes.

• gh-91051: Fix abort / segfault when using all eight type watcher slots, on platforms where char is signed by default.

• gh-107724: In pre-release versions of 3.12, up to rc1, the sys.monitoring callback function for the PY_THROW event was missing the third, exception argument. That is now fixed.

• gh-107080: Trace refs builds (--with-trace-refs) were crashing when used with isolated subinterpreters. The problematic global state has been isolated to each interpreter. Other fixing the crashes, this change does not affect users.

• gh-77377: Ensure that multiprocessing synchronization objects created in a fork context are not sent to a different process created in a spawn context. This changes a segfault into an actionable RuntimeError in the parent process.

Library

• gh-108469: ast.unparse() now supports new f-string syntax introduced in Python 3.12. Note that the f-string quotes are reselected for simplicity under the new syntax. (Patch by Steven Sun)

• gh-108682: Enum: raise TypeError if super().__new__() is called from a custom __new__.

• gh-108295: Fix crashes related to use of weakrefs on typing.TypeVar.

• gh-64662: Fix support for virtual tables in sqlite3.Connection.iterdump(). Patch by Aviv Palivoda.

• gh-108111: Fix a regression introduced in gh-101251 for 3.12, resulting in an incorrect offset calculation in gzip.GzipFile.seek().

• gh-105736: Harmonized the pure Python version of OrderedDict with the C version. Now, both versions set up their internal state in __new__. Formerly, the pure Python version did the set up in __init__.

• gh-108083: Fix bugs in the constructor of sqlite3.Connection and sqlite3.Connection.close() where exceptions could be leaked. Patch by Erlend E. Aasland.

• gh-107963: Fix multiprocessing.set_forkserver_preload() to check the given list of modules names. Patch by Donghee Na.

• gh-106242: Fixes os.path.normpath() to handle embedded null characters without truncating the path.

• gh-107913: Fix possible losses of errno and winerror values in OSError exceptions if they were cleared or modified by the cleanup code before creating the exception object.

• gh-107845: tarfile.data_filter() now takes the location of symlinks into account when determining their target, so it will no longer reject some valid tarballs with LinkOutsideDestinationError.

• gh-107805: Fix signatures of module-level generated functions in turtle.

• gh-107715: Fix doctest.DocTestFinder.find() in presence of class names with special characters. Patch by Gertjan van Zwieten.

• gh-100814: Passing a callable object as an option value to a Tkinter image now raises the expected TclError instead of an AttributeError.

• gh-106684: Close asyncio.StreamWriter when it is not closed by application leading to memory leaks. Patch by Kumar Aditya.

• gh-107396: tarfiles; Fixed use before assignment of self.exception for gzip decompression

• gh-106052: re module: fix the matching of possessive quantifiers in the case of a subpattern containing backtracking.

• gh-100061: Fix a bug that causes wrong matches for regular expressions with possessive qualifier.

• gh-99203: Restore following CPython <= 3.10.5 behavior of shutil.make_archive(): do not create an empty archive if root_dir is not a directory, and, in that case, raise FileNotFoundError or NotADirectoryError regardless of format choice. Beyond the brought-back behavior, the function may now also raise these exceptions in dry_run mode.

Documentation

• gh-105052: Update timeit doc to specify that time in seconds is just the default.

Tests

• gh-89392: Removed support of test_main() function in tests. They now always use normal unittest test runner.

• gh-108388: Convert test_concurrent_futures to a package of 7 sub-tests. Patch by Victor Stinner.

• gh-108388: Split test_multiprocessing_fork, test_multiprocessing_forkserver and test_multiprocessing_spawn into test packages. Each package is made of 4 sub-tests: processes, threads, manager and misc. It allows running more tests in parallel and so reduce the total test duration. Patch by Victor Stinner.

• gh-105776: Fix test_cppext when the C compiler command -std=c11 option: remove -std= options from the compiler command. Patch by Victor Stinner.

• gh-107178: Add the C API test for functions in the Mapping Protocol, the Sequence Protocol and some functions in the Object Protocol.

Build

• gh-63760: Fix Solaris build: no longer redefine the gethostname() function. Solaris defines the function since 2005. Patch by Victor Stinner, original patch by Jakub Kulík.

• gh-107814: When calling find_python.bat with -q it did not properly silence the output of nuget. That is now fixed.

Windows

• gh-107565: Update Windows build to use OpenSSL 3.0.10.

• gh-106242: Fixes realpath() to behave consistently when passed a path containing an embedded null character on Windows. In strict mode, it now raises OSError instead of the unexpected ValueError, and in non-strict mode will make the path absolute.

• gh-106844: Fix integer overflow and truncating by the null character in _winapi.LCMapStringEx() which affects ntpath.normcase().

macOS

• gh-107565: Update macOS installer to use OpenSSL 3.0.10.

Tools/Demos

• gh-107565: Update multissltests and GitHub CI workflows to use OpenSSL 1.1.1v, 3.0.10, and 3.1.2.

• gh-95065: Argument Clinic now supports overriding automatically generated signature by using directive @text_signature.

C API

• gh-107916: C API functions PyErr_SetFromErrnoWithFilename(), PyErr_SetExcFromWindowsErrWithFilename() and PyErr_SetFromWindowsErrWithFilename() save now the error code before calling PyUnicode_DecodeFSDefault().

• gh-107915: Such C API functions as PyErr_SetString(), PyErr_Format(), PyErr_SetFromErrnoWithFilename() and many others no longer crash or ignore errors if it failed to format the error message or decode the filename. Instead, they keep a corresponding error.

• gh-107810: Improve DeprecationWarning for uses of PyType_Spec with metaclasses that have custom tp_new.

Python 3.12.0 release candidate 1

Release date: 2023-08-05

Security

• gh-102988: Reverted the email.utils security improvement change released in 3.12beta4 that unintentionally caused email.utils.getaddresses to fail to parse email addresses with a comma in the quoted name field. See gh-106669.

• gh-102509: Start initializing ob_digit during creation of PyLongObject objects. Patch by Illia Volochii.

Core and Builtins

• gh-107263: Increase C recursion limit for functions other than the main interpreter from 800 to 1500. This should allow functions like list.__repr__ and json.dumps to handle all the inputs that they could prior to 3.12

• gh-104432: Fix potential unaligned memory access on C APIs involving returned sequences of char * pointers within the grp and socket modules. These were revealed using a -fsaniziter=alignment build on ARM macOS. Patch by Christopher Chavez.

• gh-106898: Add the exception as the third argument to PY_UNIND callbacks in sys.monitoring. This makes the PY_UNWIND callback consistent with the other exception hanlding callbacks.

• gh-106895: Raise a ValueError when a monitoring callback funtion returns DISABLE for events that cannot be disabled locally.

• gh-106897: Add a RERAISE event to sys.monitoring, which occurs when an exception is reraised, either explicitly by a plain raise statement, or implicitly in an except or finally block.

• gh-104621: Unsupported modules now always fail to be imported.

• gh-106917: Fix classmethod-style super() method calls (i.e., where the second argument to super(), or the implied second argument drawn from self/cls in the case of zero-arg super, is a type) when the target of the call is not a classmethod.

• gh-105699: Python no longer crashes due an infrequent race when initialzing per-interpreter interned strings. The crash would manifest when the interpreter was finalized.

• gh-105699: Python no longer crashes due to an infrequent race in setting Py_FileSystemDefaultEncoding and Py_FileSystemDefaultEncodeErrors (both deprecated), when simultaneously initializing two isolated subinterpreters. Now they are only set during runtime initialization.

• gh-106092: Fix a segmentation fault caused by a use-after-free bug in frame_dealloc when the trashcan delays the deallocation of a PyFrameObject.

• gh-106719: No longer suppress arbitrary errors in the __annotations__ getter and setter in the type and module types.

• gh-106723: Propagate frozen_modules to multiprocessing spawned process interpreters.

• gh-105235: Prevent out-of-bounds memory access during mmap.find() calls.

Library

• gh-107077: Seems that in some conditions, OpenSSL will return SSL_ERROR_SYSCALL instead of SSL_ERROR_SSL when a certification verification has failed, but the error parameters will still contain ERR_LIB_SSL and SSL_R_CERTIFICATE_VERIFY_FAILED. We are now detecting this situation and raising the appropiate ssl.SSLCertVerificationError. Patch by Pablo Galindo

• gh-107576: Fix types.get_original_bases() to only return __orig_bases__ if it is present on cls directly. Patch by James Hilton-Balfe.

• gh-46376: Prevent memory leak and use-after-free when using pointers to pointers with ctypes

• gh-62519: Make gettext.pgettext() search plural definitions when translation is not found.

• gh-83006: Document behavior of shutil.disk_usage() for non-mounted filesystems on Unix.

• gh-106186: Do not report MultipartInvariantViolationDefect defect when the email.parser.Parser class is used to parse emails with headersonly=True.

• gh-105002: Fix invalid result from PurePath.relative_to() method when attempting to walk a “..” segment in other with walk_up enabled. A ValueError exception is now raised in this case.

• gh-106831: Fix potential missing NULL check of d2i_SSL_SESSION result in _ssl.c.

• gh-106774: Update the bundled copy of pip to version 23.2.1.

• gh-106752: Fixed several bugs in zipfile.Path, including: in Path.match, Windows separators are no longer honored (and never were meant to be); Fixed name/suffix/suffixes/stem operations when no filename is present and the Path is not at the root of the zipfile; Reworked glob for performance and more correct matching behavior.

• gh-106602: Add __copy__ and __deepcopy__ in enum

• gh-106530: Revert a change to colorsys.rgb_to_hls() that caused division by zero for certain almost-white inputs. Patch by Terry Jan Reedy.

• gh-106403: Instances of typing.TypeVar, typing.ParamSpec, typing.ParamSpecArgs, typing.ParamSpecKwargs, and typing.TypeVarTuple once again support weak references, fixing a regression introduced in Python 3.12.0 beta 1. Patch by Jelle Zijlstra.

• gh-106350: Detect possible memory allocation failure in the libtommath function mp_init() used by the _tkinter module.

• gh-106263: Fix crash when calling repr with a manually constructed SignalDict object. Patch by Charlie Zhao.

• gh-105626: Change the default return value of http.client.HTTPConnection.get_proxy_response_headers() to be None and not {}.

• bpo-18319: Ensure gettext(msg) retrieve translations even if a plural form exists. In other words: gettext(msg) == ngettext(msg, '', 1).

Documentation

• gh-107305: Add documentation for PyInterpreterConfig and Py_NewInterpreterFromConfig(). Also clarify some of the nearby docs relative to per-interpreter GIL.

• gh-107008: Document the curses module variables LINES and COLS.

• gh-106948: Add a number of standard external names to nitpick_ignore.

• gh-54738: Add documentation on how to localize the argparse module.

Tests

• gh-107237: test_logging: Fix test_udp_reconnection() by increasing the timeout from 100 ms to 5 minutes (LONG_TIMEOUT). Patch by Victor Stinner.

• gh-106714: test_capi: Fix test_no_FatalError_infinite_loop() to no longer write a coredump, by using test.support.SuppressCrashReport. Patch by Victor Stinner.

• gh-104090: Avoid creating a reference to the test object in collectedDurations().

• gh-106752: Moved tests for zipfile.Path into Lib/test/test_zipfile/_path. Made zipfile._path a package.

Build

• gh-106881: Check for linux/limits.h before including it in Modules/posixmodule.c.

• gh-106962: Detect MPI compilers in configure.

• gh-101538: Add experimental wasi-threads support. Patch by Takashi Yamamoto.

Windows

• gh-99079: Update Windows build to use OpenSSL 3.0.9

macOS

• gh-99079: Update macOS installer to use OpenSSL 3.0.9.

Tools/Demos

• gh-106970: Fix bugs in the Argument Clinic destination <name> clear command; the destination buffers would never be cleared, and the destination directive parser would simply continue to the fault handler after processing the command. Patch by Erlend E. Aasland.

• gh-103186: freeze now fetches CONFIG_ARGS from the original CPython instance the Makefile uses to call utility scripts. Patch by Ijtaba Hussain.

C API

• gh-107226: PyModule_AddObjectRef() is now only available in the limited API version 3.10 or later.

Python 3.12.0 beta 4

Release date: 2023-07-11

Security

• gh-102988: CVE-2023-27043: Prevent email.utils.parseaddr() and email.utils.getaddresses() from returning the realname portion of an invalid RFC2822 email header in the email address portion of the 2-tuple returned after being parsed by email._parseaddr.AddressList.

Core and Builtins

• gh-106396: When the format specification of an f-string expression is empty, the parser now generates an empty ast.JoinedStr node for it instead of an one-element ast.JoinedStr with an empty string ast.Constant.

• gh-106145: Make end_lineno and end_col_offset required on type_param ast nodes.

• gh-105979: Fix crash in _imp.get_frozen_object() due to improper exception handling.

• gh-98931: Ensure custom SyntaxError error messages are raised for invalid imports with multiple targets. Patch by Pablo Galindo

• gh-105908: Fixed bug where gh-99111 breaks future import barry_as_FLUFL in the Python REPL.

• gh-105340: Include the comprehension iteration variable in locals() inside a module- or class-scope comprehension.

• gh-105486: Change the repr of ParamSpec list of args in types.GenericAlias.

• gh-101006: Improve error handling when read marshal data.

Library

• gh-106524: Fix crash in _sre.template() with templates containing invalid group indices.

• gh-106510: Improve debug output for atomic groups in regular expressions.

• gh-106503: Fix ref cycle in asyncio._SelectorSocketTransport by removing _write_ready in close.

• gh-105497: Fix flag mask inversion when unnamed flags exist.

• gh-90876: Prevent multiprocessing.spawn from failing to import in environments where sys.executable is None. This regressed in 3.11 with the addition of support for path-like objects in multiprocessing.

• gh-106292: Check for an instance-dict cached value in the __get__() method of functools.cached_property(). This better matches the pre-3.12 behavior and improves compatibility for users subclassing functools.cached_property() and adding a __set__() method.

• gh-106330: Fix incorrect matching of empty paths in pathlib.PurePath.match(). This bug was introduced in Python 3.12.0 beta 1.

• gh-102541: Make pydoc.doc catch bad module ImportError when output stream is not None.

• gh-106152: Added PY_THROW event hook for cProfile for generators

• gh-106075: Added asyncio.taskgroups.__all__ to asyncio.__all__ for export in star imports.

• gh-105987: Fix crash due to improper reference counting in asyncio eager task factory internal routines.

• gh-105974: Fix bug where a typing.Protocol class that had one or more non-callable members would raise TypeError when issubclass() was called against it, even if it defined a custom __subclasshook__ method. The behaviour in Python 3.11 and lower – which has now been restored – was not to raise TypeError in these situations if a custom __subclasshook__ method was defined. Patch by Alex Waygood.

• gh-96145: Reverted addition of json.AttrDict.

• gh-105497: Fix flag inversion when alias/mask members exist.

• gh-104554: Add RTSPS scheme support in urllib.parse

• gh-94777: Fix hanging multiprocessing ProcessPoolExecutor when a child process crashes while data is being written in the call queue.

Documentation

• gh-106232: Make timeit doc command lines compatible with Windows by using double quotes for arguments. This works on linux and macOS also.

Tests

• gh-101634: When running the Python test suite with -jN option, if a worker stdout cannot be decoded from the locale encoding report a failed testn so the exitcode is non-zero. Patch by Victor Stinner.

Build

• gh-106118: Fix compilation for platforms without O_CLOEXEC. The issue was introduced with Python 3.12b1 in gh-103295. Patch by Erlend Aasland.

• gh-104692: Include commoninstall as a prerequisite for bininstall

  This ensures that commoninstall is completed before bininstall is started when parallel builds are used (make -j install), and so the python3 symlink is only installed after all standard library modules are installed.

Tools/Demos

• gh-106359: Argument Clinic now explicitly forbids “kwarg splats” in function calls used as annotations.

C API

• gh-105227: The new PyType_GetDict() provides the dictionary for the given type object that is normally exposed by cls.__dict__. Normally it’s sufficient to use tp_dict, but for the static builtin types tp_dict is now always NULL. PyType_GetDict() provides the correct dict object instead.

Python 3.12.0 beta 3

Release date: 2023-06-19

Core and Builtins

• gh-105840: Fix possible crashes when specializing function calls with too many __defaults__.

• gh-105831: Fix an f-string bug, where using a debug expression (the = sign) that appears in the last line of a file results to the debug buffer that holds the expression text being one character too small.

• gh-105800: Correctly issue SyntaxWarning in f-strings if invalid sequences are used. Patch by Pablo Galindo

• gh-105587: The runtime can’t guarantee that immortal objects will not be mutated by Extensions. Thus, this modifies _PyStaticObject_CheckRefcnt to warn instead of asserting.

• gh-105564: Don’t include artificil newlines in the line attribute of tokens in the APIs of the tokenize module. Patch by Pablo Galindo

• gh-105549: Tokenize separately NUMBER and NAME tokens that are not ambiguous. Patch by Pablo Galindo.

• gh-105588: Fix an issue that could result in crashes when compiling malformed ast nodes.

• gh-105375: Fix bugs in the builtins module where exceptions could end up being overwritten.

• gh-105375: Fix bug in the compiler where an exception could end up being overwritten.

• gh-105375: Improve error handling in PyUnicode_BuildEncodingMap() where an exception could end up being overwritten.

• gh-105435: Fix spurious newline character if file ends on a comment without a newline. Patch by Pablo Galindo

• gh-105390: Correctly raise tokenize.TokenError exceptions instead of SyntaxError for tokenize errors such as incomplete input. Patch by Pablo Galindo

• gh-104812: The “pending call” machinery now works for all interpreters, not just the main interpreter, and runs in all threads, not just the main thread. Some calls are still only done in the main thread, ergo in the main interpreter. This change does not affect signal handling nor the existing public C-API (Py_AddPendingCall()), which both still only target the main thread. The new functionality is meant strictly for internal use for now, since consequences of its use are not well understood yet outside some very restricted cases. This change brings the capability in line with the intention when the state was made per-interpreter several years ago.

Library

• gh-105808: Fix a regression introduced in gh-101251 for 3.12, causing gzip.GzipFile.flush() to not flush the compressor (nor pass along the zip_mode argument).

• gh-104799: Enable ast.unparse() to unparse function and class definitions created without the new type_params field from PEP 695. Patch by Jelle Zijlstra.

• gh-105745: Fix webbrowser.Konqueror.open method.

• gh-105375: Fix a bug in _Unpickler_SetInputStream() where an exception could end up being overwritten in case of failure.

• gh-105375: Fix bugs in sys where exceptions could end up being overwritten because of deferred error handling.

• gh-105605: Harden pyexpat error handling during module initialisation to prevent exceptions from possibly being overwritten, and objects from being dereferenced twice.

• gh-105375: Fix bug in decimal where an exception could end up being overwritten.

• gh-105375: Fix bugs in _datetime where exceptions could be overwritten in case of module initialisation failure.

• gh-105375: Fix bugs in _ssl initialisation which could lead to leaked references and overwritten exceptions.

• gh-105375: Fix a bug in array.array where an exception could end up being overwritten.

• gh-105375: Fix bugs in _ctypes where exceptions could end up being overwritten.

• gh-105375: Fix a bug in the posix module where an exception could be overwritten.

• gh-105375: Fix bugs in _elementtree where exceptions could be overwritten.

• gh-105375: Fix bugs in zoneinfo where exceptions could be overwritten.

• gh-105375: Fix bugs in errno where exceptions could be overwritten.

• gh-105375: Fix bugs in pickle where exceptions could be overwritten.

• gh-105375: Fix a bug in sqlite3 where an exception could be overwritten in the collation callback.

• gh-105332: Revert pickling method from by-name back to by-value.

• gh-104310: In the beta 1 release we added a utility function for extension module authors, to use when testing their module for support in multiple interpreters or under a per-interpreter GIL. The name of that function has changed from allowing_all_extensions to _incompatible_extension_module_restrictions. The default for the “disable_check” argument has change from True to False, to better match the new function name.

• gh-104996: Improve performance of pathlib.PurePath initialisation by deferring joining of paths when multiple arguments are given.

• gh-102541: Hide traceback in help() prompt, when import failed.

Tests

• gh-105084: When the Python build is configured --with-wheel-pkg-dir, tests requiring the setuptools and wheel wheels will search for the wheels in WHEEL_PKG_DIR.

Windows

• gh-105436: Ensure that an empty environment block is terminated by two null characters, as is required by Windows.

C API

• gh-105375: Fix a bug in PyErr_WarnExplicit() where an exception could end up being overwritten if the API failed internally.

• gh-105603: We’ve renamed the new (in 3.12) PyInterpreterConfig.own_gil to PyInterpreterConfig.gil and changed the meaning of the value from “bool” to an integer with supported values of PyInterpreterConfig_DEFAULT_GIL, PyInterpreterConfig_SHARED_GIL, and PyInterpreterConfig_OWN_GIL. The default is “shared”.

• gh-105387: In the limited C API version 3.12, Py_INCREF() and Py_DECREF() functions are now implemented as opaque function calls to hide implementation details. Patch by Victor Stinner.

• gh-103968: PyType_FromMetaclass() now allows metaclasses with tp_new set to NULL.

Python 3.12.0 beta 2

Release date: 2023-06-06

Security

• gh-103142: The version of OpenSSL used in our binary builds has been upgraded to 1.1.1u to address several CVEs.

• gh-99108: Refresh our new HACL* built-in hashlib code from upstream. Built-in SHA2 should be faster and an issue with SHA3 on 32-bit platforms is fixed.

Core and Builtins

• gh-105259: Don’t include newline character for trailing NEWLINE tokens emitted in the tokenize module. Patch by Pablo Galindo

• gh-105324: Fix the main function of the tokenize module when reading from sys.stdin. Patch by Pablo Galindo

• gh-98963: Restore the ability for a subclass of property to define __slots__ or otherwise be dict-less by ignoring failures to set a docstring on such a class. This behavior had regressed in 3.12beta1. An AttributeError where there had not previously been one was disruptive to existing code.

• gh-105194: Do not escape with backslashes f-string format specifiers. Patch by Pablo Galindo

• gh-105162: Fixed bug in generator.close()/throw() where an inner iterator would be ignored when the outer iterator was instrumented.

• gh-105164: Ensure annotations are set up correctly if the only annotation in a block is within a match block. Patch by Jelle Zijlstra.

• gh-104799: Attributes of ast nodes that are lists now default to the empty list if omitted. This means that some code that previously raised TypeError when the AST node was used will now proceed with the empty list instead. Patch by Jelle Zijlstra.

• gh-105035: Fix super() calls on types with custom tp_getattro implementation (e.g. meta-types.)

• gh-105017: Show CRLF lines in the tokenize string attribute in both NL and NEWLINE tokens. Patch by Marta Gómez.

• gh-105013: Fix handling of multiline parenthesized lambdas in inspect.getsource(). Patch by Pablo Galindo

• gh-105017: Do not include an additional final NL token when parsing files having CRLF lines. Patch by Marta Gómez.

• gh-104976: Ensure that trailing DEDENT tokenize.TokenInfo objects emitted by the tokenize module are reported as in Python 3.11. Patch by Pablo Galindo

• gh-104972: Ensure that the line attribute in tokenize.TokenInfo objects in the tokenize module are always correct. Patch by Pablo Galindo

• gh-104955: Fix signature for the new __release_buffer__() slot. Patch by Jelle Zijlstra.

• gh-104690: Starting new threads and process creation through os.fork() during interpreter shutdown (such as from atexit handlers) is no longer supported. It can lead to race condition between the main Python runtime thread freeing thread states while internal threading routines are trying to allocate and use the state of just created threads. Or forked children trying to use the mid-shutdown runtime and thread state in the child process.

• gh-104879: Fix crash when accessing the __module__ attribute of type aliases defined outside a module. Patch by Jelle Zijlstra.

• gh-104825: Tokens emitted by the tokenize module do not include an implicit \n character in the line attribute anymore. Patch by Pablo Galindo

Library

• gh-105280: Fix bug where isinstance([], collections.abc.Mapping) could evaluate to True if garbage collection happened at the wrong time. The bug was caused by changes to the implementation of typing.Protocol in Python 3.12.

• gh-105239: Fix longstanding bug where issubclass(object, typing.Protocol) would evaluate to True in some edge cases. Patch by Alex Waygood.

• gh-105080: Fixed inconsistent signature on derived classes for inspect.signature()

• gh-105144: Fix a recent regression in the typing module. The regression meant that doing class Foo(X, typing.Protocol), where X was a class that had abc.ABCMeta as its metaclass, would then cause subsequent isinstance(1, X) calls to erroneously raise TypeError. Patch by Alex Waygood.

• gh-105113: Improve performance of pathlib.PurePath.match() by compiling an re.Pattern object for the entire pattern.

• gh-101588: Deprecate undocumented copy/deepcopy/pickle support for itertools.

• gh-103631: Fix pathlib.PurePosixPath(pathlib.PureWindowsPath(...)) not converting path separators to restore 3.11 compatible behavior.

• gh-104947: Make comparisons between pathlib.PureWindowsPath objects consistent across Windows and Posix to match 3.11 behavior.

• gh-104935: Fix bugs with the interaction between typing.runtime_checkable() and typing.Generic that were introduced by the PEP 695 implementation. Patch by Jelle Zijlstra.

• gh-104874: Document the __name__ and __supertype__ attributes of typing.NewType. Patch by Jelle Zijlstra.

• gh-104799: Adjust the location of the (see PEP 695) type_params field on ast.ClassDef, ast.AsyncFunctionDef, and ast.FunctionDef to better preserve backward compatibility. Patch by Jelle Zijlstra

• gh-104797: Allow typing.Protocol classes to inherit from collections.abc.Buffer. Patch by Jelle Zijlstra.

• gh-104372: On Linux where subprocess can use the vfork() syscall for faster spawning, prevent the parent process from blocking other threads by dropping the GIL while it waits for the vfork’ed child process exec() outcome. This prevents spawning a binary from a slow filesystem from blocking the rest of the application.

• gh-99108: We now release the GIL around built-in hashlib computations of reasonable size for the SHA families and MD5 hash functions, matching what our OpenSSL backed hash computations already does.

• gh-104399: Prepare the _tkinter module for building with Tcl 9.0 and future libtommath by replacing usage of deprecated functions mp_to_unsigned_bin_n() and mp_unsigned_bin_size() when necessary.

• gh-102024: Reduce calls of _idle_semaphore.release() in concurrent.futures.thread._worker().

Documentation

• gh-89455: Add missing documentation for the max_group_depth and max_group_width parameters and the exceptions attribute of the traceback.TracebackException class.

• gh-89412: Add missing documentation for the end_lineno and end_offset attributes of the traceback.TracebackException class.

• gh-104943: Remove mentions of old Python versions in typing.NamedTuple.

Build

• gh-90005: Fix a regression in configure where we could end up unintentionally linking with libbsd.

• gh-89886: Autoconf 2.71 and aclocal 1.16.4 is now required to regenerate !configure.

Windows

• gh-105146: Updated the links at the end of the installer to point to Discourse rather than the mailing lists.

• gh-103646: When installed from the Microsoft Store, pip no longer defaults to per-user installs. However, as the install directory is unwritable, it should automatically decide to do a per-user install anyway. This should resolve issues when pip is passed an option that conflicts with --user.

• gh-88745: Improve performance of shutil.copy2() by using the operating system’s CopyFile2 function. This may result in subtle changes to metadata copied along with some files, bringing them in line with normal OS behavior.

• gh-104820: Fixes stat() and related functions on file systems that do not support file ID requests. This includes FAT32 and exFAT.

• gh-104803: Add os.path.isdevdrive() to detect whether a path is on a Windows Dev Drive. Returns False on platforms that do not support Dev Drive, and is absent on non-Windows platforms.

macOS

• gh-103142: Update macOS installer to use OpenSSL 1.1.1u.

IDLE

• gh-104719: Remove IDLE’s modification of tokenize.tabsize and test other uses of tokenize data and methods.

C API

• gh-105115: PyTypeObject.tp_bases (and tp_mro) for builtin static types are now shared by all interpreters, whereas in 3.12-beta1 they were stored on PyInterpreterState. Also note that now the tuples are immortal objects.

• gh-105071: Add PyUnstable_Exc_PrepReraiseStar to the unstable C api to expose the implementation of except*.

• gh-104668: Don’t call PyOS_InputHook or PyOS_ReadlineFunctionPointer in subinterpreters, since it’s generally difficult to avoid using global state in their registered callbacks. This also avoids situations where extensions may find themselves running in a subinterpreter they don’t support (or haven’t yet been loaded in).

Python 3.12.0 beta 1

Release date: 2023-05-22

Security

• gh-99889: Fixed a security in flaw in uu.decode() that could allow for directory traversal based on the input if no out_file was specified.

• gh-104049: Do not expose the local on-disk location in directory indexes produced by http.client.SimpleHTTPRequestHandler.

• gh-99108: Upgrade built-in hashlib SHA3 implementation to a verified implementation from the HACL* project. Used when OpenSSL is not present or lacks SHA3.

• gh-102153: urllib.parse.urlsplit() now strips leading C0 control and space characters following the specification for URLs defined by WHATWG in response to CVE-2023-24329. Patch by Illia Volochii.

Core and Builtins

• gh-102856: Implement PEP 701 changes in the tokenize module. Patch by Marta Gómez Macías and Pablo Galindo Salgado

• gh-104615: Fix wrong ordering of assignments in code like a, a = x, y. Contributed by Carl Meyer.

• gh-104572: Improve syntax error message for invalid constructs in PEP 695 contexts and in annotations when from __future__ import annotations is active.

• gh-104482: Fix three error handling bugs in ast.c’s validation of pattern matching statements.

• gh-102818: Do not add a frame to the traceback in the sys.setprofile and sys.settrace trampoline functions. This ensures that frames are not duplicated if an exception is raised in the callback function, and ensures that frames are not omitted if a C callback is used and that does not add the frame.

• gh-104405: Fix an issue where some bytecode instructions could ignore PEP 523 when “inlining” calls.

• gh-103082: Change behavior of sys.monitoring.events.LINE events in sys.monitoring: Line events now occur when a new line is reached dynamically, instead of using a static approximation, as before. This makes the behavior very similar to that of “line” events in sys.settrace. This should ease porting of tools from 3.11 to 3.12.

• gh-104263: Fix float("nan") to produce a quiet NaN on platforms (like MIPS) where the meaning of the signalling / quiet bit is inverted from its usual meaning. Also introduce a new macro Py_INFINITY matching C99’s INFINITY, and refactor internals to rely on C99’s NAN and INFINITY macros instead of hard-coding bit patterns for infinities and NaNs. Thanks Sebastian Berg.

• gh-99113: Multi-phase init extension modules may now indicate that they support running in subinterpreters that have their own GIL. This is done by using Py_MOD_PER_INTERPRETER_GIL_SUPPORTED as the value for the Py_mod_multiple_interpreters module def slot. Otherwise the module, by default, cannot be imported in such subinterpreters. (This does not affect the main interpreter or subinterpreters that do not have their own GIL.) In addition to the isolation that multi-phase init already normally requires, support for per-interpreter GIL involves one additional constraint: thread-safety. If the module has external (linked) dependencies and those libraries have any state that isn’t thread-safe then the module must do the additional work to add thread-safety. This should be an uncommon case.

• gh-99113: The GIL is now (optionally) per-interpreter. This is the fundamental change for PEP 684. This is all made possible by virtue of the isolated state of each interpreter in the process. The behavior of the main interpreter remains unchanged. Likewise, interpreters created using Py_NewInterpreter() are not affected. To get an interpreter with its own GIL, call Py_NewInterpreterFromConfig().

• gh-104108: Multi-phase init extension modules may now indicate whether or not they actually support multiple interpreters. By default such modules are expected to support use in multiple interpreters. In the uncommon case that one does not, it may use the new Py_mod_multiple_interpreters module def slot. A value of 0 means the module does not support them. 1 means it does. The default is 1.

• gh-104142: Fix an issue where list or tuple repetition could fail to respect PEP 683.

• gh-104078: Improve the performance of PyObject_HasAttrString()

• gh-104066: Improve the performance of hasattr() for module objects with a missing attribute.

• gh-104028: Reduce object creation while calling callback function from gc. Patch by Donghee Na.

• gh-104018: Disallow the “z” format specifier in %-format of bytes objects.

• gh-102213: Fix performance loss when accessing an object’s attributes with __getattr__ defined.

• gh-103895: Improve handling of edge cases in showing Exception.__notes__. Ensures that the messages always end with a newline and that string/bytes are not exploded over multiple lines. Patch by Carey Metcalfe.

• gh-103907: Don’t modify the refcounts of known immortal objects (True, False, and None) in the main interpreter loop.

• gh-103899: Provide a helpful hint in the TypeError message when accidentally calling a module object that has a callable attribute of the same name (such as dis.dis() or datetime.datetime).

• gh-103845: Remove both line and instruction instrumentation before adding new ones for monitoring, to avoid newly added instrumentation being removed immediately.

• gh-103763: Implement PEP 695, adding syntactic support for generic classes, generic functions, and type aliases.

  A new type X = ... syntax is added for type aliases, which resolves at runtime to an instance of the new class typing.TypeAliasType. The value is lazily evaluated and is accessible through the .__value__ attribute. This is implemented as a new AST node ast.TypeAlias.

  New syntax (class X[T]: ..., def func[T](): ...) is added for defining generic functions and classes. This is implemented as a new type_params attribute on the AST nodes for classes and functions. This node holds instances of the new AST classes ast.TypeVar, ast.ParamSpec, and ast.TypeVarTuple.

  typing.TypeVar, typing.ParamSpec, typing.ParamSpecArgs, typing.ParamSpecKwargs, typing.TypeVarTuple, and typing.Generic are now implemented in C rather than Python.

  There are new bytecode instructions LOAD_LOCALS, LOAD_CLASSDICT_OR_GLOBAL, and LOAD_CLASSDICT_OR_DEREF to support correct resolution of names in class namespaces.

  Patch by Eric Traut, Larry Hastings, and Jelle Zijlstra.

• gh-103801: Adds three minor linting fixes to the wasm module caught that were caught by ruff.

• gh-103793: Optimized asyncio Task creation by deferring expensive string formatting (task name generation) from Task creation to the first time get_name is called. This makes asyncio benchmarks up to 5% faster.

• gh-102310: Change the error range for invalid bytes literals.

• gh-103590: Do not wrap a single exception raised from a try-except* construct in an ExceptionGroup.

• gh-103650: Change the perf map format to remove the ‘0x’ prefix from the addresses

• gh-102856: Implement the required C tokenizer changes for PEP 701. Patch by Pablo Galindo Salgado, Lysandros Nikolaou, Batuhan Taskaya, Marta Gómez Macías and sunmy2019.

• gh-100530: Clarify the error message raised when the called part of a class pattern isn’t actually a class.

• gh-101517: Fix bug in line numbers of instructions emitted for except*.

• gh-103492: Clarify SyntaxWarning with literal is comparison by specifying which literal is problematic, since comparisons using is with e.g. None and bool literals are idiomatic.

• gh-87729: Add LOAD_SUPER_ATTR (and a specialization for super().method()) to speed up super().method() and super().attr. This makes super().method() roughly 2.3x faster and brings it within 20% of the performance of a simple method call. Patch by Vladimir Matveev and Carl Meyer.

• gh-103488: Change the internal offset distinguishing yield and return target addresses, so that the instruction pointer is correct for exception handling and other stack unwinding.

• gh-82012: The bitwise inversion operator (~) on bool is deprecated. It returns the bitwise inversion of the underlying int representation such that bool(~True) == True, which can be confusing. Use not for logical negation of bools. In the rare case that you really need the bitwise inversion of the underlying int, convert to int explicitly ~int(x).

• gh-77757: Exceptions raised in a typeobject’s __set_name__ method are no longer wrapped by a RuntimeError. Context information is added to the exception as a PEP 678 note.

• gh-103333: AttributeError now retains the name attribute when pickled and unpickled.

• gh-103242: Migrate set_ecdh_curve() method not to use deprecated OpenSSL APIs. Patch by Donghee Na.

• gh-103323: We’ve replaced our use of _PyRuntime.tstate_current with a thread-local variable. This is a fairly low-level implementation detail, and there should be no change in behavior.

• gh-84436: The implementation of PEP-683 which adds Immortal Objects by using a fixed reference count that skips reference counting to make objects truly immutable.

• gh-102700: Allow built-in modules to be submodules. This allows submodules to be statically linked into a CPython binary.

• gh-103082: Implement PEP 669 Low Impact Monitoring for CPython.

• gh-88691: Reduce the number of inline CACHE entries for CALL.

• gh-102500: Make the buffer protocol accessible in Python code using the new __buffer__ and __release_buffer__ magic methods. See PEP 688 for details. Patch by Jelle Zijlstra.

• gh-97933: PEP 709: inline list, dict and set comprehensions to improve performance and reduce bytecode size.

• gh-99184: Bypass instance attribute access of __name__ in repr of weakref.ref.

• gh-98003: Complex function calls are now faster and consume no C stack space.

• bpo-39610: len() for 0-dimensional memoryview objects (such as memoryview(ctypes.c_uint8(42))) now raises a TypeError. Previously this returned 1, which was not consistent with mem_0d[0] raising an IndexError.

• bpo-31821: Fix pause_reading() to work when called from connection_made() in asyncio.

Library

• gh-104600: functools.update_wrapper() now sets the __type_params__ attribute (added by PEP 695).

• gh-104340: When an asyncio pipe protocol loses its connection due to an error, and the caller doesn’t await wait_closed() on the corresponding StreamWriter, don’t log a warning about an exception that was never retrieved. After all, according to the StreamWriter.close() docs, the wait_closed() call is optional (“not mandatory”).

• gh-104555: Fix issue where an issubclass() check comparing a class X against a runtime-checkable protocol Y with non-callable members would not cause TypeError to be raised if an isinstance() call had previously been made comparing an instance of X to Y. This issue was present in edge cases on Python 3.11, but became more prominent in 3.12 due to some unrelated changes that were made to runtime-checkable protocols. Patch by Alex Waygood.

• gh-104372: Refactored the _posixsubprocess internals to avoid Python C API usage between fork and exec when marking pass_fds= file descriptors inheritable.

• gh-104484: Added case_sensitive argument to pathlib.PurePath.match()

• gh-75367: Fix data descriptor detection in inspect.getattr_static().

• gh-104536: Fix a race condition in the internal multiprocessing.process cleanup logic that could manifest as an unintended AttributeError when calling process.close().

• gh-103857: Update datetime deprecations’ stracktrace to point to the calling line

• gh-101520: Move the core functionality of the tracemalloc module in the Python/ folder, leaving just the module wrapper in Modules/.

• gh-104392: Remove undocumented and unused _paramspec_tvars attribute from some classes in typing.

• gh-102613: Fix issue where pathlib.Path.glob() raised RecursionError when walking deep directory trees.

• gh-103000: Improve performance of dataclasses.asdict() for the common case where dict_factory is dict. Patch by David C Ellis.

• gh-104301: Allow leading whitespace in disambiguated statements in pdb.

• gh-104139: Teach urllib.parse.unsplit() to retain the "//" when assembling itms-services://?action=generate-bugs style Apple Platform Deployment URLs.

• gh-104307: socket.getnameinfo() now releases the GIL while contacting the DNS server

• gh-104310: Users may now use importlib.util.allowing_all_extensions() (a context manager) to temporarily disable the strict compatibility checks for importing extension modules in subinterpreters.

• gh-87695: Fix issue where pathlib.Path.glob() raised OSError when it encountered a symlink to an overly long path.

• gh-104265: Prevent possible crash by disallowing instantiation of the _csv.Reader and _csv.Writer types. The regression was introduced in 3.10.0a4 with PR 23224 (bpo-14935). Patch by Radislav Chugunov.

• gh-102613: Improve performance of pathlib.Path.glob() when expanding recursive wildcards (”**”) by merging adjacent wildcards and de-duplicating results only when necessary.

• gh-65772: Remove unneeded comments and code in turtle.py.

• gh-90208: Fixed issue where pathlib.Path.glob() returned incomplete results when it encountered a PermissionError. This method now suppresses all OSError exceptions, except those raised from calling is_dir() on the top-level path.

• gh-104144: Optimize asyncio.TaskGroup when using asyncio.eager_task_factory(). Skip scheduling a done callback if a TaskGroup task completes eagerly.

• gh-104144: Optimize asyncio.gather() when using asyncio.eager_task_factory() to complete eagerly if all fututres completed eagerly. Avoid scheduling done callbacks for futures that complete eagerly.

• gh-104114: Fix issue where pathlib.Path.glob() returns paths using the case of non-wildcard segments for corresponding path segments, rather than the real filesystem case.

• gh-104104: Improve performance of pathlib.Path.glob() by using re.IGNORECASE to implement case-insensitive matching.

• gh-104102: Improve performance of pathlib.Path.glob() when evaluating patterns that contain '../' segments.

• gh-103822: Update the return type of weekday to the newly added Day attribute

• gh-103629: Update the repr of typing.Unpack according to PEP 692.

• gh-103963: Make dis display the names of the args for CALL_INTRINSIC_*.

• gh-104035: Do not ignore user-defined __getstate__ and __setstate__ methods for slotted frozen dataclasses.

• gh-103987: In mmap, fix several bugs that could lead to access to memory-mapped files after they have been invalidated.

• gh-103977: Improve import time of platform module.

• gh-88773: Added turtle.teleport() to the turtle module to move a turtle to a new point without tracing a line, visible or invisible. Patch by Liam Gersten.

• gh-103935: Use io.open_code() for files to be executed instead of raw open()

• gh-68968: Fixed garbled output of assertEqual() when an input lacks final newline.

• gh-100370: Fix potential OverflowError in sqlite3.Connection.blobopen() for 32-bit builds. Patch by Erlend E. Aasland.

• gh-102628: Substitute CTRL-D with CTRL-Z in sqlite3 CLI banner when running on Windows.

• gh-103636: Module-level attributes Janu