Repository navigation
Releases
PyTorch 2.15.0 Release
PyTorch 2.15.0 Release Notes
- Highlights
- Backwards Incompatible Changes
- Deprecations
- New Features
- Improvements
- Bug fixes
- Performance
- Documentation
- Security
- Developers
Highlights
Rewritten NCCL backend becomes the default, with a flight recorder that dumps on hangs and TORCH_DIST_USE_NCCL2=0 or nccl-legacy as rollback paths |
One-sided transports for torch.distributed, with a NIXL backend and CUDA-graph-capturable transfers |
| FSDP2 and pipelining gain finer memory and communication control, with fp32 for precision-sensitive parameters, CUDA-graph-captured gradient accumulation, and gradient reduction overlapped with compute |
| Faster 4-bit (NVFP4) LLM decoding on NVIDIA Blackwell GPUs through fewer, better-overlapped kernel launches in NVGEMM |
Apple Silicon gains FP8 and faster decode, with float8_e4m3fn and _scaled_mm, a new grouped_mm with a decode fast path (1.32–1.34x tokens/s on MoE decode), and public torch.mps.Stream APIs |
Broader platform support with FlyDSL kernels on AMD gfx950, Tensor Data Mover support for gfx1250 (MI450), ROCm 10.0 wheels, simpler SYCL kernel authoring and multi-GPU profiling on Intel GPUs, CUDA 13.2 as the default wheel with CUDA 13.4 alongside it, and torch.compile on Python 3.15 |
For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.
Backwards Incompatible Changes
Python Frontend
import torch can no longer be re-executed in the same process, including via importlib.reload(torch) (#194654)
importlib.reload(torch) and del sys.modules["torch"] + re-import now raise, as does retrying import torch after it failed. Re-running the module body was already unsafe in exactly the same way as the retry, so the new check does not distinguish the two. Normal use is unaffected: a successful import leaves torch in sys.modules so the body never re-runs, and re-entrant imports from a backend extension during autoload still short-circuit on sys.modules without reaching the check.
Version 2.14:
import importlib
import torch
importlib.reload(torch)
# RuntimeError: Only a single TORCH_LIBRARY can be used to register the namespace triton; ...
# (re-importing after a failed `import torch` failed with similarly misleading errors or could segfault)Version 2.15:
import importlib
import torch
importlib.reload(torch)
# ImportError: `torch` can only be initialized once per process, so this module cannot
# be imported again: neither via `importlib.reload(torch)`, nor by re-importing after
# a failed `import torch`.Nested tensors are no longer allowlisted for torch.load(..., weights_only=True) (#194380)
Nested jagged and strided nested tensors are no longer on the default weights_only allowlist. Loading a checkpoint that contains one with weights_only=True (the default) now fails with Unsupported global: GLOBAL torch.nested._internal.nested_tensor._rebuild_njt was not an allowed global by default (or GLOBAL torch._utils._rebuild_nested_tensor for strided nested tensors). To load checkpoints containing nested tensors, allow them explicitly with torch.serialization.safe_globals (see below), or use weights_only=False for trusted files.
Version 2.14:
loaded = torch.load("njt.pt", weights_only=True) # worksVersion 2.15:
from torch.nested._internal.nested_tensor import _rebuild_njt, NestedTensor
with torch.serialization.safe_globals([_rebuild_njt, NestedTensor]):
loaded = torch.load("njt.pt", weights_only=True)Operators called through torch.ops.* and torch._C._dispatch_call_boxed now use PyTorch's own warning and exception translation instead of pybind11's (#193451)
We make torch.ops.* calls go through the same exception and warning translation as the rest of the library. This means that some exception translation that used to be handled by pybind is now handled natively. User-defined pybind exception translation is thus lost. Restoring it may be done in a future release as part of a broader exception handler improvement.
In addition, TORCH_WARN warnings from these operators now reach Python's warnings module, so they raise when warnings are turned into errors (-W error). std::out_of_range still becomes IndexError (#195713).
Version 2.14:
# C++ op: throw std::invalid_argument("bad size");
try:
torch.ops.my_lib.my_op(x)
except ValueError: # pybind11 translation
...Version 2.15:
# C++ op: TORCH_CHECK_VALUE(false, "bad size");
try:
torch.ops.my_lib.my_op(x)
except ValueError:
...
# TORCH_WARN from the op is now a Python warning
with warnings.catch_warnings():
warnings.simplefilter("ignore")
torch.ops.my_lib.my_op(x)Autograd
Selective activation checkpointing now saves, rather than recomputes, the outputs of operators with an ordered effect unless the policy returns CheckpointPolicy.MUST_RECOMPUTE (#196956)
In eager mode, if a create_selective_checkpoint_contexts policy returns CheckpointPolicy.PREFER_RECOMPUTE for a non-aliasing operator with an ordered effect (e.g. via register_effect(torch.library.EffectType.ORDERED)), that choice is now overridden and its outputs are saved, so the side effect is no longer replayed during backward. This can increase activation memory. To keep the 2.14 behavior, return CheckpointPolicy.MUST_RECOMPUTE for that operator. Operators in the c10d namespace are excluded. Under torch.compile, CheckpointPolicy.MUST_RECOMPUTE on an effectful operator now raises at compile time: RuntimeError: AOTAutograd does not support MUST_RECOMPUTE for effectful operations because forward effects cannot join the backward effect-token chain. Use a save policy instead.
policy = lambda ctx, op, *args, **kwargs: CheckpointPolicy.PREFER_RECOMPUTE
checkpoint(fn, x, use_reentrant=False, context_fn=lambda: create_selective_checkpoint_contexts(policy))
# 2.14: an effectful op in fn runs twice; 2.15: once (use CheckpointPolicy.MUST_RECOMPUTE to replay)Distributed (c10d)
The nccl backend now uses a rewritten, cleaned-up NCCL implementation by default (#192281, #192895, #193237, #193241)
backend="nccl" (also the default CUDA backend of init_process_group()) now creates the new implementation. The legacy implementation is available as backend="nccl-legacy" for this release only: nccl-legacy and TORCH_DIST_USE_NCCL2=0 will be removed in the next release, so migrate to nccl or nccl-lazy.
Use nccl (eager initialization) by default. Use nccl-lazy when you need independent point-to-point communicators: collectives run on one eagerly created communicator, and each send/recv peer pair gets its own lazily created communicator and stream, so concurrent P2P to different peers can overlap (as in the legacy backend).
Differences from the legacy backend:
- Communicators are created eagerly in
init_process_group()/new_group(), allocating GPU memory even for unused groups. - Each group is bound to one CUDA device:
device_id, elseLOCAL_RANK % torch.cuda.device_count()(or the global rank modulo the device count);torch.cuda.set_device()is not used. Tensors on another GPU fail withExpected tensor on cuda:X but found tensor on cuda:YorProcessGroupNCCL is bound to device cuda:X but an operation was issued on cuda:Y. Passdevice_idif needed; several GPUs per process requiresnccl-legacy. dist.get_backend_impl(...)returns the new backend class, soisinstance(..., ProcessGroupNCCL)isFalseand legacy-only methods such ascomm_split_count()are missing.- An async
all_gather'swork.get_future()value is the list of per-rank outputs, not a one-element list holding a stacked tensor; read the output list passed toall_gatherinstead (works on both backends).
To temporarily restore the legacy behavior in 2.15, set TORCH_DIST_USE_NCCL2=0 on every rank before creating the process group, or use nccl-legacy:
Version 2.14:
dist.init_process_group("nccl") # ProcessGroupNCCLVersion 2.15:
dist.init_process_group("nccl", device_id=torch.device("cuda", local_rank)) # new implementation
dist.init_process_group("nccl-lazy", device_id=torch.device("cuda", local_rank)) # independent P2P communicators
dist.init_process_group("nccl-legacy") # legacy implementation, removed in the next releaseDistributed Checkpointing
Distributed checkpoint metadata now records tensor strides, so checkpoints saved with PyTorch 2.15 cannot be loaded by PyTorch 2.14 or earlier (#194251)
TensorProperties gains an optional strides field in the pickled .metadata file. In older versions dcp.load raises a CheckpointException reporting too many values to unpack (expected 5). There is no flag to write the old format.
Version 2.14 (loading a 2.15 checkpoint):
dcp.load(state_dict, checkpoint_id="/ckpt/step_100")
# CheckpointException: ... too many values to unpack (expected 5)Version 2.15:
dcp.load(state_dict, checkpoint_id="/ckpt/step_100") # loads 2.14 and 2.15 checkpointsDistributed FSDP2
With MixedPrecisionPolicy(reduce_dtype=None), fully_shard now accumulates and reduces gradients in the parameter's grad_dtype (its original dtype by default) instead of in param_dtype (#194434)
This is BC-breaking: mp_policy.reduce_dtype=None no longer ...
PyTorch 2.14.1 Release
This release is meant to fix the following regressions and silent correctness issues:
Silent correctness fixes
- Fix incorrect
torch.linalg.lstsqsolutions on MPS for complex batched underdetermined systems (#196113), fixed by #196128 - Fix non-orthogonal
Uand inaccurate small singular values fromtorch.linalg.svdon MPS for rank-deficient and ill-conditioned inputs (#196112), fixed by #196139 and #199063 - Update the CUDA 13.2 Linux binaries to CUDA 13.2.2 (#196351). This NVIDIA update resolves two critical issues that could produce incorrect results (CUDA 13.2.2 release notes):
- cuBLAS:
cublasLtMatmul()could ignore tensor-wide scaling for NVFP4 matrix multiplications (introduced in CUDA 13.2 Update 1) - Compiler: failed thread reconvergence could leave stale or corrupted register values in kernels with nested thread divergence (present since CUDA 12.8)
- cuBLAS:
Regression fixes
PyTorch 2.14.0 Release
PyTorch 2.14.0 Release Notes
- Highlights
- Backwards Incompatible Changes
- Deprecations
- New Features
- Improvements
- Bug fixes
- Performance
- Documentation
- Security
- Developers
Highlights
| NVGEMM brings CuTeDSL-generated CUTLASS kernels to Inductor, with epilogue fusion, scaled and NVFP4 GEMM, and grouped-reduction epilogues autotuned alongside Triton and ATen |
torch.switch generalizes torch.cond to multi-way branching, and torch.while_loop can now be captured in a CUDA graph |
Declarative dynamic shapes via @dynamic_spec, shared across torch.compile, torch.export and make_fx |
Experimental torch.compile support for complex-valued tensors: Opt-in support decomposes supported complex operations into real and imaginary computations, enabling compiler backends to optimize more complex-number workloads. |
| A preview of our rewritten NCCL backend for PyTorch, ported from torchcomms, implementing the full collective contract with nonblocking communicators and eager communicator splitting and advanced features such as fault tolerance and windows designed as a drop-in replacement of existing NCCL c10d backend |
Fault tolerance becomes a first-class c10d concept, with in-place process-group reconfiguration, one-sided RMA windows, and a Flight Recorder that works for any backend rather than only NCCL |
Apple Silicon gains native linear algebra, including Jacobi-kernel SVD, eigh, QR and Cholesky, alongside a five-part reduction rewrite and a further MPSGraph to Metal kernel migration |
Broader platform support: ROCm 7.14 wheels are produced from the TheRock pip SDK, Intel XPU adds native graph capture, and Inductor targets Rubin (sm_107) |
For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.
Backwards Incompatible Changes
torch.nn
-
torch.nn.LinearCrossEntropyOptionsno longer acceptsacc_policy="balanced"; use"compact"instead (#188283)The
"balanced"policy was removed because"compact"provides the same weight-gradient accumulation precision with lower memory use on CUDA, already uses the equivalent scratch layout for mixed-precision inputs on other devices, and was never selected by"auto". Constructing the options withacc_policy="balanced"now raisesValueError: invalid acc_policy: 'balanced'; expected one of 'auto', 'accurate', 'compact'.Before:
options = torch.nn.LinearCrossEntropyOptions(acc_policy="balanced") loss = torch.nn.functional.linear_cross_entropy( input, linear_weight, target, options=options )
After:
options = torch.nn.LinearCrossEntropyOptions(acc_policy="compact") loss = torch.nn.functional.linear_cross_entropy( input, linear_weight, target, options=options )
Autograd
-
Clamp and min/max boundary subgradients now follow the selected dispatcher schema's input space (#191142)
This affects gradients exactly at nondifferentiable bounds or ties. A scalar clamp bound is a fixed parameter, so the input gradient at equality changes from
1to the minimum-norm subgradient0. A Tensor bound is part of the differentiable input space, soclamp,clamp_min, andclamp_maxnow split the gradient evenly between the input and bound at an ordinary tie instead of assigning it entirely to the input.fminandfmaxuse the same even tie split, and forward-mode AD for the min/max family is aligned with these rules. Code that intentionally depends on the old tie-breaking behavior can express it explicitly withtorch.where, such astorch.where(value >= bound, value, bound).Version 2.13:
import torch x = torch.tensor(0.0, requires_grad=True) torch.clamp_min(x, 0.0).backward() print(x.grad) # tensor(1.) value = torch.tensor(0.0, requires_grad=True) bound = torch.tensor(0.0, requires_grad=True) torch.clamp_min(value, bound).backward() print(value.grad, bound.grad) # tensor(1.) tensor(0.)
Version 2.14:
import torch x = torch.tensor(0.0, requires_grad=True) torch.clamp_min(x, 0.0).backward() print(x.grad) # tensor(0.) value = torch.tensor(0.0, requires_grad=True) bound = torch.tensor(0.0, requires_grad=True) torch.clamp_min(value, bound).backward() print(value.grad, bound.grad) # tensor(0.5000) tensor(0.5000)
Distributed
-
Custom Python process groups that implement
new_group()must now accept abackendkeyword argument (#188489)This applies when the default process group supplies its own
new_group()method andtorch.distributed.new_group()delegates subgroup creation to it. PyTorch now forwards the resolved backend so custom implementations can construct the requested subgroup correctly. Existing implementations without this parameter will raiseTypeError: ... got an unexpected keyword argument 'backend'. Accept and use the argument, or accept and ignore it when the implementation has only one backend.Before:
class MyProcessGroup(...): def new_group( self, ranks, *, timeout=None, pg_options=None, group_name=None, group_desc=None ): ...
After:
class MyProcessGroup(...): def new_group( self, ranks, *, timeout=None, backend=None, pg_options=None, group_name=None, group_desc=None ): ...
-
NCCL symmetric-memory pools no longer automatically upgrade segments allocated after
register_mem_pool(..., symm=True)to symmetric windows (#192112)Registering those late segments from the CUDA allocator callback could invoke a collective NCCL operation on only some ranks while holding the allocator lock, causing an unrecoverable hang. Late segments now remain ordinary registered NCCL buffers. Applications that need newly allocated segments to use symmetric-window algorithms must collectively deregister and register the pool again after those allocations are created.
Before:
backend.register_mem_pool(pool, symm=True) with torch.cuda.use_mem_pool(pool): tensor = torch.empty(size, device="cuda") # Newly allocated segments were automatically upgraded, but this could hang.
After:
backend.register_mem_pool(pool, symm=True) with torch.cuda.use_mem_pool(pool): tensor = torch.empty(size, device="cuda") # Collectively refresh registration after the pool grows. backend.deregister_mem_pool(pool) backend.register_mem_pool(pool, symm=True)
-
Nonmember ranks now receive
GroupMember.NON_GROUP_MEMBERinstead ofNonefrom experimentaltorch.distributed.split_group()(#190725)When the calling rank is absent from every requested split,
split_group()now returns the same nonmember sentinel asnew_group(). Code that identifies nonmembers withis Nonemust compare againsttorch.distributed.GroupMember.NON_GROUP_MEMBERinstead.Before:
group = torch.distributed.split_group( split_ranks=[[0, 1], [2, 3]] ) if group is None: return
After:
group = torch.distributed.split_group( split_ranks=[[0, 1], [2, 3]] ) if group == torch.distributed.GroupMember.NON_GROUP_MEMBER: return
Linear Algebra Frontend
-
Remove the deprecated
torch.cholesky()andTensor.cholesky()APIs (#186817)Calls now raise a
RuntimeErrordirecting users totorch.linalg.cholesky(). The replacement returns a lower-triangular factor; callers that previously requestedupper=Trueshould take the conjugate transpose with.mH.Version 2.13:
lower = torch.cholesky(a) upper = torch.cholesky(a, upper=True)
Version 2.14:
lower = torch.linalg.cholesky(a) upper = torch.linalg.cholesky(a).mH
-
Remove the deprecated
torch.qr()andTensor.qr()APIs (#186815)Calls now raise a
RuntimeErrordirecting users totorch.linalg.qr(). Replace the Booleansomeargument withmode="reduced"ormode="complete".Version 2.13:
q, r = torch.qr(a) q_full, r_full = torch.qr(a, some=False)
Version 2.14:
q, r = torch.linalg.qr(a, mode="reduced") q_full, r_full = torch.linalg.qr(a, mode="complete")
Profiler
-
The deprecated
use_cudaargument has been removed fromtorch.profiler.profileandtorch.autograd.profiler.profile(#192543)Passing
use_cudato either profiler now raisesTypeError: profile.__init__() got an unexpected keyword argument 'use_cuda'. Select CUDA explicitly throughactivitieswhen usingtorch.profiler.profile, or useuse_device="cuda"withtorch.autograd.profiler.profile.Version 2.13:
with torch.profiler.profile(use_cuda=True) as prof: run_workload() with torch.autograd.profiler.profile(use_cuda=True) as prof: run_workload()
Version 2.14:
with torch.profiler.profile( activities=[ torch.profiler.ProfilerActivity.CPU, torch.profiler.ProfilerActivity.CUDA, ] ) as prof: run_workload() with torch.autograd...
PyTorch 2.13.0 Release
PyTorch 2.13.0 Release Notes
- Highlights
- Backwards Incompatible Changes
- Deprecations
- New Features
- Improvements
- Bug fixes
- Performance
- Documentation
- Developers
Highlights
| FlexAttention lands on Apple Silicon (MPS), with up to ~12x speedup over SDPA on sparse patterns, and gains a deterministic backward path on CUDA for reproducible gradient computation. |
| CuTeDSL "Native DSL" backend gives Inductor a second high-performance code path (alongside Triton) for key GPU operations, with faster compilation. [Prototype] |
nn.LinearCrossEntropyLoss combines the final prediction and loss computation to cut peak GPU memory by up to 4x for large-vocabulary language model training. |
| torchcomms, a new communications backend for PyTorch Distributed, improves fault tolerance, scalability, and debuggability for large-cluster training. |
| FSDP2 now overlaps reduce-scatter and all-gather communications via a dedicated process group (opt-in), increasing distributed training throughput. |
| Python 3.15 wheel support for PyTorch on Linux via the pytorch repository index, including builds compatible with free-threaded 3.15t. |
Broader platform support: ROCm gains AOTriton 0.12b with native HIP CMake, Arm adds Armv9-A torch.compile targeting, and Intel XPU exposes new device telemetry APIs. |
For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.
Tracked Regressions
ROCm wheels break torch.compile on CPU in environments without a GPU
Running a torch==2.13.0+rocm7.2 wheel in an environment where no GPU is available (torch.cuda.is_available() is False) breaks torch.compile on the CPU path: the first compile raises RuntimeError: Can't detect vectorized ISA for CPU (#189194). This is a regression from torch==2.12.1+rocm7.2, which compiles CPU code fine (detecting e.g. VecAVX2) in the same setup. The 2.13 ROCm wheel appears to rely on something present in the ROCm builder image to detect the CPU vectorized ISA, so it works when run on a ROCm image but fails on a plain CPU-only image.
Workaround: run the +rocm wheel on a ROCm image, or install a standard CPU/CUDA build for GPU-less environments.
Backwards Incompatible Changes
-
Stop building CPython 3.13t (free-threaded) binaries (#182951)
Upstream
pypa/manylinuxremoved CPython 3.13t (free-threaded) on 2026-05-07, because 3.13t
was experimental and has been superseded by the now-non-experimental CPython 3.14t. As a result,
PyTorch 2.13 no longer shipscp313twheels (Linux, Triton, and related artifacts). Users on the
free-threaded interpreter should move to Python 3.14t.PyTorch 2.12:
# cp313t (free-threaded 3.13) wheels were available python3.13t -m pip install torchPyTorch 2.13:
# Use free-threaded Python 3.14t instead python3.14t -m pip install torch -
Bare
PyObjectis no longer allowed in operator schemas (#184209)Bare
PyObjectwas accidentally accepted in operator schema strings in
PyTorch 2.12. This was undocumented and is now rejected, sincetorch.compile
does not support arbitraryPyObjectinputs to custom ops. If
you parse or register a schema with a barePyObjectargument or return type,
you will now get a schema parse error.PyTorch 2.12:
>>> from torch._C import parse_schema >>> parse_schema("foo(PyObject x) -> ()") # accepted
PyTorch 2.13:
>>> from torch._C import parse_schema >>> parse_schema("foo(PyObject x) -> ()") # raises a schema parse error
-
Remove Bazel build support (#180883)
The Bazel build was never broadly adopted and still depended on the antiquated Bazel 6,
while the wider ecosystem has since moved to Bazel 9. All Bazel build files and CI jobs have
been removed. Users building PyTorch with Bazel should migrate to the supported CMake/pip install
build flow.PyTorch 2.12:
# Build PyTorch with Bazel bazel build //:torchPyTorch 2.13:
# Bazel build files have been removed; build from source with pip instead pip install --no-build-isolation -e .
-
StorageImpl's built-in copy-on-write (COW) materialization is replaced by a pluggable materializer hook (#179063)StorageImplno longer knows about COW directly. Its internal COW entry points
StorageImpl::is_cow(),StorageImpl::maybe_materialize_cow(), and the friend
cow::materialize_cow_storage()have been removed in favor of a single pluggable
MaterializeFnhook (void(*)(StorageImpl*)) that a backend registers to run once,
on the first mutable data-pointer access. COW is now just one consumer of this hook
(c10::impl::cow::materialize_cow), and all COW behavior (lazy clone, refcounted
shared data, copy-on-write) is unchanged. This also gives accelerator backends and
eager-mode graph compilers a zero-fast-path-cost place to commit deferred allocations
or materialize symbolic buffers on first mutation.This is a C++-only change. It affects out-of-tree backends/extensions that called the
removedStorageImplCOW symbols directly; they will fail to compile against 2.13
with errors such asno member named 'is_cow' in 'c10::StorageImpl'. Migrate to the
new hook API (set_materializer()/has_materializer()/clear_materializer()).PyTorch 2.12:
// Detect a COW storage and force it to materialize. if (storage.is_cow()) { storage.maybe_materialize_cow(); }
PyTorch 2.13:
// Register a one-shot materializer; it runs on the next mutable-data access // and then clears itself. COW registers c10::impl::cow::materialize_cow this way. storage.set_materializer(&my_backend_materialize); // void(StorageImpl*) // `has_materializer()` replaces `is_cow()` for "is a deferred materialization pending?" if (storage.has_materializer()) { /* ... */ }
-
Convert
shared_ptr<Node>tointrusive_ptr<Node>(#181139). This changes the signature ofTensor::grad_fn. Accesses toTensor.grad_fn()should change fromstd::shared_ptr<Node>toc10::intrusive_ptr<Node>. Similarly, construction of a C++ autograd function should change:PyTorch 2.12:
std::shared_ptr<CustomCppNode> node(new CustomCppNode(), torch::autograd::deleteNode);
PyTorch 2.13:
auto node = c10::make_intrusive<CustomCppNode>(); -
The minimum supported NCCL version when building from source is now 2.23 (#186292)
PyTorch now requires NCCL >= 2.23 at compile time, and the preprocessor/runtime gates that guarded NCCL features introduced in 2.23 or earlier have been removed. Users who build PyTorch from source against a system NCCL older than 2.23 will hit compile errors against the dropped gates. Upgrade the NCCL installation to >= 2.23 to build. The prebuilt PyTorch wheels already bundle a compatible NCCL, so pip/conda users are unaffected.
-
Remove named tensors (#173895)
The named tensor feature (a long-deprecated prototype) has been fully removed to reduce overhead and code bloat. All associated Python and C++ APIs are gone, including
Tensor.names,Tensor.rename(),Tensor.refine_names(),Tensor.align_to(),Tensor.align_as(),torch.align_tensors(), thenames=keyword on factory functions (e.g.torch.zeros,torch.empty,torch.ones), and the C++Dimname/DimnameListAPIs. Code that previously relied on named dimensions must track dimension order positionally and avoid usage of any of these now-removed APIs or op overloads. -
The
onednn::qconv2d_pointwise.binaryand.binary_tensoroperators no longer alias their input but rather return fresh tensors. Previously these ops mutated theqaccuminput buffer and returned it directly, violating the PyTorch invariant that custom operator outputs must not alias inputs. This silently bypassed aliasing checks via the old-> Tensor(a!)schema and would become a hard error in a future PyTorch version (as mentioned in #182063), so the schema and implementation were corrected to return a fresh output. Most users are unaffected, only code that calls these ops directly and relies on the in-place mutation of qaccum must now read the returned tensor instead. (#177171)
Deprecations
-
Custom operators that return an output aliasing one of their inputs are deprecated (#182063)
When a custom operator returns an output that is the same tensor as (or otherwise aliases) one of its inputs under
torch.compile, PyTorch now emits aUserWarningstating that this is deprecated and will become an error in a future version of PyTorch. Previously the warning stated the change would land in PyTorch 2.12; that timeline has been pushed back. To update your code, return a clone of the offending output instead of the input, or refactor the operator so it does not return the aliased tensor.Deprecated:
@torch.library.custom_op("mylib::foo", mutates_args=()) def foo(x: torch.Tensor) -> torch.Tensor: return x # output aliases the input -- deprecated
Updated:
@torch.library.custom_op("mylib::foo", mutates_args=()) def foo(x: torch.Tensor) -> torch.Tensor: return x.clone() # return a clone instead
-
Creating tensors with the qua...
PyTorch 2.12.1 Release, bug fix release
This release is meant to fix the following regressions and silent correctness issues:
Regression fixes
- Fix nondeterministic outputs in test_batch_invariance with FLASH_ATTN on NVIDIA B200 GPUs (#181248), fixed by updating Triton to 3.7.1 (#186814)
- Fix illegal memory access in the Triton convolution2d_bwd_weight kernel on B100/B200 (sm100) GPUs (#187081), fixed by updating Triton to 3.7.1 (#186814)
- Fix fill_ on byte-dtype views with misaligned storage offset (#186821)
Releng / Build
- Drop CPython 3.13t from the binary build matrix (#182951)
PyTorch 2.12.0 Release
PyTorch 2.12.0 Release Notes
- Highlights
- Backwards Incompatible Changes
- Deprecations
- New Features
- Improvements
- Bug fixes
- Performance
- Documentation
- Developers
- Security
Highlights
| Batched linalg.eigh on CUDA is up to 100x faster due to updated cuSolver backend selection. |
| New torch.accelerator.Graph API unifies graph capture and replay across CUDA, XPU, and out-of-tree backends. |
| torch.export.save now supports Microscaling (MX) quantization formats, enabling full export of aggressively compressed models. |
Adagrad now supports fused=True, joining Adam, AdamW, and SGD with a single-kernel optimizer implementation. |
| torch.cond control flow can now be captured and replayed inside CUDA Graphs. |
| ROCm users gain expandable memory segments, rocSHMEM symmetric memory collectives, and FlexAttention pipelining. |
For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.
Backwards Incompatible Changes
Build Frontend
-
Strengthened SVE compile checks in
FindARM.cmake, which may reject previously accepted but incorrect SVE configurations (#176646)Source builds that enable SVE now validate the compiler configuration more strictly. If a build previously passed with an incomplete or mismatched SVE setup, it may now fail during CMake configuration instead of later in compilation. Update the compiler/toolchain flags so they accurately describe the target SVE support, or disable SVE for that build.
-
Updated the minimum CUDA version required to build PyTorch from source to CUDA 12.6 (#178925)
Building PyTorch from source with CUDA versions older than 12.6 is no longer supported. Users building custom binaries should install CUDA 12.6 or newer and make sure
CUDA_HOMEpoints to that installation.Version 2.11:
CUDA_HOME=/usr/local/cuda-12.4 python setup.py develop
Version 2.12:
CUDA_HOME=/usr/local/cuda-12.6 python setup.py develop
-
Enforced a C++20 minimum in CMake build files (#178662)
Source builds now require a compiler and build configuration that support C++20. If you maintain custom build scripts or downstream extensions that build PyTorch from source, update the compiler and remove assumptions that PyTorch can be built as C++17.
Distributed
-
torch.distributed.nn.functionalops now raiseRuntimeErrorundertorch.compile(#177342)All ops in
torch.distributed.nn.functional(e.g.,broadcast,all_reduce,all_gather,reduce_scatter,all_to_all_single) now raiseRuntimeErrorwhen called insidetorch.compile. Users should migrate to the functional collectives API intorch.distributed._functional_collectives.Version 2.11:
@torch.compile def my_func(x): return torch.distributed.nn.functional.all_reduce(x, op=ReduceOp.SUM)
Version 2.12:
@torch.compile def my_func(x): return torch.distributed._functional_collectives.all_reduce(x, reduceOp="sum", group=group)
TorchElastic
-
torchrunnow defaults to an OS-assigned free port for single-node training instead of port 29500 (#175699)When running
torchrun --nproc-per-node=N script.pywithout specifying--master-portor--standalone, the default behavior now automatically uses an OS-assigned free port via thec10drendezvous backend. This eliminates "Address already in use" errors when running multiple training jobs concurrently. Multi-node training, explicit--master-port,PET_MASTER_PORTenv var, and--standaloneare unchanged.Version 2.11:
# Used static rendezvous on port 29500 by default torchrun --nproc-per-node=4 train.pyVersion 2.12:
# Uses OS-assigned free port by default torchrun --nproc-per-node=4 train.py # To explicitly use a fixed port: torchrun --nproc-per-node=4 --master-port=29500 train.py
MPS
-
All MPS tensors are now allocated in unified memory (#175818)
Previously, MPS tensors could be allocated in either device-only or unified memory. Now all MPS tensors use unified memory unconditionally. This simplifies memory management and enables CPU access to MPS tensor data without explicit copies. Code that relied on device-only memory placement may observe different performance characteristics.
Inductor
-
The
max_autotunelayout-constraint deferral introduced in 2.11 is now opt-in (#175330)In 2.11, Inductor deferred layout freezing for
max_autotunetemplates to expose more fusion opportunities. This caused a regional-inductor failure mode, so the default in 2.12 reverts to immediate layout freezing. Users who relied on the deferred behavior for fusion opportunities should opt in explicitly viatorch._inductor.config.max_autotune_defer_layout_freezingorTORCHINDUCTOR_MAX_AUTOTUNE_DEFER_LAYOUT_FREEZING=1.Version 2.11:
# Deferred layout freezing was the default torch.compile(model, mode="max-autotune")
Version 2.12:
import torch._inductor.config as cfg cfg.max_autotune_defer_layout_freezing = True # or set TORCHINDUCTOR_MAX_AUTOTUNE_DEFER_LAYOUT_FREEZING=1 torch.compile(model, mode="max-autotune")
Deprecations
Release Engineering
-
Deprecate CUDA 12.8 builds in favor of CUDA 13.0 (#179072)
CUDA 12.8 binaries have been removed from the PyTorch binary build matrix. CUDA 13.0 is now the stable default and CUDA 12.6 remains available for users on older drivers. Users explicitly pinning the
cu128index URL will need to switch tocu130(recommended) orcu126.Version 2.11:
pip install torch --index-url https://download.pytorch.org/whl/cu128
Version 2.12:
# Use CUDA 13.0 (default on PyPI): pip install torch # Or explicitly: pip install torch --index-url https://download.pytorch.org/whl/cu130 # Older driver fallback: pip install torch --index-url https://download.pytorch.org/whl/cu126
-
Compatibility with CMake < 3.10 will be removed in a future release (#166259)
Source builds against CMake versions older than 3.10 now emit a deprecation warning. A future release will require CMake 3.10 or newer; please upgrade CMake before then.
Linear Algebra
-
Several CUDA linear algebra operators no longer use the MAGMA backend and now dispatch to cuSolver or cuBLAS unconditionally:
torch.linalg.eighnow dispatches to cuSolver (#174619)torch.linalg.lu_solvenow dispatches to cuSolver/cuBLAS (#174248)torch.linalg.cholesky_inversenow dispatches to cuSolver (#174681)torch.linalg.cholesky_solvenow dispatches to cuSolver (#174769)
User code calling these APIs does not need to change. The practical impact is for users who depended on MAGMA-specific numerical behavior, performance characteristics, or debugging. Those calls now use the cuSolver/cuBLAS implementations on CUDA.
FullyShardedDataParallel2 (FSDP2)
-
Compiling through FSDP2 hooks without graph breaks is no longer supported (#174863, #174906). If you use compiled autograd with FSDP2, update your code to allow graph breaks around FSDP2 hooks or disable compiled autograd for the FSDP2 training step.
Version 2.11:
with torch._dynamo.config.patch(compiled_autograd=True): compiled_model = torch.compile(fsdp_model, fullgraph=True) loss = compiled_model(input).sum() loss.backward()
Version 2.12:
# Either run FSDP2 backward without fullgraph. compiled_model = torch.compile(fsdp_model, fullgraph=False) loss = compiled_model(input).sum() loss.backward() # Or apply compile before applying FSDP. compiled_model_pre_fsdp = torch.compile(model, fullgraph=True) compiled_model = fully_shard(compiled_model_pre_fsdp, ...) loss = compiled_model(input).sum() loss.backward()
Profiler
-
Profiler's
metadata_jsonfield is now deprecated; useevent_metadatainstead (#179417)Version 2.11:
metadata = event.metadata_json
Version 2.12:
metadata = event.event_metadata
Dynamo
-
torch.compile(fullgraph=True)now warns when a call runs no compiled code; will error in 2.13 (#181940)Previously
fullgraph=Truewas only validated once Dynamo actually compiled and ran the function. If Dynamo was bypassed at call time (e.g. under a user-definedTorchDispatchMode), the annotation silently had no effect. 2.12 emits a warning; 2.13 will raise. For graph-break erro...
PyTorch 2.11.0 Release
PyTorch 2.11.0 Release Notes
- Highlights
- Backwards Incompatible Changes
- Deprecations
- New Features
- Improvements
- Bug fixes
- Performance
- Documentation
- Developers
- Security
Highlights
| Added Support for Differentiable Collectives for Distributed Training |
| FlexAttention now has a FlashAttention-4 backend on Hopper and Blackwell GPUs |
| MPS (Apple Silicon) Comprehensive Operator Expansion |
| Added RNN/LSTM GPU Export Support |
| Added XPU Graph Support |
For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.
Backwards Incompatible Changes
Release Engineering
Volta (SM 7.0) GPU support removed from CUDA 12.8 and 12.9 binary builds (#172598)
Starting with PyTorch 2.11, the CUDA 12.8 and 12.9 pre-built binaries no longer include support for Volta GPUs (compute capability 7.0, e.g. V100). This change was necessary to enable updating to CuDNN 9.15.1, which is incompatible with Volta.
Users with Volta GPUs who need CUDA 12.8+ should use the CUDA 12.6 builds, which continue to include Volta support. Alternatively, build PyTorch from source with Volta included in TORCH_CUDA_ARCH_LIST.
Version 2.10:
# CUDA 12.8 builds supported Volta (SM 7.0)
pip install torch --index-url https://download.pytorch.org/whl/cu128
# Works on V100
Version 2.11:
# CUDA 12.8 builds no longer support Volta
# For V100 users, use CUDA 12.6 builds instead:
pip install torch --index-url https://download.pytorch.org/whl/cu126
PyPI wheels now ship with CUDA 13.0 instead of CUDA 12.x (#172663, announcement)
Starting with PyTorch 2.11, pip install torch on PyPI installs CUDA 13.0 wheels by default for both Linux x86_64 and Linux aarch64. Previously, PyPI wheels shipped with CUDA 12.x and only Linux x86_64 CUDA wheels were available on PyPI. Users whose systems have only CUDA 12.x drivers installed may encounter errors when running pip install torch without specifying an index URL.
Additionally, CUDA 13.0 only supports Turing (SM 7.5) and newer GPU architectures on Linux x86_64. Maxwell and Pascal GPUs are no longer supported under CUDA 13.0. Users with these older GPUs should use the CUDA 12.6 builds instead.
CUDA 12.6 and 12.8 binaries remain available via download.pytorch.org.
Version 2.10:
# PyPI wheel used CUDA 12.x
pip install torchVersion 2.11:
# PyPI wheel now uses CUDA 13.0
pip install torch
# To get CUDA 12.8 wheels instead:
pip install torch --index-url https://download.pytorch.org/whl/cu128
# To get CUDA 12.6 wheels (includes Maxwell/Pascal/Volta support):
pip install torch --index-url https://download.pytorch.org/whl/cu126Python Frontend
torch.hub.list(), torch.hub.load(), and torch.hub.help() now default the trust_repo parameter to "check" instead of None. The trust_repo=None option has been removed. (#174101)
Previously, passing trust_repo=None (or relying on the default) would silently download and run code from untrusted repositories with only a warning. Now, the default "check" behavior will prompt the user for explicit confirmation before running code from repositories not on the trusted list.
Users who were explicitly passing trust_repo=None must update their code. Users who were already passing trust_repo=True, trust_repo=False, or trust_repo="check" are not affected.
Version 2.10:
# Default trust_repo=None — downloads with a warning
torch.hub.load("user/repo", "model")
# Explicit None — same behavior
torch.hub.load("user/repo", "model", trust_repo=None)Version 2.11:
# Default trust_repo="check" — prompts for confirmation if repo is not trusted
torch.hub.load("user/repo", "model")
# To skip the prompt, explicitly trust the repo
torch.hub.load("user/repo", "model", trust_repo=True)torch.nn
Add sliding window support to varlen_attn via window_size, making optional arguments keyword-only (#172238)
The signature of torch.nn.attention.varlen_attn has changed: a * (keyword-only separator) has been inserted before the optional arguments. Previously, optional arguments like is_causal, return_aux, and scale could be passed positionally; they must now be passed as keyword arguments. A new window_size keyword argument has also been added.
# Before (2.10)
output = varlen_attn(query, key, value, cu_seq_q, cu_seq_k, max_q, max_k, True, None, 1.0)
# After (2.11) — pass as keyword argument
output = varlen_attn(query, key, value, cu_seq_q, cu_seq_k, max_q, max_k, window_size=(-1, 0), return_aux=None, scale=1.0)Remove is_causal flag from varlen_attn (#172245)
The is_causal parameter has been removed from torch.nn.attention.varlen_attn. Causal attention is now expressed through the window_size parameter: use window_size=(-1, 0) for causal masking, or window_size=(W, 0) for causal attention with a sliding window of size W. The default window_size=(-1, -1) corresponds to full (non-causal) attention.
# Before (2.10)
output = varlen_attn(query, key, value, cu_seq_q, cu_seq_k, max_q, max_k, is_causal=True)
# After (2.11) — use window_size instead
output = varlen_attn(query, key, value, cu_seq_q, cu_seq_k, max_q, max_k, window_size=(-1, 0))Distributed
DebugInfoWriter now honors $XDG_CACHE_HOME for its cache directory in C++ code, consistent with the Python side. Previously it always used ~/.cache/torch. (#168232)
This avoids issues where $HOME is not set or not writable. Users who relied on ~/.cache/torch being used regardless of $XDG_CACHE_HOME may see debug info written to a different location.
Version 2.10:
# C++ DebugInfoWriter always wrote to ~/.cache/torch
Version 2.11:
# C++ DebugInfoWriter now respects $XDG_CACHE_HOME/torch (same as Python code)
# Falls back to ~/.cache/torch if $XDG_CACHE_HOME is not set
DeviceMesh now stores a process group registry (_pg_registry) directly, enabling torch.compile to trace through get_group(). (#172272)
This may break code that skips init_process_group, loads a saved DTensor (constructing a DeviceMesh with no PGs), and later creates PGs separately — during torch.compile runtime the PG lookup will fail. Users should ensure process groups are initialized before constructing the DeviceMesh.
Version 2.10:
# PGs resolved via global _resolve_process_group at runtime
mesh = DeviceMesh(...) # PGs could be created laterVersion 2.11:
# PGs now stored on DeviceMesh._pg_registry; must exist at mesh creation
dist.init_process_group(...) # Must be called before creating mesh
mesh = DeviceMesh(...)Distributed (DTensor)
DTensor.to_local() backward now converts Partial placements to Replicate by default when grad_placements is not provided. (#173454)
Previously, calling to_local() on a Partial DTensor would preserve the Partial placement in the backward gradient, which could produce incorrect gradients when combined with from_local(). Now, the backward pass automatically maps Partial forward placements to Replicate gradient placements, matching the behavior of from_local().
Users who relied on the previous behavior (where to_local() backward preserved Partial gradients) may see different gradient values. To ensure correctness, explicitly pass grad_placements to to_local().
Version 2.10:
# Partial placement preserved in backward — could produce incorrect gradients
local_tensor = partial_dtensor.to_local()Version 2.11:
# Partial → Replicate in backward by default (correct behavior)
local_tensor = partial_dtensor.to_local()
# Or explicitly specify grad_placements for full control:
local_tensor = partial_dtensor.to_local(grad_placements=[Replicate()])_PhiloxState.seed and _PhiloxState.offset now return torch.Tensor instead of int (#173876)
The DTensor RNG internal _PhiloxState class changed its seed and offset properties to return tensors instead of Python ints, and the setters now expect tensors. This makes the RNG state compatible with PT2 tracing (the previous .item() calls were not fake-tensor friendly).
Code that directly reads _PhiloxState.seed or _PhiloxState.offset and treats them as ints will break. Call .item() to get the int value. When setting, wrap the value in a tensor.
Version 2.10:
from torch.distributed.tensor._random import _PhiloxState
philox = _PhiloxState(state)
seed: int = philox.seed # returned int
philox.offset = 42 # accepted intVersion 2.11:
from torch.distributed.tensor._random import _PhiloxState
philox = _PhiloxState(state)
seed: int = philox.seed.item() # now returns Tens...PyTorch 2.10.0 Release
PyTorch 2.10.0 Release Notes
- Highlights
- Backwards Incompatible Changes
- Deprecations
- New Features
- Improvements
- Bug fixes
- Performance
- Documentation
- Developers
- Security
Highlights
Python 3.14 support for torch.compile(). Python 3.14t (freethreaded build) is experimentally supported as well.
|
| Reduced kernel launch overhead with combo-kernels horizontal fusion in torchinductor |
| A new varlen_attn() op providing support for ragged and packed sequences |
| Efficient eigenvalue decompositions with DnXgeev |
torch.compile() now respects use_deterministic_mode |
| DebugMode for tracking dispatched calls and debugging numerical divergence - This makes it simpler to track down subtle numerical bugs. |
| Intel GPUs support: Expand PyTorch support to the latest Panther Lake on Windows and Linux by enabling FP8 (core ops and scaled matmul) and complex MatMul support, and extending SYCL support in the C++ Extension API for Windows custom ops. |
For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.
Backwards Incompatible Changes
Dataloader Frontend
- Removed unused
data_sourceargument from Sampler (#163134). This is a no-op, unless you have a custom sampler that uses this argument. Please update your custom sampler accordingly. - Removed deprecated imports for torch.utils.data.datapipes.iter.grouping (#163438).
from torch.utils.data.datapipes.iter.grouping import SHARDING_PRIORITIES, ShardingFilterIterDataPipeis no longer supported. Please import fromtorch.utils.data.datapipes.iter.shardinginstead.
torch.nn
- Remove Nested Jagged Tensor support from
nn.attention.flex_attention(#161734)
ONNX
fallback=Falseis now the default intorch.onnx.export(#162726)- The exporter now uses the
dynamo=Trueoption without fallback. This is the recommended way to use the ONNX exporter. To preserve 2.9 behavior, manually setfallback=Truein thetorch.onnx.exportcall.
Release Engineering
- Rename pytorch-triton package to triton (#169888)
Deprecations
Distributed
- DeviceMesh
- Added a warning for slicing flattened dim from root mesh and types for _get_slice_mesh_layout (#164993)
We decided to deprecate an existing behavior which goes against the PyTorch design principle (explicit over implicit) for device mesh slicing of flattened dim.
Version <2.9
import torch
from torch.distributed.device_mesh import
device_type = (
acc.type
if (acc := torch.accelerator.current_accelerator(check_available=True))
else "cpu"
)
mesh_shape = (2, 2, 2)
mesh_3d = init_device_mesh(
device_type, mesh_shape, mesh_dim_names=("dp", "cp", "tp")
)
mesh_3d["dp", "cp"]._flatten()
mesh_3["dp_cp"] # This comes with no warningVersion >=2.10
import torch
from torch.distributed.device_mesh import
device_type = (
acc.type
if (acc := torch.accelerator.current_accelerator(check_available=True))
else "cpu"
)
mesh_shape = (2, 2, 2)
mesh_3d = init_device_mesh(
device_type, mesh_shape, mesh_dim_names=("dp", "cp", "tp")
)
mesh_3d["dp", "cp"]._flatten()
mesh_3["dp_cp"] # This will come with a warning because it implicitly change the state of the original mesh. We will eventually remove this behavior in future release. User should do the bookkeeping of flattened mesh explicitly.Ahead-Of-Time Inductor (AOTI)
- Move
from/tototorch::stable::detail(#164956)
JIT
torch.jitis not guaranteed to work in Python 3.14. Deprecation warnings have been added to user-facingtorch.jitAPI (#167669).
torch.jit should be replaced with torch.compile or torch.export.
ONNX
- The
dynamic_axesoption intorch.onnx.exportis deprecated (#165769)
Users should supply the dynamic_shapes argument instead. See https://docs.pytorch.org/docs/stable/export.html#expressing-dynamism for more documentation.
Profiler
- Deprecate
export_memory_timelinemethod (#168036)
The export_memory_timeline method in torch.profiler is being deprecated in favor of the newer memory snapshot API (torch.cuda.memory._record_memory_history and torch.cuda.memory._export_memory_snapshot). This change adds the deprecated decorator from typing_extensions and updates the docstring to guide users to the recommended alternative.
New Features
Autograd
- Allow setting grad_dtype on leaf tensors (#164751)
- Add Default Autograd Fallback for PrivateUse1 in PyTorch (#165315)
- Add API to annotate disjoint backward for use with
torch.utils.checkpoint.checkpoint(#166536)
Complex Frontend
- Add
ComplexTensorsubclass (#167621)
Composability
- Support autograd in torch.cond (#165908)
cuDNN
- BFloat16 support added to cuDNN RNN (#164411)
- [cuDNN][submodule] Upgrade to cuDNN frontend 1.16.1 (#170591)
Distributed
-
LocalTensor:
LocalTensoris a powerful debugging and simulation tool in PyTorch's distributed tensor ecosystem. It allows you to simulate distributed tensor computations across multiple SPMD (Single Program, Multiple Data) ranks on a single process. This is incredibly valuable for: 1) debugging distributed code without spinning up multiple processes; 2) understanding DTensor behavior by inspecting per-rank tensor states; 3) testing DTensor operations with uneven sharding across ranks; 4) rapid prototyping of distributed algorithms. Note that LocalTensor is designed for debugging purposes only. It has significant overhead and is not suitable for production distributed training.LocalTensoris atorch.Tensorsubclass that internally holds a mapping from rank IDs to local tensor shards. When you perform a PyTorch operation on aLocalTensor, the operation is applied independently to each local shard, mimicking distributed computation (LocalTensorsimulates collective operations locally without actual network communication.).LocalTensorModeis the context manager that enablesLocalTensordispatch. It intercepts PyTorch operations and routes them appropriately. The@maybe_run_for_local_tensordecorator is essential for handling rank-specific logic when implementing distributed code.- To get started with
LocalTensor, users import fromtorch.distributed._local_tensor, initialize a fake process group, and wrap their distributed code in aLocalTensorModecontext. Within this context, DTensor operations automatically produce LocalTensors. - PRs: (#164537, #166595, #168110,#168314,#169088,#169734)
-
c10d:
- New
shrink_groupimplementation to exposencclCommShrinkAPI (#164518)
- New
Dynamo
torch.compilenow fully works in Python 3.14 (#167384)- Add option to error or disable applying side effects (#167239)
- Config flag (
skip_fwd_side_effects_in_bwd_under_checkpoint) to allow eager and compile activation-checkpointing divergence for side-effects (#165775) torch._higher_order_ops.printfor enabling printing without graph breaks or reordering (#167571)
FX
- Added node metadata annotation API
- Disable preservation of node metadata when
enable=False(#164772) - Annotation should be mapped across submod (#165202)
- Annotate bw nodes before eliminate dead code (#165782)
- Add logging for debugging annotation (#165797)
- Override metadata on regenerated node in functional mode (#166200)
- Skip copying custom meta for gradient accumulation nodes; tag with is_gradient_acc=True (#167572)
- Add metadata hook for all nodes created in runtime_assert pass (#169497)
- Update `gm.print...
PyTorch 2.9.1 Release, bug fix release
This release is meant to fix the following issues (regressions / silent correctness):
Tracked Regressions
Significant Memory Regression in F.conv3d with bfloat16 Inputs in PyTorch 2.9.0 (#166643)
This release provides work around this issue. If you are impacted please install nvidia-cudnn package version 9.15+ from pypi. (#166480) (#167111)
Torch.compile
Fix Inductor bug when compiling Gemma (#165601)
Fix InternalTorchDynamoError in bytecode_transformation (#166036)
Fix silent correctness error_on_graph_break bug where non-empty checkpoint results in unwanted graph break resumption (#166586)
Improve performance by avoiding recompilation with mark_static_address with cudagraphs (#162208)
Improve performance by caching get_free_symbol_uses in torch inductor (#166338)
Fix fix registration design for inductor graph partition for vLLM (#166458) (#165815) (#165514)
Fix warning spamming in torch.compile (#166993)
Fix exception related to uninitialized tracer_output variable (#163169)
Fix crash in torch.bmm and torch.compile with PyTorch release 2.9.0 (#166457)
Other
Fix warning spamming on new APIs to control TF32 behavior (#166956)
Fix distributed crash with non-contiguous gather inputs (#166181)
Fix indexing on large tensor causes invalid configuration argument (#166974)
Fix numeric issue in CUDNN_ATTENTION (#166912) (#166570)
Fix symmetric memory issue with fused_scaled_matmul_reduce_scatter (#165086)
Improve libtorch stable ABI documentation (#163899)
Fix image display on pypi project description section (#166404)
2.9 Release Notes
PyTorch 2.9.0 Release Notes
- Highlights
- Backwards Incompatible Changes
- Deprecations
- New Features
- Improvements
- Bug fixes
- Performance
- Documentation
- Developers
- Security
Highlights
| Unstable (API-Unstable) |
| Updates to the stable libtorch ABI for third-party C++/CUDA extensions |
| Symmetric memory that enables easy programming of multi-GPU kernels |
| The ability to arbitrarily toggle error or resume on graph breaks in torch.compile |
| Expanded wheel variant support to include ROCm, XPU and CUDA 13 |
| FlexAttention enablement on Intel GPUs |
| Flash decoding optimization based on FlexAttention on X86 CPU |
| ARM Platform improvements and optimizations |
| Enablement of Linux aarch64 binary wheel builds across all supported CUDA versions |
For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.
Backwards Incompatible Changes
Min supported Python version is now 3.10 (#162310)
The minimum version of Python required for PyTorch 2.9.0 is 3.10. We also have 3.14 and 3.14t available as preview with this release.
Undefined behavior when an output of a custom operator shares storage with an input
This is a reminder that outputs of PyTorch custom operators (that are registered using the torch.library or TORCH_LIBRARY APIs) are not allowed to return Tensors that share storage with input tensors. The violation of this condition leads to undefined behavior: sometimes the result will be correct, sometimes it will be garbage.
After #163227, custom operators that violated this condition that previously returned correct results under torch.compile may now return silently incorrect results under torch.compile. Because this is changing the behavior of undefined behavior, we do not consider this to be a bug, but we are still documenting it in this section as a "potentially unexpected behavior change".
This is one of the conditions checked for by torch.library.opcheck and is mentioned in The Custom Operators Manual
More details
Outputs of PyTorch custom operators are not allowed to return Tensors that share storage with input tensors
For example, the following two custom operators are not valid custom operators:
@torch.library.custom_op("mylib::foo", mutates_args=())
def foo(x: torch.Tensor) -> torch.Tensor:
# the result of `foo` must not directly be an input to foo.
return x
@torch.library.custom_op("mylib::bar", mutates_args=())
def bar(x: torch.Tensor) -> torch.Tensor:
# the result of bar must not be a view of an input of bar
return x.view(-1)The easiest workaround is to add an extra .clone() to the outputs:
@torch.library.custom_op("mylib::foo", mutates_args=())
def foo(x: torch.Tensor) -> torch.Tensor:
return x.clone()
@torch.library.custom_op("mylib::bar", mutates_args=())
def bar(x: torch.Tensor) -> torch.Tensor:
return x.view(-1).clone()A common way to get into this situation is for a user to want to create a custom operator that sometimes mutates the input in-place and sometimes returns a new Tensor, like in the following example.
@torch.library.custom_op("mylib::baz", mutates_args=["x"])
def baz(x: torch.Tensor) -> torch.Tensor:
if inplace:
x.sin_()
return x
else:
return x.sin()This dynamism is not supported and leads to undefined behavior. The workaround is to split the custom operator into two custom operators, one that always mutates the input in-place, and another that always returns a new Tensor.
@torch.library.custom_op("mylib::baz_outplace", mutates_args=())
def baz_outplace(x: torch.Tensor) -> torch.Tensor:
return x.sin()
@torch.library.custom_op("mylib::baz_inplace", mutates_args=["x"])
def baz_inplace(x: torch.Tensor) -> torch.Tensor:
x.sin_()
def baz(x):
if inplace:
baz_inplace(x)
return x
else:
return baz_outplace(x)Build metal kernels of MacOS-14+ and remove all pre-MacOS-14 specific logic, requires MacOS-14+ going forward (#159733, #159912)
PyTorch MPS is only supported on MacOS-14 or later. If you need to use MPS on MacOS Ventura, please avoid updating to Python-3.9 or above
Upgrade to DLPack 1.0 (#145000)
This upgrade is doing the same BC-breaking changes as the DLPack release. Objects in torch.utils.dlpack have been updated to reflect these changes, such as DLDeviceType.
See the PR for details on the exact changes and how to update your code.
Raise appropriate errors in torch.cat (#158249)
torch.cat now raises ValueError, IndexError or TypeError where appropriate instead of the generic RuntimeError. If you code was catching these errors, you can update to catch the new error type.
Default to dynamo=True for ONNX exporter (#159646, #162726)
Previously torch.onnx.export(...) used the legacy TorchScript exporter if no arguments were provied. The ONNX exporter now uses the newer torch.export.export pipeline by default (dynamo=True). This change improves graph fidelity and future-proofs exports, but may surface graph capture errors that were previously masked or handled differently.
Previously in torch 2.8.0:
# API calls the legacy exporter with dynamo=False
torch.onnx.export(...)Now in torch 2.9.0:
# To preserve the original behavior
torch.onnx.export(..., dynamo=False)
# Export onnx model through torch.export.export
torch.onnx.export(...)Recommendation: first try the new default; only fall back if you hit blocking issues and report them upstream.
Long term solution: fix the root cause instead of relying on fallback or TorchScript exporter.
Switch off runtime asserts by default in Export in favor of a shape guards function (#160111, #161178, #161794)
To enable runtime asserts, use export(..., prefer_deferred_runtime_asserts_over_guards=True). Also kills the allow_complex_guards_as_runtime_asserts flag, merging it into the former option.
Additionally, exported_program.module() will generate a call to a _guards_fn submodule that will run additional checks on inputs. Users who do not want this behavior can either remove this call in the graph, or do exported_program.module(check_guards=False) to avoid the generation.
Set default opset to 20 in ONNX (#158802)
Opset 20 enables newer operator definitions. If your tooling or downstream runtime only supports opset 18, pin it explicitly. For the latest ONNX operators, you can experiment with opset 23.
Previously in torch 2.8.0:
# opset_version=18
torch.onnx.export(...)Now in torch 2.9.0:
# To preserve the original behavior
torch.onnx.export(..., opset_version=18)
# New: opset_version=20
torch.onnx.export(...)
# Use the latest supported opset: opset_version=23
torch.onnx.export(..., opset_version=23)Drop draft_export in exporter API (#161454, #162225)
Remove implicit draft tracing from the default exporter path, achieving clearer behaviour and faster failures.
The expensive torch.export.draft_export diagnostic path is no longer auto-invoked (which could take hours on large models). You can still opt in for deep diagnostics:
Previously in torch 2.8.0:
# If both torch.export.export(..., strict=False) and
# torch.export.export(..., strict=True) fail to capture
# the model graph, torch.export.draft_export(...) will be triggered,
# and uses real tensor to trace/export the model.
#
# Inside export_to_onnx.py:
# ... torch.onnx.export(..., dynamo=True)
python export_to_onnx.pyNow in torch 2.9.0:
# To trigger torch.export.draft_export once
# torch.export.export strict=False/True both
# fail:
TORCH_ONNX_ENABLE_DRAFT_EXPORT=True python export_to_onnx.pyRemove torch.onnx.dynamo_export and the onnxrt torch compile backend (#158130, #158258)
torch.onnx.dynamo_export is removed. Please use torch.onnx.export instead.
The experimental ONNX Runtime compile backend (torch.compile(backend="onnxrt")) is no longer supported.
Remove torch.onnx.enable_fake_mode (#161222)
The dynamo=True mode uses FakeTensors by default which is memory efficient.
Some public facing ONNX utility APIs for the TorchScript based exporter are now private (#161323)
Deprecated members in torch.onnx.verification are removed. Previously private torch.onnx.symbolic_opsets* functions will no longer be accessible. Consider making a copy of the source code if you need to access any private functions for compatibility with the TorchScript based exporter.
Remove torch.onnx.symbolic_caffe2 (#157102)
Support for caffe2 in the ONNX exporter has ended and is removed.
Remove /d2implyavx512upperregs flag that slows build (#159431)
Re-introduced AVX512 optimizations for Windows VS2022 builds, may cause issues with specific versions of VS2022, see #145702
Add ScalarType to shim conversion and stable::Tensor.scalar_type (#160557)
Before, user extensions could only in abstract...