Skip to content

Releases

PyTorch 2.15.0 Release

Choose a tag to compare

@soulitzer soulitzer released this 28 Oct 17:09
31a77fc

PyTorch 2.15.0 Release Notes

Highlights

Rewritten NCCL backend becomes the default, with a flight recorder that dumps on hangs and TORCH_DIST_USE_NCCL2=0 or nccl-legacy as rollback paths
One-sided transports for torch.distributed, with a NIXL backend and CUDA-graph-capturable transfers
FSDP2 and pipelining gain finer memory and communication control, with fp32 for precision-sensitive parameters, CUDA-graph-captured gradient accumulation, and gradient reduction overlapped with compute
Faster 4-bit (NVFP4) LLM decoding on NVIDIA Blackwell GPUs through fewer, better-overlapped kernel launches in NVGEMM
Apple Silicon gains FP8 and faster decode, with float8_e4m3fn and _scaled_mm, a new grouped_mm with a decode fast path (1.32–1.34x tokens/s on MoE decode), and public torch.mps.Stream APIs
Broader platform support with FlyDSL kernels on AMD gfx950, Tensor Data Mover support for gfx1250 (MI450), ROCm 10.0 wheels, simpler SYCL kernel authoring and multi-GPU profiling on Intel GPUs, CUDA 13.2 as the default wheel with CUDA 13.4 alongside it, and torch.compile on Python 3.15

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Backwards Incompatible Changes

Python Frontend

import torch can no longer be re-executed in the same process, including via importlib.reload(torch) (#194654)

importlib.reload(torch) and del sys.modules["torch"] + re-import now raise, as does retrying import torch after it failed. Re-running the module body was already unsafe in exactly the same way as the retry, so the new check does not distinguish the two. Normal use is unaffected: a successful import leaves torch in sys.modules so the body never re-runs, and re-entrant imports from a backend extension during autoload still short-circuit on sys.modules without reaching the check.

Version 2.14:

import importlib
import torch

importlib.reload(torch)
# RuntimeError: Only a single TORCH_LIBRARY can be used to register the namespace triton; ...
# (re-importing after a failed `import torch` failed with similarly misleading errors or could segfault)

Version 2.15:

import importlib
import torch

importlib.reload(torch)
# ImportError: `torch` can only be initialized once per process, so this module cannot
# be imported again: neither via `importlib.reload(torch)`, nor by re-importing after
# a failed `import torch`.

Nested tensors are no longer allowlisted for torch.load(..., weights_only=True) (#194380)

Nested jagged and strided nested tensors are no longer on the default weights_only allowlist. Loading a checkpoint that contains one with weights_only=True (the default) now fails with Unsupported global: GLOBAL torch.nested._internal.nested_tensor._rebuild_njt was not an allowed global by default (or GLOBAL torch._utils._rebuild_nested_tensor for strided nested tensors). To load checkpoints containing nested tensors, allow them explicitly with torch.serialization.safe_globals (see below), or use weights_only=False for trusted files.

Version 2.14:

loaded = torch.load("njt.pt", weights_only=True)  # works

Version 2.15:

from torch.nested._internal.nested_tensor import _rebuild_njt, NestedTensor
with torch.serialization.safe_globals([_rebuild_njt, NestedTensor]):
    loaded = torch.load("njt.pt", weights_only=True)

Operators called through torch.ops.* and torch._C._dispatch_call_boxed now use PyTorch's own warning and exception translation instead of pybind11's (#193451)

We make torch.ops.* calls go through the same exception and warning translation as the rest of the library. This means that some exception translation that used to be handled by pybind is now handled natively. User-defined pybind exception translation is thus lost. Restoring it may be done in a future release as part of a broader exception handler improvement.

In addition, TORCH_WARN warnings from these operators now reach Python's warnings module, so they raise when warnings are turned into errors (-W error). std::out_of_range still becomes IndexError (#195713).

Version 2.14:

# C++ op: throw std::invalid_argument("bad size");
try:
    torch.ops.my_lib.my_op(x)
except ValueError:  # pybind11 translation
    ...

Version 2.15:

# C++ op: TORCH_CHECK_VALUE(false, "bad size");
try:
    torch.ops.my_lib.my_op(x)
except ValueError:
    ...

# TORCH_WARN from the op is now a Python warning
with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    torch.ops.my_lib.my_op(x)

Autograd

Selective activation checkpointing now saves, rather than recomputes, the outputs of operators with an ordered effect unless the policy returns CheckpointPolicy.MUST_RECOMPUTE (#196956)

In eager mode, if a create_selective_checkpoint_contexts policy returns CheckpointPolicy.PREFER_RECOMPUTE for a non-aliasing operator with an ordered effect (e.g. via register_effect(torch.library.EffectType.ORDERED)), that choice is now overridden and its outputs are saved, so the side effect is no longer replayed during backward. This can increase activation memory. To keep the 2.14 behavior, return CheckpointPolicy.MUST_RECOMPUTE for that operator. Operators in the c10d namespace are excluded. Under torch.compile, CheckpointPolicy.MUST_RECOMPUTE on an effectful operator now raises at compile time: RuntimeError: AOTAutograd does not support MUST_RECOMPUTE for effectful operations because forward effects cannot join the backward effect-token chain. Use a save policy instead.

policy = lambda ctx, op, *args, **kwargs: CheckpointPolicy.PREFER_RECOMPUTE
checkpoint(fn, x, use_reentrant=False, context_fn=lambda: create_selective_checkpoint_contexts(policy))
# 2.14: an effectful op in fn runs twice; 2.15: once (use CheckpointPolicy.MUST_RECOMPUTE to replay)

Distributed (c10d)

The nccl backend now uses a rewritten, cleaned-up NCCL implementation by default (#192281, #192895, #193237, #193241)

backend="nccl" (also the default CUDA backend of init_process_group()) now creates the new implementation. The legacy implementation is available as backend="nccl-legacy" for this release only: nccl-legacy and TORCH_DIST_USE_NCCL2=0 will be removed in the next release, so migrate to nccl or nccl-lazy.

Use nccl (eager initialization) by default. Use nccl-lazy when you need independent point-to-point communicators: collectives run on one eagerly created communicator, and each send/recv peer pair gets its own lazily created communicator and stream, so concurrent P2P to different peers can overlap (as in the legacy backend).

Differences from the legacy backend:

  • Communicators are created eagerly in init_process_group()/new_group(), allocating GPU memory even for unused groups.
  • Each group is bound to one CUDA device: device_id, else LOCAL_RANK % torch.cuda.device_count() (or the global rank modulo the device count); torch.cuda.set_device() is not used. Tensors on another GPU fail with Expected tensor on cuda:X but found tensor on cuda:Y or ProcessGroupNCCL is bound to device cuda:X but an operation was issued on cuda:Y. Pass device_id if needed; several GPUs per process requires nccl-legacy.
  • dist.get_backend_impl(...) returns the new backend class, so isinstance(..., ProcessGroupNCCL) is False and legacy-only methods such as comm_split_count() are missing.
  • An async all_gather's work.get_future() value is the list of per-rank outputs, not a one-element list holding a stacked tensor; read the output list passed to all_gather instead (works on both backends).

To temporarily restore the legacy behavior in 2.15, set TORCH_DIST_USE_NCCL2=0 on every rank before creating the process group, or use nccl-legacy:

Version 2.14:

dist.init_process_group("nccl")  # ProcessGroupNCCL

Version 2.15:

dist.init_process_group("nccl", device_id=torch.device("cuda", local_rank))  # new implementation
dist.init_process_group("nccl-lazy", device_id=torch.device("cuda", local_rank))  # independent P2P communicators
dist.init_process_group("nccl-legacy")  # legacy implementation, removed in the next release

Distributed Checkpointing

Distributed checkpoint metadata now records tensor strides, so checkpoints saved with PyTorch 2.15 cannot be loaded by PyTorch 2.14 or earlier (#194251)

TensorProperties gains an optional strides field in the pickled .metadata file. In older versions dcp.load raises a CheckpointException reporting too many values to unpack (expected 5). There is no flag to write the old format.

Version 2.14 (loading a 2.15 checkpoint):

dcp.load(state_dict, checkpoint_id="/ckpt/step_100")
# CheckpointException: ... too many values to unpack (expected 5)

Version 2.15:

dcp.load(state_dict, checkpoint_id="/ckpt/step_100")  # loads 2.14 and 2.15 checkpoints

Distributed FSDP2

With MixedPrecisionPolicy(reduce_dtype=None), fully_shard now accumulates and reduces gradients in the parameter's grad_dtype (its original dtype by default) instead of in param_dtype (#194434)

This is BC-breaking: mp_policy.reduce_dtype=None no longer ...

Read more

PyTorch 2.14.1 Release

Choose a tag to compare

@atalman atalman released this 30 Sep 19:14
5c48869

This release is meant to fix the following regressions and silent correctness issues:

Silent correctness fixes

  • Fix incorrect torch.linalg.lstsq solutions on MPS for complex batched underdetermined systems (#196113), fixed by #196128
  • Fix non-orthogonal U and inaccurate small singular values from torch.linalg.svd on MPS for rank-deficient and ill-conditioned inputs (#196112), fixed by #196139 and #199063
  • Update the CUDA 13.2 Linux binaries to CUDA 13.2.2 (#196351). This NVIDIA update resolves two critical issues that could produce incorrect results (CUDA 13.2.2 release notes):
    • cuBLAS: cublasLtMatmul() could ignore tensor-wide scaling for NVFP4 matrix multiplications (introduced in CUDA 13.2 Update 1)
    • Compiler: failed thread reconvergence could leave stale or corrupted register values in kernels with nested thread divergence (present since CUDA 12.8)

Regression fixes

  • Fix torch.linalg.svd, torch.linalg.svdvals and torch.linalg.lstsq failing on MPS with a Metal pipeline-state error for inputs above 8192 elements (#195937), fixed by #195949 and #195950
  • Fix internal assert in torch.svd(out=) on MPS for complex inputs (#195822), fixed by #195872

PyTorch 2.14.0 Release

Choose a tag to compare

@ethche ethche released this 02 Sep 17:40
2b3ec34

PyTorch 2.14.0 Release Notes

Highlights

NVGEMM brings CuTeDSL-generated CUTLASS kernels to Inductor, with epilogue fusion, scaled and NVFP4 GEMM, and grouped-reduction epilogues autotuned alongside Triton and ATen
torch.switch generalizes torch.cond to multi-way branching, and torch.while_loop can now be captured in a CUDA graph
Declarative dynamic shapes via @dynamic_spec, shared across torch.compile, torch.export and make_fx
Experimental torch.compile support for complex-valued tensors: Opt-in support decomposes supported complex operations into real and imaginary computations, enabling compiler backends to optimize more complex-number workloads.
A preview of our rewritten NCCL backend for PyTorch, ported from torchcomms, implementing the full collective contract with nonblocking communicators and eager communicator splitting and advanced features such as fault tolerance and windows designed as a drop-in replacement of existing NCCL c10d backend
Fault tolerance becomes a first-class c10d concept, with in-place process-group reconfiguration, one-sided RMA windows, and a Flight Recorder that works for any backend rather than only NCCL
Apple Silicon gains native linear algebra, including Jacobi-kernel SVD, eigh, QR and Cholesky, alongside a five-part reduction rewrite and a further MPSGraph to Metal kernel migration
Broader platform support: ROCm 7.14 wheels are produced from the TheRock pip SDK, Intel XPU adds native graph capture, and Inductor targets Rubin (sm_107)

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Backwards Incompatible Changes

torch.nn

  • torch.nn.LinearCrossEntropyOptions no longer accepts acc_policy="balanced"; use "compact" instead (#188283)

    The "balanced" policy was removed because "compact" provides the same weight-gradient accumulation precision with lower memory use on CUDA, already uses the equivalent scratch layout for mixed-precision inputs on other devices, and was never selected by "auto". Constructing the options with acc_policy="balanced" now raises ValueError: invalid acc_policy: 'balanced'; expected one of 'auto', 'accurate', 'compact'.

    Before:

    options = torch.nn.LinearCrossEntropyOptions(acc_policy="balanced")
    loss = torch.nn.functional.linear_cross_entropy(
        input, linear_weight, target, options=options
    )

    After:

    options = torch.nn.LinearCrossEntropyOptions(acc_policy="compact")
    loss = torch.nn.functional.linear_cross_entropy(
        input, linear_weight, target, options=options
    )

Autograd

  • Clamp and min/max boundary subgradients now follow the selected dispatcher schema's input space (#191142)

    This affects gradients exactly at nondifferentiable bounds or ties. A scalar clamp bound is a fixed parameter, so the input gradient at equality changes from 1 to the minimum-norm subgradient 0. A Tensor bound is part of the differentiable input space, so clamp, clamp_min, and clamp_max now split the gradient evenly between the input and bound at an ordinary tie instead of assigning it entirely to the input. fmin and fmax use the same even tie split, and forward-mode AD for the min/max family is aligned with these rules. Code that intentionally depends on the old tie-breaking behavior can express it explicitly with torch.where, such as torch.where(value >= bound, value, bound).

    Version 2.13:

    import torch
    
    x = torch.tensor(0.0, requires_grad=True)
    torch.clamp_min(x, 0.0).backward()
    print(x.grad)  # tensor(1.)
    
    value = torch.tensor(0.0, requires_grad=True)
    bound = torch.tensor(0.0, requires_grad=True)
    torch.clamp_min(value, bound).backward()
    print(value.grad, bound.grad)  # tensor(1.) tensor(0.)

    Version 2.14:

    import torch
    
    x = torch.tensor(0.0, requires_grad=True)
    torch.clamp_min(x, 0.0).backward()
    print(x.grad)  # tensor(0.)
    
    value = torch.tensor(0.0, requires_grad=True)
    bound = torch.tensor(0.0, requires_grad=True)
    torch.clamp_min(value, bound).backward()
    print(value.grad, bound.grad)  # tensor(0.5000) tensor(0.5000)

Distributed

  • Custom Python process groups that implement new_group() must now accept a backend keyword argument (#188489)

    This applies when the default process group supplies its own new_group() method and torch.distributed.new_group() delegates subgroup creation to it. PyTorch now forwards the resolved backend so custom implementations can construct the requested subgroup correctly. Existing implementations without this parameter will raise TypeError: ... got an unexpected keyword argument 'backend'. Accept and use the argument, or accept and ignore it when the implementation has only one backend.

    Before:

    class MyProcessGroup(...):
        def new_group(
            self, ranks, *, timeout=None, pg_options=None,
            group_name=None, group_desc=None
        ):
            ...

    After:

    class MyProcessGroup(...):
        def new_group(
            self, ranks, *, timeout=None, backend=None, pg_options=None,
            group_name=None, group_desc=None
        ):
            ...
  • NCCL symmetric-memory pools no longer automatically upgrade segments allocated after register_mem_pool(..., symm=True) to symmetric windows (#192112)

    Registering those late segments from the CUDA allocator callback could invoke a collective NCCL operation on only some ranks while holding the allocator lock, causing an unrecoverable hang. Late segments now remain ordinary registered NCCL buffers. Applications that need newly allocated segments to use symmetric-window algorithms must collectively deregister and register the pool again after those allocations are created.

    Before:

    backend.register_mem_pool(pool, symm=True)
    with torch.cuda.use_mem_pool(pool):
        tensor = torch.empty(size, device="cuda")
    # Newly allocated segments were automatically upgraded, but this could hang.

    After:

    backend.register_mem_pool(pool, symm=True)
    with torch.cuda.use_mem_pool(pool):
        tensor = torch.empty(size, device="cuda")
    
    # Collectively refresh registration after the pool grows.
    backend.deregister_mem_pool(pool)
    backend.register_mem_pool(pool, symm=True)
  • Nonmember ranks now receive GroupMember.NON_GROUP_MEMBER instead of None from experimental torch.distributed.split_group() (#190725)

    When the calling rank is absent from every requested split, split_group() now returns the same nonmember sentinel as new_group(). Code that identifies nonmembers with is None must compare against torch.distributed.GroupMember.NON_GROUP_MEMBER instead.

    Before:

    group = torch.distributed.split_group(
        split_ranks=[[0, 1], [2, 3]]
    )
    if group is None:
        return

    After:

    group = torch.distributed.split_group(
        split_ranks=[[0, 1], [2, 3]]
    )
    if group == torch.distributed.GroupMember.NON_GROUP_MEMBER:
        return

Linear Algebra Frontend

  • Remove the deprecated torch.cholesky() and Tensor.cholesky() APIs (#186817)

    Calls now raise a RuntimeError directing users to torch.linalg.cholesky(). The replacement returns a lower-triangular factor; callers that previously requested upper=True should take the conjugate transpose with .mH.

    Version 2.13:

    lower = torch.cholesky(a)
    upper = torch.cholesky(a, upper=True)

    Version 2.14:

    lower = torch.linalg.cholesky(a)
    upper = torch.linalg.cholesky(a).mH
  • Remove the deprecated torch.qr() and Tensor.qr() APIs (#186815)

    Calls now raise a RuntimeError directing users to torch.linalg.qr(). Replace the Boolean some argument with mode="reduced" or mode="complete".

    Version 2.13:

    q, r = torch.qr(a)
    q_full, r_full = torch.qr(a, some=False)

    Version 2.14:

    q, r = torch.linalg.qr(a, mode="reduced")
    q_full, r_full = torch.linalg.qr(a, mode="complete")

Profiler

  • The deprecated use_cuda argument has been removed from torch.profiler.profile and torch.autograd.profiler.profile (#192543)

    Passing use_cuda to either profiler now raises TypeError: profile.__init__() got an unexpected keyword argument 'use_cuda'. Select CUDA explicitly through activities when using torch.profiler.profile, or use use_device="cuda" with torch.autograd.profiler.profile.

    Version 2.13:

    with torch.profiler.profile(use_cuda=True) as prof:
        run_workload()
    
    with torch.autograd.profiler.profile(use_cuda=True) as prof:
        run_workload()

    Version 2.14:

    with torch.profiler.profile(
        activities=[
            torch.profiler.ProfilerActivity.CPU,
            torch.profiler.ProfilerActivity.CUDA,
        ]
    ) as prof:
        run_workload()
    
    with torch.autograd...
Read more

PyTorch 2.13.0 Release

Choose a tag to compare

@angelayi angelayi released this 08 Jul 17:39
cf30153

PyTorch 2.13.0 Release Notes

Highlights

FlexAttention lands on Apple Silicon (MPS), with up to ~12x speedup over SDPA on sparse patterns, and gains a deterministic backward path on CUDA for reproducible gradient computation.
CuTeDSL "Native DSL" backend gives Inductor a second high-performance code path (alongside Triton) for key GPU operations, with faster compilation. [Prototype]
nn.LinearCrossEntropyLoss combines the final prediction and loss computation to cut peak GPU memory by up to 4x for large-vocabulary language model training.
torchcomms, a new communications backend for PyTorch Distributed, improves fault tolerance, scalability, and debuggability for large-cluster training.
FSDP2 now overlaps reduce-scatter and all-gather communications via a dedicated process group (opt-in), increasing distributed training throughput.
Python 3.15 wheel support for PyTorch on Linux via the pytorch repository index, including builds compatible with free-threaded 3.15t.
Broader platform support: ROCm gains AOTriton 0.12b with native HIP CMake, Arm adds Armv9-A torch.compile targeting, and Intel XPU exposes new device telemetry APIs.

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Tracked Regressions

ROCm wheels break torch.compile on CPU in environments without a GPU

Running a torch==2.13.0+rocm7.2 wheel in an environment where no GPU is available (torch.cuda.is_available() is False) breaks torch.compile on the CPU path: the first compile raises RuntimeError: Can't detect vectorized ISA for CPU (#189194). This is a regression from torch==2.12.1+rocm7.2, which compiles CPU code fine (detecting e.g. VecAVX2) in the same setup. The 2.13 ROCm wheel appears to rely on something present in the ROCm builder image to detect the CPU vectorized ISA, so it works when run on a ROCm image but fails on a plain CPU-only image.

Workaround: run the +rocm wheel on a ROCm image, or install a standard CPU/CUDA build for GPU-less environments.

Backwards Incompatible Changes

  • Stop building CPython 3.13t (free-threaded) binaries (#182951)

    Upstream pypa/manylinux removed CPython 3.13t (free-threaded) on 2026-05-07, because 3.13t
    was experimental and has been superseded by the now-non-experimental CPython 3.14t. As a result,
    PyTorch 2.13 no longer ships cp313t wheels (Linux, Triton, and related artifacts). Users on the
    free-threaded interpreter should move to Python 3.14t.

    PyTorch 2.12:

    # cp313t (free-threaded 3.13) wheels were available
    python3.13t -m pip install torch

    PyTorch 2.13:

    # Use free-threaded Python 3.14t instead
    python3.14t -m pip install torch
  • Bare PyObject is no longer allowed in operator schemas (#184209)

    Bare PyObject was accidentally accepted in operator schema strings in
    PyTorch 2.12. This was undocumented and is now rejected, since torch.compile
    does not support arbitrary PyObject inputs to custom ops. If
    you parse or register a schema with a bare PyObject argument or return type,
    you will now get a schema parse error.

    PyTorch 2.12:

    >>> from torch._C import parse_schema
    >>> parse_schema("foo(PyObject x) -> ()")  # accepted

    PyTorch 2.13:

    >>> from torch._C import parse_schema
    >>> parse_schema("foo(PyObject x) -> ()")  # raises a schema parse error
  • Remove Bazel build support (#180883)

    The Bazel build was never broadly adopted and still depended on the antiquated Bazel 6,
    while the wider ecosystem has since moved to Bazel 9. All Bazel build files and CI jobs have
    been removed. Users building PyTorch with Bazel should migrate to the supported CMake/pip install
    build flow.

    PyTorch 2.12:

    # Build PyTorch with Bazel
    bazel build //:torch

    PyTorch 2.13:

    # Bazel build files have been removed; build from source with pip instead
    pip install --no-build-isolation -e .
  • StorageImpl's built-in copy-on-write (COW) materialization is replaced by a pluggable materializer hook (#179063)

    StorageImpl no longer knows about COW directly. Its internal COW entry points
    StorageImpl::is_cow(), StorageImpl::maybe_materialize_cow(), and the friend
    cow::materialize_cow_storage() have been removed in favor of a single pluggable
    MaterializeFn hook (void(*)(StorageImpl*)) that a backend registers to run once,
    on the first mutable data-pointer access. COW is now just one consumer of this hook
    (c10::impl::cow::materialize_cow), and all COW behavior (lazy clone, refcounted
    shared data, copy-on-write) is unchanged. This also gives accelerator backends and
    eager-mode graph compilers a zero-fast-path-cost place to commit deferred allocations
    or materialize symbolic buffers on first mutation.

    This is a C++-only change. It affects out-of-tree backends/extensions that called the
    removed StorageImpl COW symbols directly; they will fail to compile against 2.13
    with errors such as no member named 'is_cow' in 'c10::StorageImpl'. Migrate to the
    new hook API (set_materializer() / has_materializer() / clear_materializer()).

    PyTorch 2.12:

    // Detect a COW storage and force it to materialize.
    if (storage.is_cow()) {
      storage.maybe_materialize_cow();
    }

    PyTorch 2.13:

    // Register a one-shot materializer; it runs on the next mutable-data access
    // and then clears itself. COW registers c10::impl::cow::materialize_cow this way.
    storage.set_materializer(&my_backend_materialize);  // void(StorageImpl*)
    
    // `has_materializer()` replaces `is_cow()` for "is a deferred materialization pending?"
    if (storage.has_materializer()) { /* ... */ }
  • Convert shared_ptr<Node> to intrusive_ptr<Node> (#181139). This changes the signature of Tensor::grad_fn. Accesses to Tensor.grad_fn() should change from std::shared_ptr<Node> to c10::intrusive_ptr<Node>. Similarly, construction of a C++ autograd function should change:

    PyTorch 2.12:

    std::shared_ptr<CustomCppNode> node(new CustomCppNode(), torch::autograd::deleteNode);

    PyTorch 2.13:

    auto node = c10::make_intrusive<CustomCppNode>();
  • The minimum supported NCCL version when building from source is now 2.23 (#186292)

    PyTorch now requires NCCL >= 2.23 at compile time, and the preprocessor/runtime gates that guarded NCCL features introduced in 2.23 or earlier have been removed. Users who build PyTorch from source against a system NCCL older than 2.23 will hit compile errors against the dropped gates. Upgrade the NCCL installation to >= 2.23 to build. The prebuilt PyTorch wheels already bundle a compatible NCCL, so pip/conda users are unaffected.

  • Remove named tensors (#173895)

    The named tensor feature (a long-deprecated prototype) has been fully removed to reduce overhead and code bloat. All associated Python and C++ APIs are gone, including Tensor.names, Tensor.rename(), Tensor.refine_names(), Tensor.align_to(), Tensor.align_as(), torch.align_tensors(), the names= keyword on factory functions (e.g. torch.zeros, torch.empty, torch.ones), and the C++ Dimname / DimnameList APIs. Code that previously relied on named dimensions must track dimension order positionally and avoid usage of any of these now-removed APIs or op overloads.

  • The onednn::qconv2d_pointwise.binary and .binary_tensor operators no longer alias their input but rather return fresh tensors. Previously these ops mutated the qaccum input buffer and returned it directly, violating the PyTorch invariant that custom operator outputs must not alias inputs. This silently bypassed aliasing checks via the old -> Tensor(a!) schema and would become a hard error in a future PyTorch version (as mentioned in #182063), so the schema and implementation were corrected to return a fresh output. Most users are unaffected, only code that calls these ops directly and relies on the in-place mutation of qaccum must now read the returned tensor instead. (#177171)

Deprecations

  • Custom operators that return an output aliasing one of their inputs are deprecated (#182063)

    When a custom operator returns an output that is the same tensor as (or otherwise aliases) one of its inputs under torch.compile, PyTorch now emits a UserWarning stating that this is deprecated and will become an error in a future version of PyTorch. Previously the warning stated the change would land in PyTorch 2.12; that timeline has been pushed back. To update your code, return a clone of the offending output instead of the input, or refactor the operator so it does not return the aliased tensor.

    Deprecated:

    @torch.library.custom_op("mylib::foo", mutates_args=())
    def foo(x: torch.Tensor) -> torch.Tensor:
        return x  # output aliases the input -- deprecated

    Updated:

    @torch.library.custom_op("mylib::foo", mutates_args=())
    def foo(x: torch.Tensor) -> torch.Tensor:
        return x.clone()  # return a clone instead
  • Creating tensors with the qua...

Read more

PyTorch 2.12.1 Release, bug fix release

Choose a tag to compare

@atalman atalman released this 18 Jun 00:41
7269437

This release is meant to fix the following regressions and silent correctness issues:

Regression fixes

  • Fix nondeterministic outputs in test_batch_invariance with FLASH_ATTN on NVIDIA B200 GPUs (#181248), fixed by updating Triton to 3.7.1 (#186814)
  • Fix illegal memory access in the Triton convolution2d_bwd_weight kernel on B100/B200 (sm100) GPUs (#187081), fixed by updating Triton to 3.7.1 (#186814)
  • Fix fill_ on byte-dtype views with misaligned storage offset (#186821)

Releng / Build

  • Drop CPython 3.13t from the binary build matrix (#182951)

PyTorch 2.12.0 Release

Choose a tag to compare

@danielvegamyhre danielvegamyhre released this 13 May 17:38
0d62256

PyTorch 2.12.0 Release Notes

Highlights

Batched linalg.eigh on CUDA is up to 100x faster due to updated cuSolver backend selection.
New torch.accelerator.Graph API unifies graph capture and replay across CUDA, XPU, and out-of-tree backends.
torch.export.save now supports Microscaling (MX) quantization formats, enabling full export of aggressively compressed models.
Adagrad now supports fused=True, joining Adam, AdamW, and SGD with a single-kernel optimizer implementation.
torch.cond control flow can now be captured and replayed inside CUDA Graphs.
ROCm users gain expandable memory segments, rocSHMEM symmetric memory collectives, and FlexAttention pipelining.

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Backwards Incompatible Changes

Build Frontend

  • Strengthened SVE compile checks in FindARM.cmake, which may reject previously accepted but incorrect SVE configurations (#176646)

    Source builds that enable SVE now validate the compiler configuration more strictly. If a build previously passed with an incomplete or mismatched SVE setup, it may now fail during CMake configuration instead of later in compilation. Update the compiler/toolchain flags so they accurately describe the target SVE support, or disable SVE for that build.

  • Updated the minimum CUDA version required to build PyTorch from source to CUDA 12.6 (#178925)

    Building PyTorch from source with CUDA versions older than 12.6 is no longer supported. Users building custom binaries should install CUDA 12.6 or newer and make sure CUDA_HOME points to that installation.

    Version 2.11:

    CUDA_HOME=/usr/local/cuda-12.4 python setup.py develop

    Version 2.12:

    CUDA_HOME=/usr/local/cuda-12.6 python setup.py develop
  • Enforced a C++20 minimum in CMake build files (#178662)

    Source builds now require a compiler and build configuration that support C++20. If you maintain custom build scripts or downstream extensions that build PyTorch from source, update the compiler and remove assumptions that PyTorch can be built as C++17.

Distributed

  • torch.distributed.nn.functional ops now raise RuntimeError under torch.compile (#177342)

    All ops in torch.distributed.nn.functional (e.g., broadcast, all_reduce, all_gather, reduce_scatter, all_to_all_single) now raise RuntimeError when called inside torch.compile. Users should migrate to the functional collectives API in torch.distributed._functional_collectives.

    Version 2.11:

    @torch.compile
    def my_func(x):
        return torch.distributed.nn.functional.all_reduce(x, op=ReduceOp.SUM)

    Version 2.12:

    @torch.compile
    def my_func(x):
        return torch.distributed._functional_collectives.all_reduce(x, reduceOp="sum", group=group)

TorchElastic

  • torchrun now defaults to an OS-assigned free port for single-node training instead of port 29500 (#175699)

    When running torchrun --nproc-per-node=N script.py without specifying --master-port or --standalone, the default behavior now automatically uses an OS-assigned free port via the c10d rendezvous backend. This eliminates "Address already in use" errors when running multiple training jobs concurrently. Multi-node training, explicit --master-port, PET_MASTER_PORT env var, and --standalone are unchanged.

    Version 2.11:

    # Used static rendezvous on port 29500 by default
    torchrun --nproc-per-node=4 train.py

    Version 2.12:

    # Uses OS-assigned free port by default
    torchrun --nproc-per-node=4 train.py
    
    # To explicitly use a fixed port:
    torchrun --nproc-per-node=4 --master-port=29500 train.py

MPS

  • All MPS tensors are now allocated in unified memory (#175818)

    Previously, MPS tensors could be allocated in either device-only or unified memory. Now all MPS tensors use unified memory unconditionally. This simplifies memory management and enables CPU access to MPS tensor data without explicit copies. Code that relied on device-only memory placement may observe different performance characteristics.

Inductor

  • The max_autotune layout-constraint deferral introduced in 2.11 is now opt-in (#175330)

    In 2.11, Inductor deferred layout freezing for max_autotune templates to expose more fusion opportunities. This caused a regional-inductor failure mode, so the default in 2.12 reverts to immediate layout freezing. Users who relied on the deferred behavior for fusion opportunities should opt in explicitly via torch._inductor.config.max_autotune_defer_layout_freezing or TORCHINDUCTOR_MAX_AUTOTUNE_DEFER_LAYOUT_FREEZING=1.

    Version 2.11:

    # Deferred layout freezing was the default
    torch.compile(model, mode="max-autotune")

    Version 2.12:

    import torch._inductor.config as cfg
    cfg.max_autotune_defer_layout_freezing = True
    # or set TORCHINDUCTOR_MAX_AUTOTUNE_DEFER_LAYOUT_FREEZING=1
    torch.compile(model, mode="max-autotune")

Deprecations

Release Engineering

  • Deprecate CUDA 12.8 builds in favor of CUDA 13.0 (#179072)

    CUDA 12.8 binaries have been removed from the PyTorch binary build matrix. CUDA 13.0 is now the stable default and CUDA 12.6 remains available for users on older drivers. Users explicitly pinning the cu128 index URL will need to switch to cu130 (recommended) or cu126.

    Version 2.11:

    pip install torch --index-url https://download.pytorch.org/whl/cu128

    Version 2.12:

    # Use CUDA 13.0 (default on PyPI):
    pip install torch
    # Or explicitly:
    pip install torch --index-url https://download.pytorch.org/whl/cu130
    # Older driver fallback:
    pip install torch --index-url https://download.pytorch.org/whl/cu126
  • Compatibility with CMake < 3.10 will be removed in a future release (#166259)

    Source builds against CMake versions older than 3.10 now emit a deprecation warning. A future release will require CMake 3.10 or newer; please upgrade CMake before then.

Linear Algebra

  • Several CUDA linear algebra operators no longer use the MAGMA backend and now dispatch to cuSolver or cuBLAS unconditionally:

    • torch.linalg.eigh now dispatches to cuSolver (#174619)
    • torch.linalg.lu_solve now dispatches to cuSolver/cuBLAS (#174248)
    • torch.linalg.cholesky_inverse now dispatches to cuSolver (#174681)
    • torch.linalg.cholesky_solve now dispatches to cuSolver (#174769)

    User code calling these APIs does not need to change. The practical impact is for users who depended on MAGMA-specific numerical behavior, performance characteristics, or debugging. Those calls now use the cuSolver/cuBLAS implementations on CUDA.

FullyShardedDataParallel2 (FSDP2)

  • Compiling through FSDP2 hooks without graph breaks is no longer supported (#174863, #174906). If you use compiled autograd with FSDP2, update your code to allow graph breaks around FSDP2 hooks or disable compiled autograd for the FSDP2 training step.

    Version 2.11:

    with torch._dynamo.config.patch(compiled_autograd=True):
        compiled_model = torch.compile(fsdp_model, fullgraph=True)
        loss = compiled_model(input).sum()
        loss.backward()

    Version 2.12:

    # Either run FSDP2 backward without fullgraph.
    compiled_model = torch.compile(fsdp_model, fullgraph=False)
    loss = compiled_model(input).sum()
    loss.backward()
    
    # Or apply compile before applying FSDP.
    compiled_model_pre_fsdp = torch.compile(model, fullgraph=True)
    compiled_model = fully_shard(compiled_model_pre_fsdp, ...)
    loss = compiled_model(input).sum()
    loss.backward()

Profiler

  • Profiler's metadata_json field is now deprecated; use event_metadata instead (#179417)

    Version 2.11:

    metadata = event.metadata_json

    Version 2.12:

    metadata = event.event_metadata

Dynamo

  • torch.compile(fullgraph=True) now warns when a call runs no compiled code; will error in 2.13 (#181940)

    Previously fullgraph=True was only validated once Dynamo actually compiled and ran the function. If Dynamo was bypassed at call time (e.g. under a user-defined TorchDispatchMode), the annotation silently had no effect. 2.12 emits a warning; 2.13 will raise. For graph-break erro...

Read more

PyTorch 2.11.0 Release

Choose a tag to compare

@vkuzo vkuzo released this 23 Mar 18:38
70d99e9

PyTorch 2.11.0 Release Notes

Highlights

Added Support for Differentiable Collectives for Distributed Training
FlexAttention now has a FlashAttention-4 backend on Hopper and Blackwell GPUs
MPS (Apple Silicon) Comprehensive Operator Expansion
Added RNN/LSTM GPU Export Support
Added XPU Graph Support

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Backwards Incompatible Changes

Release Engineering

Volta (SM 7.0) GPU support removed from CUDA 12.8 and 12.9 binary builds (#172598)

Starting with PyTorch 2.11, the CUDA 12.8 and 12.9 pre-built binaries no longer include support for Volta GPUs (compute capability 7.0, e.g. V100). This change was necessary to enable updating to CuDNN 9.15.1, which is incompatible with Volta.

Users with Volta GPUs who need CUDA 12.8+ should use the CUDA 12.6 builds, which continue to include Volta support. Alternatively, build PyTorch from source with Volta included in TORCH_CUDA_ARCH_LIST.

Version 2.10:

# CUDA 12.8 builds supported Volta (SM 7.0)
pip install torch --index-url https://download.pytorch.org/whl/cu128
# Works on V100

Version 2.11:

# CUDA 12.8 builds no longer support Volta
# For V100 users, use CUDA 12.6 builds instead:
pip install torch --index-url https://download.pytorch.org/whl/cu126

PyPI wheels now ship with CUDA 13.0 instead of CUDA 12.x (#172663, announcement)

Starting with PyTorch 2.11, pip install torch on PyPI installs CUDA 13.0 wheels by default for both Linux x86_64 and Linux aarch64. Previously, PyPI wheels shipped with CUDA 12.x and only Linux x86_64 CUDA wheels were available on PyPI. Users whose systems have only CUDA 12.x drivers installed may encounter errors when running pip install torch without specifying an index URL.

Additionally, CUDA 13.0 only supports Turing (SM 7.5) and newer GPU architectures on Linux x86_64. Maxwell and Pascal GPUs are no longer supported under CUDA 13.0. Users with these older GPUs should use the CUDA 12.6 builds instead.

CUDA 12.6 and 12.8 binaries remain available via download.pytorch.org.

Version 2.10:

# PyPI wheel used CUDA 12.x
pip install torch

Version 2.11:

# PyPI wheel now uses CUDA 13.0
pip install torch

# To get CUDA 12.8 wheels instead:
pip install torch --index-url https://download.pytorch.org/whl/cu128

# To get CUDA 12.6 wheels (includes Maxwell/Pascal/Volta support):
pip install torch --index-url https://download.pytorch.org/whl/cu126

Python Frontend

torch.hub.list(), torch.hub.load(), and torch.hub.help() now default the trust_repo parameter to "check" instead of None. The trust_repo=None option has been removed. (#174101)

Previously, passing trust_repo=None (or relying on the default) would silently download and run code from untrusted repositories with only a warning. Now, the default "check" behavior will prompt the user for explicit confirmation before running code from repositories not on the trusted list.

Users who were explicitly passing trust_repo=None must update their code. Users who were already passing trust_repo=True, trust_repo=False, or trust_repo="check" are not affected.

Version 2.10:

# Default trust_repo=None — downloads with a warning
torch.hub.load("user/repo", "model")
# Explicit None — same behavior
torch.hub.load("user/repo", "model", trust_repo=None)

Version 2.11:

# Default trust_repo="check" — prompts for confirmation if repo is not trusted
torch.hub.load("user/repo", "model")
# To skip the prompt, explicitly trust the repo
torch.hub.load("user/repo", "model", trust_repo=True)

torch.nn

Add sliding window support to varlen_attn via window_size, making optional arguments keyword-only (#172238)

The signature of torch.nn.attention.varlen_attn has changed: a * (keyword-only separator) has been inserted before the optional arguments. Previously, optional arguments like is_causal, return_aux, and scale could be passed positionally; they must now be passed as keyword arguments. A new window_size keyword argument has also been added.

# Before (2.10)
output = varlen_attn(query, key, value, cu_seq_q, cu_seq_k, max_q, max_k, True, None, 1.0)

# After (2.11) — pass as keyword argument
output = varlen_attn(query, key, value, cu_seq_q, cu_seq_k, max_q, max_k, window_size=(-1, 0), return_aux=None, scale=1.0)

Remove is_causal flag from varlen_attn (#172245)

The is_causal parameter has been removed from torch.nn.attention.varlen_attn. Causal attention is now expressed through the window_size parameter: use window_size=(-1, 0) for causal masking, or window_size=(W, 0) for causal attention with a sliding window of size W. The default window_size=(-1, -1) corresponds to full (non-causal) attention.

# Before (2.10)
output = varlen_attn(query, key, value, cu_seq_q, cu_seq_k, max_q, max_k, is_causal=True)

# After (2.11) — use window_size instead
output = varlen_attn(query, key, value, cu_seq_q, cu_seq_k, max_q, max_k, window_size=(-1, 0))

Distributed

DebugInfoWriter now honors $XDG_CACHE_HOME for its cache directory in C++ code, consistent with the Python side. Previously it always used ~/.cache/torch. (#168232)

This avoids issues where $HOME is not set or not writable. Users who relied on ~/.cache/torch being used regardless of $XDG_CACHE_HOME may see debug info written to a different location.

Version 2.10:

# C++ DebugInfoWriter always wrote to ~/.cache/torch

Version 2.11:

# C++ DebugInfoWriter now respects $XDG_CACHE_HOME/torch (same as Python code)
# Falls back to ~/.cache/torch if $XDG_CACHE_HOME is not set

DeviceMesh now stores a process group registry (_pg_registry) directly, enabling torch.compile to trace through get_group(). (#172272)

This may break code that skips init_process_group, loads a saved DTensor (constructing a DeviceMesh with no PGs), and later creates PGs separately — during torch.compile runtime the PG lookup will fail. Users should ensure process groups are initialized before constructing the DeviceMesh.

Version 2.10:

# PGs resolved via global _resolve_process_group at runtime
mesh = DeviceMesh(...)  # PGs could be created later

Version 2.11:

# PGs now stored on DeviceMesh._pg_registry; must exist at mesh creation
dist.init_process_group(...)  # Must be called before creating mesh
mesh = DeviceMesh(...)

Distributed (DTensor)

DTensor.to_local() backward now converts Partial placements to Replicate by default when grad_placements is not provided. (#173454)

Previously, calling to_local() on a Partial DTensor would preserve the Partial placement in the backward gradient, which could produce incorrect gradients when combined with from_local(). Now, the backward pass automatically maps Partial forward placements to Replicate gradient placements, matching the behavior of from_local().

Users who relied on the previous behavior (where to_local() backward preserved Partial gradients) may see different gradient values. To ensure correctness, explicitly pass grad_placements to to_local().

Version 2.10:

# Partial placement preserved in backward — could produce incorrect gradients
local_tensor = partial_dtensor.to_local()

Version 2.11:

# Partial → Replicate in backward by default (correct behavior)
local_tensor = partial_dtensor.to_local()
# Or explicitly specify grad_placements for full control:
local_tensor = partial_dtensor.to_local(grad_placements=[Replicate()])

_PhiloxState.seed and _PhiloxState.offset now return torch.Tensor instead of int (#173876)

The DTensor RNG internal _PhiloxState class changed its seed and offset properties to return tensors instead of Python ints, and the setters now expect tensors. This makes the RNG state compatible with PT2 tracing (the previous .item() calls were not fake-tensor friendly).

Code that directly reads _PhiloxState.seed or _PhiloxState.offset and treats them as ints will break. Call .item() to get the int value. When setting, wrap the value in a tensor.

Version 2.10:

from torch.distributed.tensor._random import _PhiloxState

philox = _PhiloxState(state)
seed: int = philox.seed          # returned int
philox.offset = 42               # accepted int

Version 2.11:

from torch.distributed.tensor._random import _PhiloxState

philox = _PhiloxState(state)
seed: int = philox.seed.item()   # now returns Tens...
Read more

PyTorch 2.10.0 Release

Choose a tag to compare

@slayton58 slayton58 released this 21 Jan 17:05
449b176

PyTorch 2.10.0 Release Notes

Highlights

Python 3.14 support for torch.compile(). Python 3.14t (freethreaded build) is experimentally supported as well.
Reduced kernel launch overhead with combo-kernels horizontal fusion in torchinductor
A new varlen_attn() op providing support for ragged and packed sequences
Efficient eigenvalue decompositions with DnXgeev
torch.compile() now respects use_deterministic_mode
DebugMode for tracking dispatched calls and debugging numerical divergence - This makes it simpler to track down subtle numerical bugs.
Intel GPUs support: Expand PyTorch support to the latest Panther Lake on Windows and Linux by enabling FP8 (core ops and scaled matmul) and complex MatMul support, and extending SYCL support in the C++ Extension API for Windows custom ops.

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Backwards Incompatible Changes

Dataloader Frontend

  • Removed unused data_source argument from Sampler (#163134). This is a no-op, unless you have a custom sampler that uses this argument. Please update your custom sampler accordingly.
  • Removed deprecated imports for torch.utils.data.datapipes.iter.grouping (#163438). from torch.utils.data.datapipes.iter.grouping import SHARDING_PRIORITIES, ShardingFilterIterDataPipe is no longer supported. Please import from torch.utils.data.datapipes.iter.sharding instead.

torch.nn

  • Remove Nested Jagged Tensor support from nn.attention.flex_attention (#161734)

ONNX

  • fallback=False is now the default in torch.onnx.export (#162726)
  • The exporter now uses the dynamo=True option without fallback. This is the recommended way to use the ONNX exporter. To preserve 2.9 behavior, manually set fallback=True in the torch.onnx.export call.

Release Engineering

  • Rename pytorch-triton package to triton (#169888)

Deprecations

Distributed

  • DeviceMesh
    • Added a warning for slicing flattened dim from root mesh and types for _get_slice_mesh_layout (#164993)

We decided to deprecate an existing behavior which goes against the PyTorch design principle (explicit over implicit) for device mesh slicing of flattened dim.

Version <2.9

import torch
from torch.distributed.device_mesh import

device_type = (
    acc.type
    if (acc := torch.accelerator.current_accelerator(check_available=True))
    else "cpu"
)
mesh_shape = (2, 2, 2)
mesh_3d = init_device_mesh(
    device_type, mesh_shape, mesh_dim_names=("dp", "cp", "tp")
)

mesh_3d["dp", "cp"]._flatten()
mesh_3["dp_cp"]  # This comes with no warning

Version >=2.10

import torch
from torch.distributed.device_mesh import

device_type = (
    acc.type
    if (acc := torch.accelerator.current_accelerator(check_available=True))
    else "cpu"
)
mesh_shape = (2, 2, 2)
mesh_3d = init_device_mesh(
    device_type, mesh_shape, mesh_dim_names=("dp", "cp", "tp")
)

mesh_3d["dp", "cp"]._flatten()
mesh_3["dp_cp"]  # This will come with a warning because it implicitly change the state of the original mesh. We will eventually remove this behavior in future release. User should do the bookkeeping of flattened mesh explicitly.

Ahead-Of-Time Inductor (AOTI)

  • Move from/to to torch::stable::detail (#164956)

JIT

  • torch.jit is not guaranteed to work in Python 3.14. Deprecation warnings have been added to user-facing torch.jit API (#167669).

torch.jit should be replaced with torch.compile or torch.export.

ONNX

  • The dynamic_axes option in torch.onnx.export is deprecated (#165769)

Users should supply the dynamic_shapes argument instead. See https://docs.pytorch.org/docs/stable/export.html#expressing-dynamism for more documentation.

Profiler

  • Deprecate export_memory_timeline method (#168036)

The export_memory_timeline method in torch.profiler is being deprecated in favor of the newer memory snapshot API (torch.cuda.memory._record_memory_history and torch.cuda.memory._export_memory_snapshot). This change adds the deprecated decorator from typing_extensions and updates the docstring to guide users to the recommended alternative.

New Features

Autograd

  • Allow setting grad_dtype on leaf tensors (#164751)
  • Add Default Autograd Fallback for PrivateUse1 in PyTorch (#165315)
  • Add API to annotate disjoint backward for use with torch.utils.checkpoint.checkpoint (#166536)

Complex Frontend

  • Add ComplexTensor subclass (#167621)

Composability

  • Support autograd in torch.cond (#165908)

cuDNN

  • BFloat16 support added to cuDNN RNN (#164411)
  • [cuDNN][submodule] Upgrade to cuDNN frontend 1.16.1 (#170591)

Distributed

  • LocalTensor:

    • LocalTensor is a powerful debugging and simulation tool in PyTorch's distributed tensor ecosystem. It allows you to simulate distributed tensor computations across multiple SPMD (Single Program, Multiple Data) ranks on a single process. This is incredibly valuable for: 1) debugging distributed code without spinning up multiple processes; 2) understanding DTensor behavior by inspecting per-rank tensor states; 3) testing DTensor operations with uneven sharding across ranks; 4) rapid prototyping of distributed algorithms. Note that LocalTensor is designed for debugging purposes only. It has significant overhead and is not suitable for production distributed training.
    • LocalTensor is a torch.Tensor subclass that internally holds a mapping from rank IDs to local tensor shards. When you perform a PyTorch operation on a LocalTensor, the operation is applied independently to each local shard, mimicking distributed computation (LocalTensor simulates collective operations locally without actual network communication.). LocalTensorMode is the context manager that enables LocalTensor dispatch. It intercepts PyTorch operations and routes them appropriately. The @maybe_run_for_local_tensor decorator is essential for handling rank-specific logic when implementing distributed code.
    • To get started with LocalTensor, users import from torch.distributed._local_tensor, initialize a fake process group, and wrap their distributed code in a LocalTensorMode context. Within this context, DTensor operations automatically produce LocalTensors.
    • PRs: (#164537, #166595, #168110,#168314,#169088,#169734)
  • c10d:

    • New shrink_group implementation to expose ncclCommShrink API (#164518)

Dynamo

  • torch.compile now fully works in Python 3.14 (#167384)
  • Add option to error or disable applying side effects (#167239)
  • Config flag (skip_fwd_side_effects_in_bwd_under_checkpoint) to allow eager and compile activation-checkpointing divergence for side-effects (#165775)
  • torch._higher_order_ops.print for enabling printing without graph breaks or reordering (#167571)

FX

  • Added node metadata annotation API
  • Disable preservation of node metadata when enable=False (#164772)
  • Annotation should be mapped across submod (#165202)
  • Annotate bw nodes before eliminate dead code (#165782)
  • Add logging for debugging annotation (#165797)
  • Override metadata on regenerated node in functional mode (#166200)
  • Skip copying custom meta for gradient accumulation nodes; tag with is_gradient_acc=True (#167572)
  • Add metadata hook for all nodes created in runtime_assert pass (#169497)
  • Update `gm.print...
Read more

PyTorch 2.9.1 Release, bug fix release

Choose a tag to compare

@atalman atalman released this 12 Nov 19:27
d38164a

This release is meant to fix the following issues (regressions / silent correctness):

Tracked Regressions

Significant Memory Regression in F.conv3d with bfloat16 Inputs in PyTorch 2.9.0 (#166643)
This release provides work around this issue. If you are impacted please install nvidia-cudnn package version 9.15+ from pypi. (#166480) (#167111)

Torch.compile

Fix Inductor bug when compiling Gemma (#165601)
Fix InternalTorchDynamoError in bytecode_transformation (#166036)
Fix silent correctness error_on_graph_break bug where non-empty checkpoint results in unwanted graph break resumption (#166586)
Improve performance by avoiding recompilation with mark_static_address with cudagraphs (#162208)
Improve performance by caching get_free_symbol_uses in torch inductor (#166338)
Fix fix registration design for inductor graph partition for vLLM (#166458) (#165815) (#165514)
Fix warning spamming in torch.compile (#166993)
Fix exception related to uninitialized tracer_output variable (#163169)
Fix crash in torch.bmm and torch.compile with PyTorch release 2.9.0 (#166457)

Other

Fix warning spamming on new APIs to control TF32 behavior (#166956)
Fix distributed crash with non-contiguous gather inputs (#166181)
Fix indexing on large tensor causes invalid configuration argument (#166974)
Fix numeric issue in CUDNN_ATTENTION (#166912) (#166570)
Fix symmetric memory issue with fused_scaled_matmul_reduce_scatter (#165086)
Improve libtorch stable ABI documentation (#163899)
Fix image display on pypi project description section (#166404)

2.9 Release Notes

Choose a tag to compare

@seemethere seemethere released this 15 Oct 17:12
0fabc3b

PyTorch 2.9.0 Release Notes

Highlights

Unstable (API-Unstable)
Updates to the stable libtorch ABI for third-party C++/CUDA extensions
Symmetric memory that enables easy programming of multi-GPU kernels
The ability to arbitrarily toggle error or resume on graph breaks in torch.compile
Expanded wheel variant support to include ROCm, XPU and CUDA 13
FlexAttention enablement on Intel GPUs
Flash decoding optimization based on FlexAttention on X86 CPU
ARM Platform improvements and optimizations
Enablement of Linux aarch64 binary wheel builds across all supported CUDA versions

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Backwards Incompatible Changes

Min supported Python version is now 3.10 (#162310)

The minimum version of Python required for PyTorch 2.9.0 is 3.10. We also have 3.14 and 3.14t available as preview with this release.

Undefined behavior when an output of a custom operator shares storage with an input

This is a reminder that outputs of PyTorch custom operators (that are registered using the torch.library or TORCH_LIBRARY APIs) are not allowed to return Tensors that share storage with input tensors. The violation of this condition leads to undefined behavior: sometimes the result will be correct, sometimes it will be garbage.

After #163227, custom operators that violated this condition that previously returned correct results under torch.compile may now return silently incorrect results under torch.compile. Because this is changing the behavior of undefined behavior, we do not consider this to be a bug, but we are still documenting it in this section as a "potentially unexpected behavior change".

This is one of the conditions checked for by torch.library.opcheck and is mentioned in The Custom Operators Manual

More details

Outputs of PyTorch custom operators are not allowed to return Tensors that share storage with input tensors

For example, the following two custom operators are not valid custom operators:

@torch.library.custom_op("mylib::foo", mutates_args=())
def foo(x: torch.Tensor) -> torch.Tensor:
    # the result of `foo` must not directly be an input to foo.
    return x

@torch.library.custom_op("mylib::bar", mutates_args=())
def bar(x: torch.Tensor) -> torch.Tensor:
    # the result of bar must not be a view of an input of bar
    return x.view(-1)

The easiest workaround is to add an extra .clone() to the outputs:

@torch.library.custom_op("mylib::foo", mutates_args=())
def foo(x: torch.Tensor) -> torch.Tensor:
    return x.clone()

@torch.library.custom_op("mylib::bar", mutates_args=())
def bar(x: torch.Tensor) -> torch.Tensor:
    return x.view(-1).clone()

A common way to get into this situation is for a user to want to create a custom operator that sometimes mutates the input in-place and sometimes returns a new Tensor, like in the following example.

@torch.library.custom_op("mylib::baz", mutates_args=["x"])
def baz(x: torch.Tensor) -> torch.Tensor:
    if inplace:
        x.sin_()
        return x
    else:
        return x.sin()

This dynamism is not supported and leads to undefined behavior. The workaround is to split the custom operator into two custom operators, one that always mutates the input in-place, and another that always returns a new Tensor.

@torch.library.custom_op("mylib::baz_outplace", mutates_args=())
def baz_outplace(x: torch.Tensor) -> torch.Tensor:
    return x.sin()

@torch.library.custom_op("mylib::baz_inplace", mutates_args=["x"])
def baz_inplace(x: torch.Tensor) -> torch.Tensor:
    x.sin_()

def baz(x):
    if inplace:
        baz_inplace(x)
        return x
    else:
        return baz_outplace(x)

Build metal kernels of MacOS-14+ and remove all pre-MacOS-14 specific logic, requires MacOS-14+ going forward (#159733, #159912)

PyTorch MPS is only supported on MacOS-14 or later. If you need to use MPS on MacOS Ventura, please avoid updating to Python-3.9 or above

Upgrade to DLPack 1.0 (#145000)

This upgrade is doing the same BC-breaking changes as the DLPack release. Objects in torch.utils.dlpack have been updated to reflect these changes, such as DLDeviceType.

See the PR for details on the exact changes and how to update your code.

Raise appropriate errors in torch.cat (#158249)

torch.cat now raises ValueError, IndexError or TypeError where appropriate instead of the generic RuntimeError. If you code was catching these errors, you can update to catch the new error type.

Default to dynamo=True for ONNX exporter (#159646, #162726)

Previously torch.onnx.export(...) used the legacy TorchScript exporter if no arguments were provied. The ONNX exporter now uses the newer torch.export.export pipeline by default (dynamo=True). This change improves graph fidelity and future-proofs exports, but may surface graph capture errors that were previously masked or handled differently.

Previously in torch 2.8.0:

# API calls the legacy exporter with dynamo=False
torch.onnx.export(...)

Now in torch 2.9.0:

# To preserve the original behavior
torch.onnx.export(..., dynamo=False)

# Export onnx model through torch.export.export
torch.onnx.export(...)

Recommendation: first try the new default; only fall back if you hit blocking issues and report them upstream.
Long term solution: fix the root cause instead of relying on fallback or TorchScript exporter.

Switch off runtime asserts by default in Export in favor of a shape guards function (#160111, #161178, #161794)

To enable runtime asserts, use export(..., prefer_deferred_runtime_asserts_over_guards=True). Also kills the allow_complex_guards_as_runtime_asserts flag, merging it into the former option.

Additionally, exported_program.module() will generate a call to a _guards_fn submodule that will run additional checks on inputs. Users who do not want this behavior can either remove this call in the graph, or do exported_program.module(check_guards=False) to avoid the generation.

Set default opset to 20 in ONNX (#158802)

Opset 20 enables newer operator definitions. If your tooling or downstream runtime only supports opset 18, pin it explicitly. For the latest ONNX operators, you can experiment with opset 23.

Previously in torch 2.8.0:

# opset_version=18
torch.onnx.export(...)

Now in torch 2.9.0:

# To preserve the original behavior
torch.onnx.export(..., opset_version=18)

# New: opset_version=20
torch.onnx.export(...)

# Use the latest supported opset: opset_version=23
torch.onnx.export(..., opset_version=23)

Drop draft_export in exporter API (#161454, #162225)

Remove implicit draft tracing from the default exporter path, achieving clearer behaviour and faster failures.
The expensive torch.export.draft_export diagnostic path is no longer auto-invoked (which could take hours on large models). You can still opt in for deep diagnostics:

Previously in torch 2.8.0:

# If both torch.export.export(..., strict=False) and
# torch.export.export(..., strict=True) fail to capture
# the model graph, torch.export.draft_export(...) will be triggered,
# and uses real tensor to trace/export the model.
#
# Inside export_to_onnx.py:
#  ... torch.onnx.export(..., dynamo=True)
python export_to_onnx.py

Now in torch 2.9.0:

# To trigger torch.export.draft_export once
# torch.export.export strict=False/True both
# fail:

TORCH_ONNX_ENABLE_DRAFT_EXPORT=True python export_to_onnx.py

Remove torch.onnx.dynamo_export and the onnxrt torch compile backend (#158130, #158258)

torch.onnx.dynamo_export is removed. Please use torch.onnx.export instead.
The experimental ONNX Runtime compile backend (torch.compile(backend="onnxrt")) is no longer supported.

Remove torch.onnx.enable_fake_mode (#161222)

The dynamo=True mode uses FakeTensors by default which is memory efficient.

Some public facing ONNX utility APIs for the TorchScript based exporter are now private (#161323)

Deprecated members in torch.onnx.verification are removed. Previously private torch.onnx.symbolic_opsets* functions will no longer be accessible. Consider making a copy of the source code if you need to access any private functions for compatibility with the TorchScript based exporter.

Remove torch.onnx.symbolic_caffe2 (#157102)

Support for caffe2 in the ONNX exporter has ended and is removed.

Remove /d2implyavx512upperregs flag that slows build (#159431)

Re-introduced AVX512 optimizations for Windows VS2022 builds, may cause issues with specific versions of VS2022, see #145702

Add ScalarType to shim conversion and stable::Tensor.scalar_type (#160557)

Before, user extensions could only in abstract...

Read more