🚀 The feature, motivation and pitch

Filing issue for enabling CUDA 12.6.2 due to an observed performance gap in PyTorch's Eager mode for plain matrix multiplications.
Confirmed that upgrading PyTorch to CTK 12.6.2 with CUBLAS 12.6.3.3 will boost the performance.

Issue can be used to track related PRs.

Reference - #132202

Docker Images & Windows AMI Update

• add CUDA 12.6 to conda docker image #138417
• add magma build for CUDA 12.6 builder#2020
• [aarch64] CUDA 12.6 installation for aarch64 dockerfile #138562
• Add CUDA 12.6 to Linux CD docker images #138563
• [aarch64] build cuda 12.6 manywheel dockers #139988
• Windows AMI - Cuda 12.6 test-infra#5880
• Windows AMI update cudnn-windows-x86_64-9.5.1.17_cuda12-archive test-infra#5924

CD Update

• Add CUDA 12.6 Linux Builds to Binaries Matrix #138899
• CUDA 12.6.2 windows build builder#2023
• Fix unhandled cuda error for Conda build with CUDA 12.6 builder#2025
• [BE]: Update 12.6 builds to CUDA 12.6.3 #141433
• Fix cuda arch full version for 12.6 #141976
• [BE] Update CUDNN to 9.4.0.58 #127925
• Fix Windows Magma Build builder#2032
• CUDA 12.6.2 windows build builder#2023
• Add windows CUDA 12.6 builds test-infra#6002
• Update generate_binary_build_matrix.py Add CUDA 12.6 entry in CUDA_CUDDN_VERSIONS test-infra#5955
• Add cuda 12.6 Linux to nightly binary matrix test-infra#5922
• Add windows CUDA 12.6 nightly builds #141805
• add cu126 torchvision and torchaudio aarch64 build test-infra#6062
• [aarch64] add CUDA 12.6 sbsa nightly binary #142335

CI Updates

• Fix cuda arch full version for 12.6 #141976
• add inductor-cu126 tag #141110
• Update CUDA installation script to 12.6.3 #141365
• Enable CUDA 12.6 OSS CI #140793

CUDA 12.1 Deprecation

• Deprecate CUDA 12.1 from binaries matrix test-infra#5904
• [EZ] Move slow job from CU12.1 to CU12.4 #142856
• Cuda 12.1 - Remove from trunk tests #143076
• Migrate pull jobs cuda12.1->cuda12.4 #141271
• Update Inductor benchmarks for CUDA 12.4
• [CI][CUDA][cuSPARSELt] cusparselt 0.6.3 and cu121 related cleanups #145696

cc @ptrblck @msaroufim @syed-ahmed @nWEIdia @atalman @malfet

Alternatives

No response

Additional context

No response