Introduce XPU implementation for PyTorch ATen operators #120891

fengyuan14 · 2024-02-29T05:58:26Z

As a follow-up to #114835 and #119682, we add limited ATen operators implementation for XPU. With this PR, the blocking issue for oneDNN operations and Inductor XPU backend will be resolved as the two components depend on these operations to support its basic features, respectively.

The added ATen operators include:

copy_, _to_copy, _copy_from_and_resize, , clone
view, view_as_real, view_as_complex,
as_strided, _reshape_alias, resize_, resize_as_,
add/add_, sub/sub_, mul/mul_, div/div_, abs,
empty, empty_strided,
fill_, zeros_.

pytorch-bot · 2024-02-29T05:58:30Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/120891

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (5 Unrelated Failures)

As of commit 23aff01 with merge base c4486d3 ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

rocm / linux-focal-rocm6.0-py3.8 / test (default, 1, 6, linux.rocm.gpu.2) (gh)
inductor/test_benchmark_fusion.py::BenchmarkFusionCudaTest::test_avoid_register_spilling_cuda
rocm / linux-focal-rocm6.0-py3.8 / test (default, 4, 6, linux.rocm.gpu.2) (gh)
inductor/test_pad_mm.py::PadMMTest::test_cat_pad_mm_dyn_m

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

rocm / linux-focal-rocm6.0-py3.8 / test (default, 2, 6, linux.rocm.gpu.2) (gh)
inductor/test_max_autotune.py::TestMaxAutotune::test_cat_addmm
rocm / linux-focal-rocm6.0-py3.8 / test (default, 3, 6, linux.rocm.gpu.2) (gh)
inductor/test_select_algorithm.py::TestSelectAlgorithm::test__int_mm
rocm / linux-focal-rocm6.0-py3.8 / test (default, 6, 6, linux.rocm.gpu.2) (gh)
test_foreach.py::TestForeachCUDA::test_big_num_tensors__foreach_norm_use_cuda_graph_True_cuda_float32

This comment was automatically generated by Dr. CI and updates every 15 minutes.

EikanWang

Please add test cases for XPU ops.

caffe2/CMakeLists.txt

malfet

Let's not add additional submodules, but figure out some sort of a soft checkout mechanism

Also, this repository feels like a huge copy-n-paste of an already existing code (i.e. src/aten/sycl/Loops.h looks very similar to aten/src/ATen/cuda/Loops.cuh, which makes me wonder if some of this code can be better reused between the two somehow)

.gitmodules

fengyuan14 · 2024-03-12T01:15:00Z

Also, this repository feels like a huge copy-n-paste of an already existing code (i.e. src/aten/sycl/Loops.h looks very similar to aten/src/ATen/cuda/Loops.cuh, which makes me wonder if some of this code can be better reused between the two somehow)

Hi, @malfet ,
Reasons of no sharing,

Sharing code will impact other backends. At the first stage, we follow a conservative approach, won't share kernels.
It takes non-trivial efforts to abstract device agnostic parts of kernels. Regarding performance, even we have same concurrency algorithm, HW specific divergence is inevitable, like different heuristic (shapes, broadcast, condition of vectorization, A32/A64 addressing ...), different register pressure (different number of General Programming Register. Small number of GRF causes register spill when using too many intermediate local variables in kernel), different native vectorization capability and so on. We have tired Abstract common helpers from CUDA implementation for code reusing in upcoming SYCL kernels #117234. The PR makes helpers used in Loops general. But it is not enough to help share all things in Loops among backends. It takes some time to make a kernel totally general.

Thanks.

EikanWang · 2024-03-12T02:17:39Z

Let's not add additional submodules, but figure out some sort of a soft checkout mechanism

@malfet , thanks for your comments. May I know your concerns about the git submodule integration mechanism? Regarding the soft checkout mechanism, we might consider using CMake's ExternalProject or custom target as a solution. This would allow us to clone XPU Aten operations during the build process if USE_XPU is set to ON. Apart from the approaches, do you have any other suggestions or proposals?

.ci/pytorch/build.sh

test/test_xpu.py

caffe2/CMakeLists.txt

EikanWang

Please update the test cases a little bit

atalman · 2024-03-18T20:44:11Z

@pytorchbot rebase

pytorchmergebot · 2024-03-18T20:45:45Z

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

pytorchmergebot · 2024-03-18T20:45:50Z

Successfully rebased fengyuan/out-of-tree-xpu-ops onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout fengyuan/out-of-tree-xpu-ops && git pull --rebase)

Signed-off-by: Feng Yuan <[email protected]>

…to reuse cases. Signed-off-by: Feng Yuan <[email protected]>

Signed-off-by: Feng Yuan <[email protected]>

…and commit info Signed-off-by: Feng Yuan <[email protected]>

EikanWang · 2024-03-22T03:18:05Z

@pytorchbot merge

pytorchmergebot · 2024-03-22T03:20:00Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-03-22T03:20:14Z

Merge failed

Reason: 5 jobs have failed, first few of them are: .github/workflows/rocm.yml / linux-focal-rocm6.0-py3.8 / test (default, 1, 6, linux.rocm.gpu.2), .github/workflows/rocm.yml / linux-focal-rocm6.0-py3.8 / test (default, 2, 6, linux.rocm.gpu.2), .github/workflows/rocm.yml / linux-focal-rocm6.0-py3.8 / test (default, 3, 6, linux.rocm.gpu.2), .github/workflows/rocm.yml / linux-focal-rocm6.0-py3.8 / test (default, 4, 6, linux.rocm.gpu.2), .github/workflows/rocm.yml / linux-focal-rocm6.0-py3.8 / test (default, 6, 6, linux.rocm.gpu.2)

Details for Dev Infra team

Raised by workflow job

EikanWang · 2024-03-22T03:29:03Z

@pytorchbot merge -i

pytorchmergebot · 2024-03-22T03:30:47Z

Merge started

Your change will be merged while ignoring the following 5 checks: .github/workflows/rocm.yml / linux-focal-rocm6.0-py3.8 / test (default, 1, 6, linux.rocm.gpu.2), .github/workflows/rocm.yml / linux-focal-rocm6.0-py3.8 / test (default, 2, 6, linux.rocm.gpu.2), .github/workflows/rocm.yml / linux-focal-rocm6.0-py3.8 / test (default, 3, 6, linux.rocm.gpu.2), .github/workflows/rocm.yml / linux-focal-rocm6.0-py3.8 / test (default, 4, 6, linux.rocm.gpu.2), .github/workflows/rocm.yml / linux-focal-rocm6.0-py3.8 / test (default, 6, 6, linux.rocm.gpu.2)

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

As a follow-up to #114835 and #119682, we add limited ATen operators implementation for XPU. With this PR, the blocking issue for oneDNN operations and Inductor XPU backend will be resolved as the two components depend on these operations to support its basic features, respectively. The added ATen operators include: - `copy_`, `_to_copy`, `_copy_from_and_resize`, , `clone` - `view`, `view_as_real`, `view_as_complex`, - `as_strided`, `_reshape_alias`, `resize_`, `resize_as_`, - `add`/`add_`, `sub`/`sub_`, `mul`/`mul_`, `div`/`div_`, `abs`, - `empty`, `empty_strided`, - `fill_`, `zeros_`. Co-authored-by: Wang, Eikan <[email protected]> Pull Request resolved: #120891 Approved by: https://github.com/EikanWang, https://github.com/jgong5, https://github.com/gujinghui, https://github.com/atalman

)" This reverts commit 148a8de. Reverted #120891 on behalf of https://github.com/huydhn due to Sorry for reverting your change but I need to revert it to resolve a conflict in trunk #121794 (comment). Please help reland the change after ([comment](#120891 (comment)))

As a follow-up to #114835 and #119682, we add limited ATen operators implementation for XPU. With this PR, the blocking issue for oneDNN operations and Inductor XPU backend will be resolved as the two components depend on these operations to support its basic features, respectively. The added ATen operators include: - `copy_`, `_to_copy`, `_copy_from_and_resize`, , `clone` - `view`, `view_as_real`, `view_as_complex`, - `as_strided`, `_reshape_alias`, `resize_`, `resize_as_`, - `add`/`add_`, `sub`/`sub_`, `mul`/`mul_`, `div`/`div_`, `abs`, - `empty`, `empty_strided`, - `fill_`, `zeros_`. Co-authored-by: Wang, Eikan <[email protected]> Pull Request resolved: #120891 Approved by: https://github.com/EikanWang, https://github.com/jgong5, https://github.com/gujinghui, https://github.com/atalman

fengyuan14 requested review from EikanWang, gujinghui and jgong5 February 29, 2024 05:58

pytorch-bot bot added the topic: not user facing topic category label Feb 29, 2024

pytorchbot added the open source label Feb 29, 2024

EikanWang requested changes Feb 29, 2024

View reviewed changes

caffe2/CMakeLists.txt Outdated Show resolved Hide resolved

caffe2/CMakeLists.txt Show resolved Hide resolved

caffe2/CMakeLists.txt Show resolved Hide resolved

EikanWang added the ciflow/xpu Run XPU CI tasks label Feb 29, 2024

jgong5 approved these changes Mar 1, 2024

View reviewed changes

EikanWang approved these changes Mar 8, 2024

View reviewed changes

gujinghui approved these changes Mar 8, 2024

View reviewed changes

fengyuan14 marked this pull request as ready for review March 8, 2024 12:09

fengyuan14 requested a review from a team as a code owner March 8, 2024 12:09

EikanWang requested review from albanD, atalman and malfet March 10, 2024 01:01

malfet requested changes Mar 11, 2024

View reviewed changes

.gitmodules Outdated Show resolved Hide resolved

albanD reviewed Mar 12, 2024

View reviewed changes

.ci/pytorch/build.sh Outdated Show resolved Hide resolved

test/test_xpu.py Outdated Show resolved Hide resolved

EikanWang reviewed Mar 13, 2024

View reviewed changes

caffe2/CMakeLists.txt Show resolved Hide resolved

EikanWang approved these changes Mar 13, 2024

View reviewed changes

EikanWang requested a review from malfet March 13, 2024 08:06

drisspg added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Mar 13, 2024

EikanWang requested a review from albanD March 14, 2024 05:35

fengyuan14 requested a review from mruberry as a code owner March 15, 2024 12:01

EikanWang reviewed Mar 18, 2024

View reviewed changes

fengyuan14 and others added 11 commits March 21, 2024 22:59

rebase torch-xpu-ops for new supported operators

44bb431

Signed-off-by: Feng Yuan <[email protected]>

Rebase torch-xpu-ops for new supported ops and add simple unit tests

2276402

Signed-off-by: Feng Yuan <[email protected]>

Use soft-dependency mechanism instead of PyTorch git submodule

55f0b83

Signed-off-by: Feng Yuan <[email protected]>

Improve XPU operators test with common operators test architecture

e00f8fe

Signed-off-by: Feng Yuan <[email protected]>

Enble device type test for TestXpu

fc33214

Signed-off-by: Feng Yuan <[email protected]>

Fix lint error

b91e8b0

Signed-off-by: Feng Yuan <[email protected]>

Remove test_dtype and will retrieve it once we have fancy test frame …

f18d3ae

…to reuse cases. Signed-off-by: Feng Yuan <[email protected]>

reduce test scope of XPU device

300f27d

Signed-off-by: Feng Yuan <[email protected]>

Pin torch-xpu-ops on a specified commit with separating CMake source …

6343368

…and commit info Signed-off-by: Feng Yuan <[email protected]>

Move xpu commit pin file from .github to third_party

e62d1a7

rebase to main

23aff01

EikanWang force-pushed the fengyuan/out-of-tree-xpu-ops branch from 3dea345 to 23aff01 Compare March 21, 2024 23:07

pytorchmergebot added the merging label Mar 22, 2024

pytorchmergebot removed the merging label Mar 22, 2024

pytorchmergebot added the merging label Mar 22, 2024

pytorchmergebot closed this in 09eb07b Mar 22, 2024

pytorchmergebot removed the merging label Mar 22, 2024

github-actions bot deleted the fengyuan/out-of-tree-xpu-ops branch April 22, 2024 01:54

EikanWang mentioned this pull request Jul 30, 2024

[Intel GPU] xpu-ops codegen via backend whitelist #130082

Closed

EikanWang mentioned this pull request Dec 4, 2024

Update torch-xpu-ops commit pin #141949

Closed

daisyden mentioned this pull request Dec 4, 2024

[RFC] Add Intel GPU Support to Torch Test Cases #142029

Open

Introduce XPU implementation for PyTorch ATen operators #120891

Introduce XPU implementation for PyTorch ATen operators #120891

Uh oh!

Conversation

fengyuan14 commented Feb 29, 2024 • edited by EikanWang Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Feb 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/120891

✅ You can merge normally! (5 Unrelated Failures)

Uh oh!

EikanWang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

malfet left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fengyuan14 commented Mar 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EikanWang commented Mar 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

EikanWang left a comment

Choose a reason for hiding this comment

Uh oh!

atalman commented Mar 18, 2024

Uh oh!

pytorchmergebot commented Mar 18, 2024

Uh oh!

pytorchmergebot commented Mar 18, 2024

Uh oh!

EikanWang commented Mar 22, 2024

Uh oh!

pytorchmergebot commented Mar 22, 2024

Merge started

Uh oh!

pytorchmergebot commented Mar 22, 2024

Merge failed

Uh oh!

EikanWang commented Mar 22, 2024

Uh oh!

pytorchmergebot commented Mar 22, 2024

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

fengyuan14 commented Feb 29, 2024 •

edited by EikanWang

Loading

pytorch-bot bot commented Feb 29, 2024 •

edited

Loading

malfet left a comment •

edited

Loading

fengyuan14 commented Mar 12, 2024 •

edited

Loading

EikanWang commented Mar 12, 2024 •

edited

Loading