[Intel GPU] Allow XPU device in copy, cdist, index_put_impl #130088

ZhiweiYan-96 · 2024-07-04T03:15:42Z

Motivation

copy, cdist, index_put_impl operators use op_stub for runtime dispatching inside operators. Extra device list is inside them to assure the accuracy, while XPU is not in them. This PRs make them allow XPU as a supported device.

Stack from ghstack (oldest at bottom):

-> [Intel GPU] Allow XPU device in copy, cdist, index_put_impl #130088

cc @gujinghui @EikanWang @fengyuan14 @guangyey

[ghstack-poisoned]

pytorch-bot · 2024-07-04T03:15:45Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/130088

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 89c3b89 with merge base 6cbb143 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 7eedcd5 Pull Request resolved: #130088

[ghstack-poisoned]

ghstack-source-id: 5274ff1 Pull Request resolved: #130088

[ghstack-poisoned]

ghstack-source-id: b7186b9 Pull Request resolved: #130088

[ghstack-poisoned]

ghstack-source-id: 7b7f255 Pull Request resolved: #130088

[ghstack-poisoned]

ghstack-source-id: 21876e0 Pull Request resolved: #130088 Add XPU into device list of cdist_impl ghstack-source-id: 21876e0 Pull Request resolved: #130412

[ghstack-poisoned]

ghstack-source-id: e657758 Pull Request resolved: #130088 Add XPU into device list of cdist_impl ghstack-source-id: e657758 Pull Request resolved: #130412

EikanWang · 2024-07-30T03:11:02Z

@ZhiweiYan-96 , please add test cases.

ZhiweiYan-96 · 2024-07-30T07:14:15Z

hi @EikanWang The test case depends on the structure codegen change in torch-xpu-ops repo. We may add test case accompanied with xpu-ops commit change after PR in torch-xpu-ops merged.

EikanWang · 2024-07-31T09:36:05Z

hi @EikanWang The test case depends on the structure codegen change in torch-xpu-ops repo. We may add test case accompanied with xpu-ops commit change after PR in torch-xpu-ops merged.

Thanks for the information. @atalman FYI

albanD · 2024-07-31T15:24:04Z

aten/src/ATen/native/Copy.cpp

    device_type = kHIP;
  } else if (iter.device_type(1) == kMPS) {
    device_type = kMPS;
+  } else if (iter.device_type(1) == kXPU){


Should we make this accelerator device check?

hi @albanD We may need keep this device check as this code is intended to use accelerator kernel once any of tensor(src/dst) is on accelerator device. I think XPU logic also should have same logic like cuda/mps here. Otherwise, we may fail to propagate the device into copy_stub following. If my understanding here is wrong, could you please correct me? Thanks.

FYI (copy_stub)

https://github.com/pytorch/pytorch/pull/130088/files#diff-5920abc01985a724ffb7a8f57b02a373a2e816615b344f0bda8a7a80bee833a0R310

From a closer look, I retract my statemnet. The only difference with handling any accelerator here would be that PrivateUse1 would be handled as well. But that mens that this would force the copy kernel to be done via copy_stub for privateuse1 as opposed to today where _copy_from() is being called and can be overwritten as any other dispatcher op.

albanD · 2024-07-31T15:24:47Z

aten/src/ATen/native/ReduceOps.cpp

    at::OptionalIntArrayRef dim, const std::optional<Scalar>& correction_opt,
    bool keepdim, bool take_sqrt) {
-  TORCH_CHECK(self.device().is_cpu() || self.device().is_cuda(),
+  TORCH_CHECK(self.device().is_cpu() || self.device().is_cuda() || self.device().is_xpu(),


Update error message here and below

Thanks for your kindly reminding! I would add the XPU into error messages

Has updated the error message, thank you so much for the comments!

[ghstack-poisoned]

ghstack-source-id: 36a7439 Pull Request resolved: #130088 Add XPU into device list of cdist_impl ghstack-source-id: 36a7439 Pull Request resolved: #130412

EikanWang · 2024-08-04T10:01:04Z

aten/src/ATen/native/ReduceOps.cpp

-  TORCH_CHECK(self.device().is_cpu() || self.device().is_cuda(),
-              "std and var only supports tensors on a CPU or CUDA device, but got: ",
+  TORCH_CHECK(self.device().is_cpu() || self.device().is_cuda() || self.device().is_xpu(),
+              "std and var only supports tensors on a CPU or CUDA/XPU device, but got: ",


How about?

Suggested change

"std and var only supports tensors on a CPU or CUDA/XPU device, but got: ",

"std and var supports tensors on CPU, CUDA, or XPU devices only, but got: ",

I have modified the message here, thanks for the suggestions!

[ghstack-poisoned]

ghstack-source-id: 7fed97a Pull Request resolved: #130088 Add XPU into device list of cdist_impl ghstack-source-id: 7fed97a Pull Request resolved: #130412

albanD

Ok!
Thanks for the update

ZhiweiYan-96 · 2024-08-06T01:21:38Z

@pytorchbot merge

pytorchmergebot · 2024-08-06T01:23:20Z

Merge failed

Reason: This PR needs a release notes: label
If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Details for Dev Infra team

Raised by workflow job

ZhiweiYan-96 · 2024-08-06T01:48:49Z

@pytorchbot merge

pytorchmergebot · 2024-08-06T01:50:26Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Update

527eadc

[ghstack-poisoned]

ZhiweiYan-96 mentioned this pull request Jul 4, 2024

[Intel GPU] Dispatch Stub support #130019

Closed

ZhiweiYan-96 mentioned this pull request Jul 4, 2024

[Intel GPU] xpu-ops codegen via backend whitelist #130082

Closed

ZhiweiYan-96 added a commit that referenced this pull request Jul 4, 2024

[Intel GPU] Add XPU into device list of copy_impl

4edcb74

ghstack-source-id: 7eedcd5 Pull Request resolved: #130088

ZhiweiYan-96 closed this Jul 4, 2024

ZhiweiYan-96 reopened this Jul 4, 2024

ZhiweiYan-96 marked this pull request as draft July 4, 2024 03:17

Update

e25eb71

[ghstack-poisoned]

pytorchbot added the open source label Jul 4, 2024

Update

2f2cc31

[ghstack-poisoned]

ZhiweiYan-96 added a commit that referenced this pull request Jul 4, 2024

[Intel GPU] Add XPU into device list of copy_impl

fa9b720

ghstack-source-id: 5274ff1 Pull Request resolved: #130088

Update

d91e743

[ghstack-poisoned]

ZhiweiYan-96 added a commit that referenced this pull request Jul 5, 2024

[Intel GPU] Add XPU into device list of copy_impl

e666f54

ghstack-source-id: b7186b9 Pull Request resolved: #130088

ZhiweiYan-96 added ciflow/xpu Run XPU CI tasks ciflow/trunk Trigger trunk jobs on your pull request labels Jul 5, 2024

Update

7867c79

[ghstack-poisoned]

Update

67319f1

[ghstack-poisoned]

ZhiweiYan-96 added a commit that referenced this pull request Jul 9, 2024

[Intel GPU] Add XPU into device list of copy_impl

f6a4f1b

ghstack-source-id: 7b7f255 Pull Request resolved: #130088

This was referenced Jul 10, 2024

[Intel GPU]Add XPU into device list of cdist_impl #130411

Closed

[Intel GPU]Add XPU into device list of cdist_impl #130412

Closed

ZhiweiYan-96 requested a review from EikanWang July 10, 2024 06:01

Update

0f5ff19

[ghstack-poisoned]

ZhiweiYan-96 added the module: xpu Intel XPU related issues label Jul 11, 2024

ZhiweiYan-96 marked this pull request as ready for review July 12, 2024 06:17

EikanWang approved these changes Jul 15, 2024

View reviewed changes

Update

741e409

[ghstack-poisoned]

ZhiweiYan-96 added a commit that referenced this pull request Jul 26, 2024

[Intel GPU] Add XPU into device list of copy_impl

a218307

ghstack-source-id: 21876e0 Pull Request resolved: #130088 Add XPU into device list of cdist_impl ghstack-source-id: 21876e0 Pull Request resolved: #130412

Update

7e1725d

[ghstack-poisoned]

Update

a05d3b3

[ghstack-poisoned]

ZhiweiYan-96 added a commit that referenced this pull request Jul 26, 2024

[Intel GPU] Add XPU into device list of copy_impl

73c1082

ghstack-source-id: e657758 Pull Request resolved: #130088 Add XPU into device list of cdist_impl ghstack-source-id: e657758 Pull Request resolved: #130412

albanD reviewed Jul 31, 2024

View reviewed changes

Update

0ccfcbb

[ghstack-poisoned]

ZhiweiYan-96 added a commit that referenced this pull request Aug 3, 2024

[Intel GPU] Add XPU into device list of copy_impl

8e46b14

ghstack-source-id: 36a7439 Pull Request resolved: #130088 Add XPU into device list of cdist_impl ghstack-source-id: 36a7439 Pull Request resolved: #130412

EikanWang reviewed Aug 4, 2024

View reviewed changes

Update

89c3b89

[ghstack-poisoned]

ZhiweiYan-96 added a commit that referenced this pull request Aug 5, 2024

[Intel GPU] Add XPU into device list of copy_impl

b18dc72

ghstack-source-id: 7fed97a Pull Request resolved: #130088 Add XPU into device list of cdist_impl ghstack-source-id: 7fed97a Pull Request resolved: #130412

EikanWang requested a review from albanD August 5, 2024 03:24

albanD approved these changes Aug 5, 2024

View reviewed changes

pytorchmergebot added the merging label Aug 6, 2024

pytorchmergebot removed the merging label Aug 6, 2024

ZhiweiYan-96 added the topic: not user facing topic category label Aug 6, 2024

pytorchmergebot added the merging label Aug 6, 2024

pytorchmergebot added the Merged label Aug 6, 2024

pytorchmergebot closed this in 2f16e68 Aug 6, 2024

pytorchmergebot removed the merging label Aug 6, 2024

github-actions bot deleted the gh/ZhiweiYan-96/18/head branch September 5, 2024 02:01

chunhuanMeng mentioned this pull request Oct 30, 2024

[Intel GPU] Allow XPU device in pdist, cdist #139266

Closed

	"std and var only supports tensors on a CPU or CUDA/XPU device, but got: ",
	"std and var supports tensors on CPU, CUDA, or XPU devices only, but got: ",

[Intel GPU] Allow XPU device in copy, cdist, index_put_impl #130088

[Intel GPU] Allow XPU device in copy, cdist, index_put_impl #130088

Uh oh!

Conversation

ZhiweiYan-96 commented Jul 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Uh oh!

pytorch-bot bot commented Jul 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/130088

✅ No Failures

Uh oh!

EikanWang commented Jul 30, 2024

Uh oh!

ZhiweiYan-96 commented Jul 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EikanWang commented Jul 31, 2024

Uh oh!

albanD Jul 31, 2024

Choose a reason for hiding this comment

Uh oh!

ZhiweiYan-96 Aug 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

EikanWang Aug 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

albanD Aug 5, 2024

Choose a reason for hiding this comment

Uh oh!

albanD Jul 31, 2024

Choose a reason for hiding this comment

Uh oh!

ZhiweiYan-96 Aug 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ZhiweiYan-96 Aug 3, 2024

Choose a reason for hiding this comment

Uh oh!

EikanWang Aug 4, 2024

Choose a reason for hiding this comment

Uh oh!

ZhiweiYan-96 Aug 5, 2024

Choose a reason for hiding this comment

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

ZhiweiYan-96 commented Aug 6, 2024

Uh oh!

pytorchmergebot commented Aug 6, 2024

Merge failed

Uh oh!

ZhiweiYan-96 commented Aug 6, 2024

Uh oh!

pytorchmergebot commented Aug 6, 2024

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

ZhiweiYan-96 commented Jul 4, 2024 •

edited

Loading

pytorch-bot bot commented Jul 4, 2024 •

edited

Loading

ZhiweiYan-96 commented Jul 30, 2024 •

edited

Loading

ZhiweiYan-96 Aug 2, 2024 •

edited

Loading

EikanWang Aug 3, 2024 •

edited

Loading

ZhiweiYan-96 Aug 2, 2024 •

edited

Loading