🚀 The feature, motivation and pitch

This RFC is a consequence of #114835 in terms of XPU ATen operation upstreaming options.

To support the Inductor benchmarks, the Inductor needs to fall 91 operations back to ATen

• 9 ATen operations utilize libraries(OneDNN & OneMKL)
• 46 ATen operations are device-agnostic
• 36 ATen operations need SYCL code

Regarding the library-related and device-agnostic operations, they are straightforward to be upstreamed as in-tree directly.

In terms of the 36 ATen operations that need SYCL implementation, the line of code is about 16, 000. And we have different upstreaming options.

• Option 1: Start from out-of-tree as a git submodule and move in-tree eventually
• Option 2: Start from out-of-tree as a separate branch and move in-tree eventually
• Option 3: Upstream in-tree directly

All three options follow the same implementation method; the primary difference lies in the code location.

To minimize community overhead before XPU gains wider adoption and to enable agile XPU development, we will start with an out-of-tree approach. Thus, at this moment, Option 1 and Option 2 are preferred.

However, hosting the 36 operations with 16,000 lines of code in a separate branch is burdensome:

• The synchronization/rebase effort for a separate branch is greater than that for a git submodule.
• A separate branch would introduce additional overhead in the release process and cadence compared to the git submodule option.
• Hosting XPU ATen operations in a separate branch could confuse PyTorch users, especially if the runtime and some ATen operations are upstreamed, while others are in a separate branch.

Considering these factors, we strongly recommend adopting the git submodule(Option 1) approach to host XPU ATen operations.

Alternatives

No response

Additional context

No response

cc @malfet @seemethere @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @atalman @albanD