Skip to content

Conversation

@XiaobingSuper
Copy link
Collaborator

This PR is about:

  1. Make mkldnn reshape can share same memory fro plain format tensor.

  2. Add mkldnn transpose operator.

@pytorchbot pytorchbot added module: mkldnn Related to Intel IDEEP or oneDNN (a.k.a. mkldnn) integration module: operators labels Jun 19, 2019
@XiaobingSuper XiaobingSuper changed the title Mkldnn transpose Add aten mkldnn transpose Jun 19, 2019
@XiaobingSuper
Copy link
Collaborator Author

@XiaobingSuper
Copy link
Collaborator Author

@bddppq, please help review, thanks!

Copy link
Contributor

@bddppq bddppq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this!
I would like to add the inplace version aten::mkldnn_transpose_ and throw an error to indicate it's not supported

ideep::direct_copy::compute<AllocForMKLDNN>(x, y);
y.reshape({inferred_size.cbegin(), inferred_size.cend()});
ideep::tensor y{x};
y.reshape<AllocForMKLDNN>({inferred_size.cbegin(), inferred_size.cend()});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm if new shapes are as same as the old shapes, ideep::tensor::reshape will not create a copy and thus y will share the same memory as x. This could be messy since then there will be two aten opaque tensors sharing the same underlying ideep tensor.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bddppq We wanted to keep the semantics of reshape as much as possible here, i.e. do shallow copy on plain-format MKL-DNN tensor (layout of which is contiguous) and do deep copy otherwise (see https://github.com/intel/ideep/blob/d7304e0345c3f0647cb020a71682d680b9f0424a/include/ideep/tensor.hpp#L1100). For the shallow copy case, ideep::tensor y{x} would make y copy the metadata of x while share its underlying storage. This is the same semantics as the shallow copy of a native CPU tensor. So this should work fine here.

Copy link
Contributor

@bddppq bddppq Jun 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jgong5 It's fine to follow the semantics of cpu tensor to return the input tensor as output if the shapes are the same, but it's not fine to let two different aten opaque tensors to hold the same underlying ideep tensor.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bddppq But y and x are two different ideep::tensor object, right? They just share the same underlying storage while hold their own metadata. It is similar to how two CPU tensors share the same underlying storage while have their own metatdata. Make sense?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bddppq @XiaobingSuper OK. Sounds like we should call y.set_descriptor(x.get_descriptor()) explicitly after ideep::tensor y{x} in order to guarantee y uses a new MKL-DNN descriptor while sharing the underlying buffer of x. We should have wrapped an explicit shallow copy API in ideep to do such a thing actually.

Copy link
Contributor

@bddppq bddppq Jun 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jgong5 @XiaobingSuper I would say just add the following lines at the beginning of this function :-)

if (self.sizes() == inferred_size) {
  return self;
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I will change it. Thanks!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bddppq, @jgong5 , changed it, thanks!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bddppq y.reshape would create a new descriptor by calling set_descriptor() so additional y.set_descriptor(x.get_descriptor()) is not necessary here. With the recent change of @XiaobingSuper , the behavior of mkldnn_reshape looks like below, mimicking the semantics of reshape of native CPU tensor:

  1. If the requested shape is the same as self, return self directly.
  2. If the format of MKL-DNN tensor is public (or plain), the storage is contiguous, the returned MKL-DNN tensor shares the storage of self and copies all metadata of self with the new shape.
  3. For other situations, return a deep-copied tensor of self with the new shape.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jgong5 @XiaobingSuper This sounds good to me.

Copy link
Contributor

@bddppq bddppq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bddppq has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

zdevito pushed a commit to zdevito/ATen that referenced this pull request Jun 20, 2019
Summary:
This PR is about:

1.  Make mkldnn reshape can share same memory fro plain format tensor.

2.  Add mkldnn transpose operator.
Pull Request resolved: pytorch/pytorch#21943

Differential Revision: D15916063

Pulled By: bddppq

fbshipit-source-id: d1971c67341f277c1e80c1fa34e213b6c27f4062
@facebook-github-bot
Copy link
Contributor

@bddppq merged this pull request in b6f542f.

variants: function, method
device_guard: False

- func: mkldnn_transpose(Tensor self, int dim0, int dim1) -> Tensor
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this part of the public API? If you want to have helper functions prepend an underscore to their name!

@soumith
Copy link
Contributor

soumith commented Jun 24, 2019

@XiaobingSuper @Jianhui-Li @jgong5 @uyongw @gujinghui @mingfeima

Can you make sure this method is made private by prefixing an _ to the function.

In general, the mkldnn integration should be the exception and not the rule, and we shouldn't expose something like mkldnn_transpose into the Tensor API, it makes no sense.

cc: @bddppq for future review.

@XiaobingSuper XiaobingSuper deleted the mkldnn_transpose branch June 25, 2019 07:16
facebook-github-bot pushed a commit that referenced this pull request Jun 25, 2019
Summary:
This PR is to make mkldnn reshape and transpose not exposes as Tensor API, please see the comments in #21943.
Pull Request resolved: #22193

Differential Revision: D15983434

Pulled By: bddppq

fbshipit-source-id: ad3514dfd8a3b0d89442eef752864e5d3f3d04f0
zdevito pushed a commit to zdevito/ATen that referenced this pull request Jun 25, 2019
Summary:
This PR is to make mkldnn reshape and transpose not exposes as Tensor API, please see the comments in pytorch/pytorch#21943.
Pull Request resolved: pytorch/pytorch#22193

Differential Revision: D15983434

Pulled By: bddppq

fbshipit-source-id: ad3514dfd8a3b0d89442eef752864e5d3f3d04f0
@Jianhui-Li
Copy link

@soumith fixed in 22193. We don't intend to make mkldnn_* function exposed as public.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Merged module: mkldnn Related to Intel IDEEP or oneDNN (a.k.a. mkldnn) integration open source

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants