Add aten mkldnn batchnorm backward operator #20572
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pytorchbot
added
module: autograd
This was referenced May 16, 2019
li-roy
added
the
triaged
facebook-github-bot
pushed a commit
that referenced
this pull request
Jun 13, 2019
Summary: ### mkldnn backward ops list: - [ ] \(#20567) Add aten mkldnn conv2d backward operator 💛 - [ ] \(#20570) Add aten mkldnn backward ops: relu, linear and reshape 💛 - [ ] \(#20571) Add aten mkldnn backward ops: max_pool2d, avg_pool2d and adaptive_avg_poo2d 💛 - [ ] \(#20572) Add aten mkldnn batchnorm backward operator 💛 - [ ] \(#20573) Add aten mkldnn zero_ operator:yellow_heart: - [ ] \(#20575) Add mkldnn mul operator 💛 Pull Request resolved: #20575 Differential Revision: D15799529 Pulled By: bddppq fbshipit-source-id: 4887d8ef1a0e316ad9db199b657d9481fc13e486
zdevito
pushed a commit
to zdevito/ATen
that referenced
this pull request
Jun 13, 2019
Summary: ### mkldnn backward ops list: - [ ] \(pytorch/pytorch#20567) Add aten mkldnn conv2d backward operator 💛 - [ ] \(pytorch/pytorch#20570) Add aten mkldnn backward ops: relu, linear and reshape 💛 - [ ] \(pytorch/pytorch#20571) Add aten mkldnn backward ops: max_pool2d, avg_pool2d and adaptive_avg_poo2d 💛 - [ ] \(pytorch/pytorch#20572) Add aten mkldnn batchnorm backward operator 💛 - [ ] \(pytorch/pytorch#20573) Add aten mkldnn zero_ operator:yellow_heart: - [ ] \(pytorch/pytorch#20575) Add mkldnn mul operator 💛 Pull Request resolved: pytorch/pytorch#20575 Differential Revision: D15799529 Pulled By: bddppq fbshipit-source-id: 4887d8ef1a0e316ad9db199b657d9481fc13e486
facebook-github-bot
pushed a commit
that referenced
this pull request
Jun 14, 2019
Summary: ### mkldnn backward ops list: - [ ] \(#20567) Add aten mkldnn conv2d backward operator 💛 - [ ] \(#20570) Add aten mkldnn backward ops: relu, linear and reshape 💛 - [ ] \(#20571) Add aten mkldnn backward ops: max_pool2d, avg_pool2d and adaptive_avg_poo2d 💛 - [ ] \(#20572) Add aten mkldnn batchnorm backward operator 💛 - [ ] \(#20573) Add aten mkldnn zero_ operator:yellow_heart: - [ ] \(#20575) Add mkldnn mul operator 💚 Pull Request resolved: #20573 Differential Revision: D15820477 Pulled By: bddppq fbshipit-source-id: 35d95f5b4e013c8db1911f52148550a2e40a2e68
zdevito
pushed a commit
to zdevito/ATen
that referenced
this pull request
Jun 14, 2019
Summary: ### mkldnn backward ops list: - [ ] \(pytorch/pytorch#20567) Add aten mkldnn conv2d backward operator 💛 - [ ] \(pytorch/pytorch#20570) Add aten mkldnn backward ops: relu, linear and reshape 💛 - [ ] \(pytorch/pytorch#20571) Add aten mkldnn backward ops: max_pool2d, avg_pool2d and adaptive_avg_poo2d 💛 - [ ] \(pytorch/pytorch#20572) Add aten mkldnn batchnorm backward operator 💛 - [ ] \(pytorch/pytorch#20573) Add aten mkldnn zero_ operator:yellow_heart: - [ ] \(pytorch/pytorch#20575) Add mkldnn mul operator 💚 Pull Request resolved: pytorch/pytorch#20573 Differential Revision: D15820477 Pulled By: bddppq fbshipit-source-id: 35d95f5b4e013c8db1911f52148550a2e40a2e68
XiaobingSuper
force-pushed
the
mkldnn_bn_bwd
branch
2 times, most recently
from
5da8a09 to
671c018
Compare
XiaobingSuper
force-pushed
the
mkldnn_bn_bwd
branch
from
671c018 to
6abeffc
Compare
XiaobingSuper
force-pushed
the
mkldnn_bn_bwd
branch
from
c5505de to
bb277aa
Compare
XiaobingSuper
force-pushed
the
mkldnn_bn_bwd
branch
from
bb277aa to
b6131f0
Compare
Contributor
|
Can you please rebase on top of the master |
XiaobingSuper
force-pushed
the
mkldnn_bn_bwd
branch
2 times, most recently
from
ce582c3 to
ded344f
Compare
Collaborator
Author
VitalyFedyunin
suggested changes
Oct 25, 2019
XiaobingSuper
force-pushed
the
mkldnn_bn_bwd
branch
from
ded344f to
53f2729
Compare
Collaborator
Author
|
@VitalyFedyunin, I change the code according to your suggestions, thanks! |
XiaobingSuper
force-pushed
the
mkldnn_bn_bwd
branch
from
53f2729 to
a11bc90
Compare
Collaborator
Author
|
@VitalyFedyunin , the two failed cases is about cuda build, which are not caused by this changes, please help see tnem, thanks! |
XiaobingSuper
force-pushed
the
mkldnn_bn_bwd
branch
2 times, most recently
from
359fae7 to
24bd4d0
Compare
Collaborator
Author
1 similar comment
Collaborator
Author
XiaobingSuper
force-pushed
the
mkldnn_bn_bwd
branch
from
24bd4d0 to
2781a4b
Compare
Collaborator
Author
VitalyFedyunin
approved these changes
Nov 19, 2019
| bool train, | ||
| double momentum, | ||
| double eps) { | ||
| TORCH_CHECK(input.dim() == 4 || input.dim() == 5, |
Contributor
There was a problem hiding this comment.
No test case for 3d
Collaborator
Author
There was a problem hiding this comment.
3d operators will be enable at next step, including conv3d, pool3d, and other mkldnn operators.
Contributor
|
Do you have specific subset of operators in mind to add mkldnn backwards support? |
Collaborator
Author
|
Now, we want to enable conv, pool, batchnorm. linear, relu and reshape operators, which can run resnet and resnext models, and then add other operators for user's requests, such as 3d operators for Unet , laynorm, concat operators for transformer models. Thanks! |
Collaborator
Author
|
@VitalyFedyunin, Can you help to merge this PR? Thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
mkldnn backward ops list: