[quantization] PackedSequence support for quantized LSTM #29585
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[ghstack-poisoned]
jianyuh
approved these changes
Nov 11, 2019
Member
jianyuh
left a comment
jianyuh
left a comment
There was a problem hiding this comment.
LGTM! Thanks for finishing this so quickly!
aten/src/ATen/native/RNN.cpp
Outdated
Comment on lines
1210
to
1215
| if (use_miopen(data, dropout_p)) { | ||
| Tensor output, hy, cy; | ||
| lstm_packed_miopen_stub(data.type().device_type(), output, hy, cy, data, batch_sizes, hx, | ||
| _params, has_biases, num_layers, dropout_p, train, bidirectional); | ||
| return std::make_tuple(std::move(output), std::move(hy), std::move(cy)); | ||
| } |
Member
There was a problem hiding this comment.
Do we need to add the same handler in Line 1160?
Collaborator
Author
There was a problem hiding this comment.
Actually, not sure this should even be here. I just directly copy-pasted from the regular LSTM function
BTW do we actually support cudnn for these ops?
Member
There was a problem hiding this comment.
Collaborator
Author
There was a problem hiding this comment.
@jianyuh I don't think those implementations (miopen and cudnn) actually support dynamic quantization
Member
There was a problem hiding this comment.
Oh I thought you were asking if lstm_packed_miopen has cuda/cudnn implementation. Yep, the dynamic quantization is only supported on the CPU side.
jianyuh
previously requested changes
Nov 11, 2019
Member
jianyuh
left a comment
jianyuh
left a comment
There was a problem hiding this comment.
Check the BC issue:
Nov 11 23:15:22 processing existing schema: aten::quantized_lstm(Tensor input, Tensor[] hx, Tensor[] params, bool has_biases, int num_layers, float dropout, bool train, bool bidirectional, bool batch_first, *, int? dtype=None, bool use_dynamic=False) -> (Tensor, Tensor, Tensor)
Nov 11 23:15:22 Can NOT find backward compatible schemas after changes for schema aten::quantized_lstm(Tensor input, Tensor[] hx, Tensor[] params, bool has_biases, int num_layers, float dropout, bool train, bool bidirectional, bool batch_first, *, int? dtype=None, bool use_dynamic=False) -> (Tensor, Tensor, Tensor) from the following candidates:
Nov 11 23:15:22 [
Nov 11 23:15:22 aten::quantized_lstm.data(Tensor data, Tensor batch_sizes, Tensor[] hx, Tensor[] params, bool has_biases, int num_layers, float dropout, bool train, bool bidirectional, *, int? dtype=None, bool use_dynamic=False) -> (Tensor, Tensor, Tensor)
Nov 11 23:15:22 aten::quantized_lstm.tensor(Tensor input, Tensor[] hx, Tensor[] params, bool has_biases, int num_layers, float dropout, bool train, bool bidirectional, bool batch_first, *, int? dtype=None, bool use_dynamic=False) -> (Tensor, Tensor, Tensor)
Nov 11 23:15:22 ]
Nov 11 23:15:22 The PR is introducing backward incompatible changes to the operator library. Please contact PyTorch team to confirm whether this change is wanted or not.
Collaborator
Author
|
@jianyuh I believe the BC issue is a false positive: the only difference from before is it has the |
houseroad
reviewed
Nov 12, 2019
Fixes #27963 Differential Revision: [D18436569](https://our.internmc.facebook.com/intern/diff/D18436569) [ghstack-poisoned]
Fixes #27963 Differential Revision: [D18436569](https://our.internmc.facebook.com/intern/diff/D18436569) [ghstack-poisoned]
Fixes #27963 Differential Revision: [D18436569](https://our.internmc.facebook.com/intern/diff/D18436569) [ghstack-poisoned]
zdevito
pushed a commit
to zdevito/ATen
that referenced
this pull request
Nov 13, 2019
Summary: Pull Request resolved: pytorch/pytorch#29585 Test Plan: Imported from OSS Differential Revision: D18436569 Pulled By: jamesr66a fbshipit-source-id: 0f32c0fcc897894e30d8e7ff203392c1a961ce60
Contributor
|
@jamesr66a merged this pull request in 20fb8a8. |
jianyuh
added a commit
that referenced
this pull request
Nov 14, 2019
…ntized LSTM As Title says. This is a follow-up for #29585. Check #27963 Differential Revision: [D18497319](https://our.internmc.facebook.com/intern/diff/D18497319/) [ghstack-poisoned]
jianyuh
added a commit
that referenced
this pull request
Nov 14, 2019
…ntized LSTM As Title says. This is a follow-up for #29585. Check #27963 Differential Revision: [D18497319](https://our.internmc.facebook.com/intern/diff/D18497319/) ghstack-source-id: 93883524 Pull Request resolved: #29786
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Fixes #27963
Differential Revision: D18436569