[quantization] Store bias in PackedConvWeight in fbgemm #25626

supriyar · 2019-09-04T02:41:44Z

Stack from ghstack:

[quantization] Store bias in PackedConvWeight in fbgemm #25626 [quantization] Store bias in PackedConvWeight in fbgemm

Add bias as an optional parameter in the packed conv weight struct.

Differential Revision: D17177723

Add bias as an optional parameter in the packed conv weight struct. Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

Add bias as an optional parameter in the packed conv weight struct. Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/) ghstack-source-id: 89450557 Pull Request resolved: #25626

Add bias as an optional parameter in the packed conv weight struct. Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

Pull Request resolved: #25626 Add bias as an optional parameter in the packed conv weight struct. ghstack-source-id: 89452640 Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

Add bias as an optional parameter in the packed conv weight struct. Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

Pull Request resolved: #25626 Add bias as an optional parameter in the packed conv weight struct. ghstack-source-id: 89454242 Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

aten/src/ATen/native/quantized/cpu/qconv.cpp

dskhudia · 2019-09-04T16:53:31Z

aten/src/ATen/native/quantized/cpu/qconv.cpp

+        for (int i = 0; i < K; ++i) {
+          bias_scale.data_ptr<double>()[i] = pack_ptr.w_scale[i] * act_scale;
+        }
+        qbias = quantize_linear_per_channel_cpu(


quantize_linear_per_channel instead of quantize_linear_per_channel_cpu

Using the quantize_linear_per_channel function causes the following error

RuntimeError: Expected object of type Variable but found type CPUFloatType for argument #1 'scales' (checked_cast_variable at caffe2/torch/csrc/autograd/VariableTypeManual.cpp:38)

Spoke to @yf225 offline and he said the error suggests that we are missing a at::AutoNonVariableTypeMode thread-local guard somewhere in the call path of QLinearInt8 operator().
quantize_linear_per_channel dispatches to VariableType dispatcher, which expects the input tensor to be a Variable.

do you know why are we missing this guard?

I am not sure. @smessmer any thoughts on if this guard should be added?

Is this issue resolved?

Not yet, I will file a separate issue for this.

aten/src/ATen/native/quantized/cpu/qconv.cpp

dskhudia · 2019-09-04T17:01:07Z

torch/nn/_intrinsic/quantized/modules/conv_relu.py

                                         stride=stride, padding=padding, dilation=dilation,
                                         groups=groups, bias=bias, padding_mode=padding_mode)

-    def weight(self):


Are these two methods being removed?

Yes, I don't see why they are needed as the parent module already defined them. Looks like linear_relu.py also doesn't have them

Add bias as an optional parameter in the packed conv weight struct. Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

Pull Request resolved: #25626 Add bias as an optional parameter in the packed conv weight struct. ghstack-source-id: 89519225 Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

Add bias as an optional parameter in the packed conv weight struct. Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

Pull Request resolved: #25626 Add bias as an optional parameter in the packed conv weight struct. ghstack-source-id: 89559766 Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

Add bias as an optional parameter in the packed conv weight struct. Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

Pull Request resolved: #25626 Add bias as an optional parameter in the packed conv weight struct. ghstack-source-id: 89601358 Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

Add bias as an optional parameter in the packed conv weight struct. Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

Pull Request resolved: #25626 Add bias as an optional parameter in the packed conv weight struct. ghstack-source-id: 89630144 Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

test/test_quantized.py

raghuramank100

Looks great, please address comments prior to submitting.

Add bias as an optional parameter in the packed conv weight struct. Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

Pull Request resolved: #25626 Add bias as an optional parameter in the packed conv weight struct. ghstack-source-id: 89780639 Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

facebook-github-bot · 2019-09-10T16:09:33Z

This pull request has been merged in c60dddb.

Summary: Pull Request resolved: pytorch/pytorch#25626 Add bias as an optional parameter in the packed conv weight struct. ghstack-source-id: 89780639 Test Plan: python test/run_test.py --exclude nn --verbose --bring-to-front quantization quantized quantized_tensor quantized_nn_mods quantizer Reviewed By: raghuramank100 Differential Revision: D17177723 fbshipit-source-id: e502f2196cb1c002db8b691124db740368944c92

Summary: Fix the patterns after changes to prepack functions(#25626) Test Plan: pytho test/test_jit.py 'TestJit.test_quant_fusion' Reviewers: mvz Subscribers: Tasks: Tags: [ghstack-poisoned]

…rns" Summary: Fix the patterns after changes to prepack functions(#25626) Test Plan: pytho test/test_jit.py 'TestJit.test_quant_fusion' Reviewers: mvz Subscribers: Tasks: Tags: [ghstack-poisoned]

…rns" Summary: Fix the patterns after changes to prepack functions(#25626) Test Plan: pytho test/test_jit.py 'TestJit.test_quant_fusion' Reviewers: mvz Subscribers: Tasks: Tags: Differential Revision: [D17465553](https://our.internmc.facebook.com/intern/diff/D17465553) [ghstack-poisoned]

Summary: Pull Request resolved: #26414 Fix the patterns after changes to prepack functions(#25626) Test Plan: pytho test/test_jit.py 'TestJit.test_quant_fusion' Imported from OSS Differential Revision: D17465553 fbshipit-source-id: 7df6a6aa8389bb4a7a370c65ade4c2585b45b882

[quantization] Store bias in PackedConvWeight in fbgemm

3d02475

Add bias as an optional parameter in the packed conv weight struct. Differential Revision: [D17177723](https://our.internmc.facebook.com/intern/diff/D17177723/)

supriyar requested a review from apaszke as a code owner September 4, 2019 02:41

pytorchbot added module: nn Related to torch.nn module: operators oncall: quantization Quantization support in PyTorch labels Sep 4, 2019

This was referenced Sep 4, 2019

[quantization] Rename fbgemm quantized operators to generic quantized ops #25338

Closed

[quantization] Store bias in PackedLinearWeight struct in fbgemm #25428

Closed

supriyar requested review from dskhudia, dzhulgakov and ljk53 September 4, 2019 02:42

dskhudia reviewed Sep 4, 2019

View reviewed changes

supriyar requested a review from dskhudia September 4, 2019 21:05

supriyar mentioned this pull request Sep 4, 2019

[quantization] Rename FBGEMM quantized operators to generic quantized ops #25678

Closed

supriyar requested a review from raghuramank100 September 6, 2019 23:48

raghuramank100 reviewed Sep 9, 2019

View reviewed changes

test/test_quantized.py Show resolved Hide resolved

raghuramank100 approved these changes Sep 9, 2019

View reviewed changes

facebook-github-bot closed this in c60dddb Sep 10, 2019

facebook-github-bot added the merged label Sep 10, 2019

jerryzh168 mentioned this pull request Sep 18, 2019

[quant][graphmode] Fix quantized::linear QuantFusion patterns #26414

Closed

facebook-github-bot deleted the gh/supriyar/9/head branch October 28, 2019 22:21

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[quantization] Store bias in PackedConvWeight in fbgemm #25626

[quantization] Store bias in PackedConvWeight in fbgemm #25626

Uh oh!

supriyar commented Sep 4, 2019 •

edited

Loading

Uh oh!

Uh oh!

dskhudia Sep 4, 2019

Uh oh!

supriyar Sep 4, 2019

Uh oh!

jerryzh168 Sep 4, 2019

Uh oh!

supriyar Sep 4, 2019

Uh oh!

raghuramank100 Sep 9, 2019

Uh oh!

supriyar Sep 10, 2019

Uh oh!

Uh oh!

dskhudia Sep 4, 2019

Uh oh!

supriyar Sep 4, 2019

Uh oh!

Uh oh!

raghuramank100 left a comment

Uh oh!

facebook-github-bot commented Sep 10, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

[quantization] Store bias in PackedConvWeight in fbgemm #25626

[quantization] Store bias in PackedConvWeight in fbgemm #25626

Uh oh!

Conversation

supriyar commented Sep 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

raghuramank100 left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Sep 10, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

supriyar commented Sep 4, 2019 •

edited

Loading