[quant][fx][bc-breaking] Add required example_inputs argument to prepare_fx and prepare_qat_fx #76496
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…_fx and prepare_qat_fx Summary: FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like `NON_OBSERVABLE_ARG_DICT` (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the `prepare_fx` and `prepare_qat_fx` api to require user to either provide sample positional arguments thrugh `sample_args` or a sample keyword arguments through `sample_kwargs`, basically `sample_args` and `sample_kwargs` can't both be None BC-breaking Note: Before: m = resnet18(...) m = prepare_fx(m, qconfig_dict) After: m = resnet18(...) m = prepare_fx(m, qconfig_dict, sample_args=(torch.randn(1, 3, 224, 224),)) Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestQuantizeFxModels Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
Contributor
🔗 Helpful links
❌ 12 New FailuresAs of commit a9b9293 (more details on the Dr. CI page): Expand to see more
🕵️ 11 new failures recognized by patternsThe following CI failures do not appear to be due to upstream breakages
|
| Job | Step | Action |
|---|---|---|
 pull / linux-xenial-cuda11.3-py3.7-gcc7 / test (distributed, 1, 2, linux.8xlarge.nvidia.gpu) |
Unknown | 🔁 rerun |
This comment was automatically generated by Dr. CI (expand for details).
Please report bugs/suggestions to the (internal) Dr. CI Users group.
jerryzh168
added a commit
that referenced
this pull request
Apr 27, 2022
…_fx and prepare_qat_fx Summary: FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like `NON_OBSERVABLE_ARG_DICT` (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the `prepare_fx` and `prepare_qat_fx` api to require user to either provide sample positional arguments thrugh `sample_args` or a sample keyword arguments through `sample_kwargs`, basically `sample_args` and `sample_kwargs` can't both be None BC-breaking Note: Before: m = resnet18(...) m = prepare_fx(m, qconfig_dict) After: m = resnet18(...) m = prepare_fx(m, qconfig_dict, sample_args=(torch.randn(1, 3, 224, 224),)) Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestQuantizeFxModels Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: d8c9c37 Pull Request resolved: #76496
Contributor
Author
|
@jerryzh168 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
jerryzh168
requested review from
HDCharles,
andrewor14,
dzdang,
vkuzo and
z-a-f
vkuzo
reviewed
Apr 28, 2022
vkuzo
reviewed
Apr 28, 2022
andrewor14
reviewed
Apr 28, 2022
… to prepare_fx and prepare_qat_fx" Summary: FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like `NON_OBSERVABLE_ARG_DICT` (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the `prepare_fx` and `prepare_qat_fx` api to require user to either provide sample positional arguments thrugh `sample_args` or a sample keyword arguments through `sample_kwargs`, basically `sample_args` and `sample_kwargs` can't both be None BC-breaking Note: Before: m = resnet18(...) m = prepare_fx(m, qconfig_dict) After: m = resnet18(...) m = prepare_fx(m, qconfig_dict, sample_args=(torch.randn(1, 3, 224, 224),)) Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestQuantizeFxModels Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35984526](https://our.internmc.facebook.com/intern/diff/D35984526) [ghstack-poisoned]
jerryzh168
changed the title
…ent to prepare_fx and prepare_qat_fx" Summary: FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like `NON_OBSERVABLE_ARG_DICT` (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the `prepare_fx` and `prepare_qat_fx` api to require user to either provide example arguments thrugh `example_inputs`, Note this api doesn't support kwargs, kwargs can make #76496 (comment) simpler, but it will be rare, and even then we can still workaround with positional arguments, also `torch.jit.trace`(https://pytorch.org/docs/stable/generated/torch.jit.trace.html) and ShapeProp: https://github.com/pytorch/pytorch/blob/master/torch/fx/passes/shape_prop.py#L140 just have single positional args, we'll just use a single `example_inputs` argument for now. If needed, we can extend the api with an optional `example_kwargs`. e.g. in case when there are a lot of arguments for forward and it makes more sense to pass the arguments by keyword BC-breaking Note: Before: m = resnet18(...) m = prepare_fx(m, qconfig_dict) After: m = resnet18(...) m = prepare_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestQuantizeFxModels Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35984526](https://our.internmc.facebook.com/intern/diff/D35984526) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Apr 28, 2022
…are_fx and prepare_qat_fx Summary: FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like `NON_OBSERVABLE_ARG_DICT` (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the `prepare_fx` and `prepare_qat_fx` api to require user to either provide example arguments thrugh `example_inputs`, Note this api doesn't support kwargs, kwargs can make #76496 (comment) simpler, but it will be rare, and even then we can still workaround with positional arguments, also `torch.jit.trace`(https://pytorch.org/docs/stable/generated/torch.jit.trace.html) and ShapeProp: https://github.com/pytorch/pytorch/blob/master/torch/fx/passes/shape_prop.py#L140 just have single positional args, we'll just use a single `example_inputs` argument for now. If needed, we can extend the api with an optional `example_kwargs`. e.g. in case when there are a lot of arguments for forward and it makes more sense to pass the arguments by keyword BC-breaking Note: Before: m = resnet18(...) m = prepare_fx(m, qconfig_dict) After: m = resnet18(...) m = prepare_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestQuantizeFxModels Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 81e9c11 Pull Request resolved: #76496
Contributor
Author
|
@jerryzh168 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Contributor
Author
|
changed to single |
…ent to prepare_fx and prepare_qat_fx" Summary: FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like `NON_OBSERVABLE_ARG_DICT` (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the `prepare_fx` and `prepare_qat_fx` api to require user to either provide example arguments thrugh `example_inputs`, Note this api doesn't support kwargs, kwargs can make #76496 (comment) simpler, but it will be rare, and even then we can still workaround with positional arguments, also `torch.jit.trace`(https://pytorch.org/docs/stable/generated/torch.jit.trace.html) and ShapeProp: https://github.com/pytorch/pytorch/blob/master/torch/fx/passes/shape_prop.py#L140 just have single positional args, we'll just use a single `example_inputs` argument for now. If needed, we can extend the api with an optional `example_kwargs`. e.g. in case when there are a lot of arguments for forward and it makes more sense to pass the arguments by keyword BC-breaking Note: Before: m = resnet18(...) m = prepare_fx(m, qconfig_dict) After: m = resnet18(...) m = prepare_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestQuantizeFxModels Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35984526](https://our.internmc.facebook.com/intern/diff/D35984526) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
May 4, 2022
…are_fx and prepare_qat_fx Summary: FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like `NON_OBSERVABLE_ARG_DICT` (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the `prepare_fx` and `prepare_qat_fx` api to require user to either provide example arguments thrugh `example_inputs`, Note this api doesn't support kwargs, kwargs can make #76496 (comment) simpler, but it will be rare, and even then we can still workaround with positional arguments, also `torch.jit.trace`(https://pytorch.org/docs/stable/generated/torch.jit.trace.html) and ShapeProp: https://github.com/pytorch/pytorch/blob/master/torch/fx/passes/shape_prop.py#L140 just have single positional args, we'll just use a single `example_inputs` argument for now. If needed, we can extend the api with an optional `example_kwargs`. e.g. in case when there are a lot of arguments for forward and it makes more sense to pass the arguments by keyword BC-breaking Note: Before: m = resnet18(...) m = prepare_fx(m, qconfig_dict) After: m = resnet18(...) m = prepare_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestQuantizeFxModels Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: c815e7e Pull Request resolved: #76496
jerryzh168
added a commit
that referenced
this pull request
May 4, 2022
…ple_inputs argument to prepare_fx and prepare_qat_fx" Summary: FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like `NON_OBSERVABLE_ARG_DICT` (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the `prepare_fx` and `prepare_qat_fx` api to require user to either provide example arguments thrugh `example_inputs`, Note this api doesn't support kwargs, kwargs can make #76496 (comment) simpler, but it will be rare, and even then we can still workaround with positional arguments, also `torch.jit.trace`(https://pytorch.org/docs/stable/generated/torch.jit.trace.html) and ShapeProp: https://github.com/pytorch/pytorch/blob/master/torch/fx/passes/shape_prop.py#L140 just have single positional args, we'll just use a single `example_inputs` argument for now. If needed, we can extend the api with an optional `example_kwargs`. e.g. in case when there are a lot of arguments for forward and it makes more sense to pass the arguments by keyword BC-breaking Note: Before: m = resnet18(...) m = prepare_fx(m, qconfig_dict) After: m = resnet18(...) m = prepare_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestQuantizeFxModels Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35984526](https://our.internmc.facebook.com/intern/diff/D35984526) [ghstack-poisoned]
vkuzo
reviewed
May 4, 2022
vkuzo
reviewed
May 4, 2022
…ent to prepare_fx and prepare_qat_fx" Summary: FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like `NON_OBSERVABLE_ARG_DICT` (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the `prepare_fx` and `prepare_qat_fx` api to require user to either provide example arguments thrugh `example_inputs`, Note this api doesn't support kwargs, kwargs can make #76496 (comment) simpler, but it will be rare, and even then we can still workaround with positional arguments, also `torch.jit.trace`(https://pytorch.org/docs/stable/generated/torch.jit.trace.html) and ShapeProp: https://github.com/pytorch/pytorch/blob/master/torch/fx/passes/shape_prop.py#L140 just have single positional args, we'll just use a single `example_inputs` argument for now. If needed, we can extend the api with an optional `example_kwargs`. e.g. in case when there are a lot of arguments for forward and it makes more sense to pass the arguments by keyword BC-breaking Note: Before: m = resnet18(...) m = prepare_fx(m, qconfig_dict) After: m = resnet18(...) m = prepare_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestQuantizeFxModels Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35984526](https://our.internmc.facebook.com/intern/diff/D35984526) [ghstack-poisoned]
pytorchmergebot
pushed a commit
that referenced
this pull request
May 21, 2022
…e_fx and prepare_qat_fx (#249) (#77608) Summary: X-link: facebookresearch/d2go#249 X-link: https://github.com/fairinternal/ClassyVision/pull/104 X-link: pytorch/benchmark#916 X-link: facebookresearch/ClassyVision#791 X-link: facebookresearch/mobile-vision#68 FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like NON_OBSERVABLE_ARG_DICT (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the prepare_fx and prepare_qat_fx api to require user to either provide example arguments thrugh example_inputs, Note this api doesn't support kwargs, kwargs can make #76496 (comment) (comment) simpler, but it will be rare, and even then we can still workaround with positional arguments, also torch.jit.trace(https://pytorch.org/docs/stable/generated/torch.jit.trace.html) and ShapeProp: https://github.com/pytorch/pytorch/blob/master/torch/fx/passes/shape_prop.py#L140 just have single positional args, we'll just use a single example_inputs argument for now. If needed, we can extend the api with an optional example_kwargs. e.g. in case when there are a lot of arguments for forward and it makes more sense to pass the arguments by keyword BC-breaking Note: Before: ```python m = resnet18(...) m = prepare_fx(m, qconfig_dict) # or m = prepare_qat_fx(m, qconfig_dict) ``` After: ```python m = resnet18(...) m = prepare_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) # or m = prepare_qat_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) ``` Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestQuantizeFxModels Imported from OSS **Static Docs Preview: classyvision** |[Full Site](https://our.intern.facebook.com/intern/staticdocs/eph/D35984526/V30/classyvision/)| |**Modified Pages**| Reviewed By: vkuzo, andrewor14 Differential Revision: D35984526 Pull Request resolved: #77608 Approved by: https://github.com/dzdang
facebook-github-bot
pushed a commit
that referenced
this pull request
May 21, 2022
…are_fx and prepare_qat_fx (#77608) Summary: Pull Request resolved: #77608 X-link: meta-pytorch/fx2trt#76 X-link: facebookresearch/d2go#249 X-link: fairinternal/ClassyVision#104 X-link: pytorch/benchmark#916 X-link: facebookresearch/ClassyVision#791 X-link: facebookresearch/mobile-vision#68 FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like NON_OBSERVABLE_ARG_DICT (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the prepare_fx and prepare_qat_fx api to require user to either provide example arguments thrugh example_inputs, Note this api doesn't support kwargs, kwargs can make #76496 (comment) (comment) simpler, but it will be rare, and even then we can still workaround with positional arguments, also torch.jit.trace(https://pytorch.org/docs/stable/generated/torch.jit.trace.html) and ShapeProp: https://github.com/pytorch/pytorch/blob/master/torch/fx/passes/shape_prop.py#L140 just have single positional args, we'll just use a single example_inputs argument for now. If needed, we can extend the api with an optional example_kwargs. e.g. in case when there are a lot of arguments for forward and it makes more sense to pass the arguments by keyword BC-breaking Note: Before: ```python m = resnet18(...) m = prepare_fx(m, qconfig_dict) # or m = prepare_qat_fx(m, qconfig_dict) ``` After: ```python m = resnet18(...) m = prepare_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) # or m = prepare_qat_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) ``` Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestQuantizeFxModels Imported from OSS **Static Docs Preview: classyvision** |[Full Site](https://our.intern.facebook.com/intern/staticdocs/eph/D35984526/V44/classyvision/)| |**Modified Pages**| Reviewed By: vkuzo, andrewor14 Differential Revision: D35984526 fbshipit-source-id: 706c8df71722c9aa5082a6491734f0144f0dd670
facebook-github-bot
pushed a commit
to facebookresearch/ClassyVision
that referenced
this pull request
May 21, 2022
… (#77608) Summary: X-link: pytorch/pytorch#77608 X-link: meta-pytorch/fx2trt#76 X-link: facebookresearch/d2go#249 X-link: fairinternal/ClassyVision#104 X-link: pytorch/benchmark#916 Pull Request resolved: #791 X-link: facebookresearch/mobile-vision#68 FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like NON_OBSERVABLE_ARG_DICT (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the prepare_fx and prepare_qat_fx api to require user to either provide example arguments thrugh example_inputs, Note this api doesn't support kwargs, kwargs can make pytorch/pytorch#76496 (comment) (comment) simpler, but it will be rare, and even then we can still workaround with positional arguments, also torch.jit.trace(https://pytorch.org/docs/stable/generated/torch.jit.trace.html) and ShapeProp: https://github.com/pytorch/pytorch/blob/master/torch/fx/passes/shape_prop.py#L140 just have single positional args, we'll just use a single example_inputs argument for now. If needed, we can extend the api with an optional example_kwargs. e.g. in case when there are a lot of arguments for forward and it makes more sense to pass the arguments by keyword BC-breaking Note: Before: ```python m = resnet18(...) m = prepare_fx(m, qconfig_dict) # or m = prepare_qat_fx(m, qconfig_dict) ``` After: ```python m = resnet18(...) m = prepare_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) # or m = prepare_qat_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) ``` Reviewed By: vkuzo, andrewor14 Differential Revision: D35984526 fbshipit-source-id: 706c8df71722c9aa5082a6491734f0144f0dd670
swang392
pushed a commit
that referenced
this pull request
May 25, 2022
…e_fx and prepare_qat_fx (#249) (#77608) Summary: X-link: facebookresearch/d2go#249 X-link: https://github.com/fairinternal/ClassyVision/pull/104 X-link: pytorch/benchmark#916 X-link: facebookresearch/ClassyVision#791 X-link: facebookresearch/mobile-vision#68 FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like NON_OBSERVABLE_ARG_DICT (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the prepare_fx and prepare_qat_fx api to require user to either provide example arguments thrugh example_inputs, Note this api doesn't support kwargs, kwargs can make #76496 (comment) (comment) simpler, but it will be rare, and even then we can still workaround with positional arguments, also torch.jit.trace(https://pytorch.org/docs/stable/generated/torch.jit.trace.html) and ShapeProp: https://github.com/pytorch/pytorch/blob/master/torch/fx/passes/shape_prop.py#L140 just have single positional args, we'll just use a single example_inputs argument for now. If needed, we can extend the api with an optional example_kwargs. e.g. in case when there are a lot of arguments for forward and it makes more sense to pass the arguments by keyword BC-breaking Note: Before: ```python m = resnet18(...) m = prepare_fx(m, qconfig_dict) # or m = prepare_qat_fx(m, qconfig_dict) ``` After: ```python m = resnet18(...) m = prepare_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) # or m = prepare_qat_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) ``` Test Plan: python test/test_quantization.py TestQuantizeFx python test/test_quantization.py TestQuantizeFxOps python test/test_quantization.py TestQuantizeFxModels Imported from OSS **Static Docs Preview: classyvision** |[Full Site](https://our.intern.facebook.com/intern/staticdocs/eph/D35984526/V30/classyvision/)| |**Modified Pages**| Reviewed By: vkuzo, andrewor14 Differential Revision: D35984526 Pull Request resolved: #77608 Approved by: https://github.com/dzdang
jerryzh168
added a commit
that referenced
this pull request
May 25, 2022
…ize_fx Summary: as a followup to #76496, we defined model specific example_inputs for the test models in common_quantization.py and used these in test_equalize_fx Test Plan: python test/test_quantization.py TestEqualizeFx Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 44cabba Pull Request resolved: #78314
jerryzh168
added a commit
that referenced
this pull request
May 25, 2022
…ize_fx Summary: as a followup to #76496, we defined model specific example_inputs for the test models in common_quantization.py and used these in test_equalize_fx Test Plan: python test/test_quantization.py TestEqualizeFx Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
pytorchmergebot
pushed a commit
that referenced
this pull request
May 26, 2022
…ize_fx Summary: as a followup to #76496, we defined model specific example_inputs for the test models in common_quantization.py and used these in test_equalize_fx Test Plan: python test/test_quantization.py TestEqualizeFx Reviewers: Subscribers: Tasks: Tags: Pull Request resolved: #78314 Approved by: https://github.com/vkuzo
facebook-github-bot
pushed a commit
that referenced
this pull request
May 27, 2022
…ize_fx (#78314) Summary: as a followup to #76496, we defined model specific example_inputs for the test models in common_quantization.py and used these in test_equalize_fx Pull Request resolved: #78314 Approved by: https://github.com/vkuzo Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/8225f42a8a760259a2adfceb2672c25ef4107037 Test plan from GitHub: python test/test_quantization.py TestEqualizeFx Reviewed By: andrewor14, mehtanirav Differential Revision: D36708225 Pulled By: jerryzh168 fbshipit-source-id: 9fe6227b532576d5ae48536d74e94040ca085ccc
jerryzh168
added a commit
to jerryzh168/ClassyVision-1
that referenced
this pull request
Jun 1, 2022
Summary: As follow up for pytorch/pytorch#76496, we fixes the TODOs in quantization tests by providing correct example_inputs in the tests Differential Revision: D36818665 fbshipit-source-id: 0754cf5d6159dbcf3202a0720f18348bef4a4d79
jerryzh168
added a commit
to jerryzh168/ClassyVision-1
that referenced
this pull request
Jun 1, 2022
…#78660) Summary: X-link: pytorch/pytorch#78660 X-link: fairinternal/ClassyVision#105 As follow up for pytorch/pytorch#76496, we fixes the TODOs in quantization tests by providing correct example_inputs in the tests Differential Revision: D36818665 fbshipit-source-id: 192cef688dc52a1ecf6b757a7758c9867a5b1814
jerryzh168
added a commit
to jerryzh168/ClassyVision-1
that referenced
this pull request
Jun 2, 2022
…acebookresearch#793) Summary: Pull Request resolved: facebookresearch#793 X-link: pytorch/pytorch#78660 X-link: fairinternal/ClassyVision#105 As follow up for pytorch/pytorch#76496, we fixes the TODOs in quantization tests by providing correct example_inputs in the tests Differential Revision: D36818665 fbshipit-source-id: 5d637488e3778949bcbb5ce24cdd19cafe12aeb6
jerryzh168
added a commit
to jerryzh168/ClassyVision-1
that referenced
this pull request
Jun 2, 2022
…acebookresearch#793) Summary: Pull Request resolved: facebookresearch#793 X-link: pytorch/pytorch#78660 X-link: fairinternal/ClassyVision#105 As follow up for pytorch/pytorch#76496, we fixes the TODOs in quantization tests by providing correct example_inputs in the tests Reviewed By: dzdang Differential Revision: D36818665 fbshipit-source-id: 2a8fc015c1ba29df02ddbaed447327553a928813
jerryzh168
added a commit
to jerryzh168/pytorch
that referenced
this pull request
Jun 2, 2022
…vision (pytorch#793) Summary: X-link: facebookresearch/ClassyVision#793 Pull Request resolved: pytorch#78660 X-link: fairinternal/ClassyVision#105 As follow up for pytorch#76496, we fixes the TODOs in quantization tests by providing correct example_inputs in the tests Test Plan: classyvision sandcastle and ossci **Static Docs Preview: classyvision** |[Full Site](https://our.intern.facebook.com/intern/staticdocs/eph/D36818665/V8/classyvision/)| |**Modified Pages**| Reviewed By: dzdang Differential Revision: D36818665 fbshipit-source-id: ad090e4bf8820777b32540e4decfe741aeac26a7
facebook-github-bot
pushed a commit
to facebookresearch/ClassyVision
that referenced
this pull request
Jun 3, 2022
…793) Summary: Pull Request resolved: #793 X-link: pytorch/pytorch#78660 X-link: fairinternal/ClassyVision#105 As follow up for pytorch/pytorch#76496, we fixes the TODOs in quantization tests by providing correct example_inputs in the tests Reviewed By: dzdang Differential Revision: D36818665 fbshipit-source-id: 07bef205c352910642983a786b4346aa7165c14a
pytorchmergebot
pushed a commit
that referenced
this pull request
Jun 3, 2022
…vision (#105) (#78660) Summary: X-link: https://github.com/fairinternal/ClassyVision/pull/105 As follow up for #76496, we fixes the TODOs in quantization tests by providing correct example_inputs in the tests Test Plan: classyvision sandcastle and ossci **Static Docs Preview: classyvision** |[Full Site](https://our.intern.facebook.com/intern/staticdocs/eph/D36818665/V1/classyvision/)| |**Modified Pages**| Differential Revision: D36818665 Pull Request resolved: #78660 Approved by: https://github.com/vkuzo
facebook-github-bot
pushed a commit
that referenced
this pull request
Jun 3, 2022
…vision (#793) Summary: X-link: facebookresearch/ClassyVision#793 Pull Request resolved: #78660 X-link: fairinternal/ClassyVision#105 As follow up for #76496, we fixes the TODOs in quantization tests by providing correct example_inputs in the tests Test Plan: classyvision sandcastle and ossci **Static Docs Preview: classyvision** |[Full Site](https://our.intern.facebook.com/intern/staticdocs/eph/D36818665/V8/classyvision/)| |**Modified Pages**| Reviewed By: dzdang Differential Revision: D36818665 fbshipit-source-id: 07bef205c352910642983a786b4346aa7165c14a
frank-wei
pushed a commit
to pytorch/TensorRT
that referenced
this pull request
Jun 4, 2022
…are_fx and prepare_qat_fx (#77608) Summary: X-link: pytorch/pytorch#77608 Pull Request resolved: meta-pytorch/fx2trt#76 X-link: facebookresearch/d2go#249 X-link: fairinternal/ClassyVision#104 X-link: pytorch/benchmark#916 X-link: facebookresearch/ClassyVision#791 X-link: facebookresearch/mobile-vision#68 FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like NON_OBSERVABLE_ARG_DICT (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the prepare_fx and prepare_qat_fx api to require user to either provide example arguments thrugh example_inputs, Note this api doesn't support kwargs, kwargs can make pytorch/pytorch#76496 (comment) (comment) simpler, but it will be rare, and even then we can still workaround with positional arguments, also torch.jit.trace(https://pytorch.org/docs/stable/generated/torch.jit.trace.html) and ShapeProp: https://github.com/pytorch/pytorch/blob/master/torch/fx/passes/shape_prop.py#L140 just have single positional args, we'll just use a single example_inputs argument for now. If needed, we can extend the api with an optional example_kwargs. e.g. in case when there are a lot of arguments for forward and it makes more sense to pass the arguments by keyword BC-breaking Note: Before: ```python m = resnet18(...) m = prepare_fx(m, qconfig_dict) # or m = prepare_qat_fx(m, qconfig_dict) ``` After: ```python m = resnet18(...) m = prepare_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) # or m = prepare_qat_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) ``` Reviewed By: vkuzo, andrewor14 Differential Revision: D35984526 fbshipit-source-id: 706c8df71722c9aa5082a6491734f0144f0dd670
jerryzh168
added a commit
to jerryzh168/d2go
that referenced
this pull request
Jul 1, 2022
Summary: Following up the bc-breaking change from fx graph mode quantization: pytorch/pytorch#76496 that added example_inputs to prepare_fx and prepare_qat_fx, we fixes the callsite related to mobile-vision and exposed extra example_inputs in some apis Reviewed By: wat3rBro Differential Revision: D37163018 fbshipit-source-id: 1d2203612d073683224809b8c3f124e4576fc745
facebook-github-bot
pushed a commit
to facebookresearch/d2go
that referenced
this pull request
Jul 2, 2022
Summary: Pull Request resolved: #321 Following up the bc-breaking change from fx graph mode quantization: pytorch/pytorch#76496 that added example_inputs to prepare_fx and prepare_qat_fx, we fixes the callsite related to mobile-vision and exposed extra example_inputs in some apis Reviewed By: wat3rBro Differential Revision: D37163018 fbshipit-source-id: 9f0bb56659345d174a39b6d3bb4408caa553b88d
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
Summary:
FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to
insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors.
Currently we have some hacks to support this by defining some rules like
NON_OBSERVABLE_ARG_DICT(https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base.As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args
so that we can infer the type in a more robust way. This PR starts with changing the
prepare_fxandprepare_qat_fxapi to require user to either provideexample arguments thrugh
example_inputs, Note this api doesn't support kwargs, kwargs can make #76496 (comment) simpler, butit will be rare, and even then we can still workaround with positional arguments, also
torch.jit.trace(https://pytorch.org/docs/stable/generated/torch.jit.trace.html) and ShapeProp: https://github.com/pytorch/pytorch/blob/master/torch/fx/passes/shape_prop.py#L140 just have single positional args, we'll just use a singleexample_inputsargument for now.If needed, we can extend the api with an optional
example_kwargs. e.g. in case when there are a lot of arguments for forward and it makes more sense topass the arguments by keyword
BC-breaking Note:
Before:
m = resnet18(...)
m = prepare_fx(m, qconfig_dict)
After:
m = resnet18(...)
m = prepare_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),))
Test Plan:
python test/test_quantization.py TestQuantizeFx
python test/test_quantization.py TestQuantizeFxOps
python test/test_quantization.py TestQuantizeFxModels
Reviewers:
Subscribers:
Tasks:
Tags:
Differential Revision: D35984526