Add some doc for export_for_training #135918
Conversation
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/135918
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit dadc6db with merge base 09519eb ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Differential Revision: [D62610491](https://our.internmc.facebook.com/intern/diff/D62610491) [ghstack-poisoned]
|
@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Differential Revision: [D62610491](https://our.internmc.facebook.com/intern/diff/D62610491) [ghstack-poisoned]
|
@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Differential Revision: [D62610491](https://our.internmc.facebook.com/intern/diff/D62610491) [ghstack-poisoned]
|
@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
avikchaudhuri
left a comment
avikchaudhuri
left a comment
There was a problem hiding this comment.
Looks good, suggested edits for grammar / style.
docs/source/export.rst
Outdated
|
|
||
| .. _Training Export: | ||
|
|
||
| Training Export |
There was a problem hiding this comment.
Not a good title. Maybe something like "Export for Training and Inference"?
docs/source/export.rst
Outdated
|
|
||
| In this API, we produce the most generic IR that contains all ATen operators | ||
| (including both functional and non-functional) which can be used to train in | ||
| eager PyTorch Autograd. This API is intended for PT2 quantization training use cases |
There was a problem hiding this comment.
I'm not sure why you need to mention quantization training. E.g., you'd want this for normal eager training too.
docs/source/export.rst
Outdated
| In this API, we produce the most generic IR that contains all ATen operators | ||
| (including both functional and non-functional) which can be used to train in | ||
| eager PyTorch Autograd. This API is intended for PT2 quantization training use cases | ||
| and will soon be the default IR of torch.export.export in the near future. To read further about |
There was a problem hiding this comment.
soon vs. near future, pick one
docs/source/export.rst
Outdated
| ) | ||
| Range constraints: {} | ||
| Here you can see that, we kept `conv2d` op in the IR while decomposing the rest. Now the IR is an functional IR |
There was a problem hiding this comment.
Edited:
Here you can see that we kept conv2d op in the IR while decomposing the rest. Now the IR is a functional IR
containing core aten operators except for conv2d.
You can do even more customization by directly registering your chosen decomposition behaviors.
docs/source/export.rst
Outdated
| as :func:`export` except for the operators in the graph. You can see that we captured `batch_norm` in the most general | ||
| form. This op is non-functional and will be lowered to different ops when running under inference mode. | ||
|
|
||
| You can also go from this IR to an inference IR via :func:`run_decompositions` with arbitrary customizations |
There was a problem hiding this comment.
Edited:
From the above output, you can see that :func:export_for_training produces pretty much the same ExportedProgram
as :func:export except for the operators in the graph. You can see that we captured batch_norm in the most general
form. This op is non-functional and will be lowered to different ops when running inference.
You can also go from this IR to an inference IR via :func:run_decompositions with arbitrary customizations.
docs/source/export.rst
Outdated
| the motivation behind this change, please refer to | ||
| https://dev-discuss.pytorch.org/t/why-pytorch-does-not-need-a-new-standardized-operator-set/2206 | ||
|
|
||
| With this API, and :func:`run_decompositions()`, you should be able to get any inference IR with |
There was a problem hiding this comment.
Edited:
When this API is combined with :func:run_decompositions(), you should be able to get inference IR with
any desired decomposition behavior.
Differential Revision: [D62610491](https://our.internmc.facebook.com/intern/diff/D62610491) [ghstack-poisoned]
|
@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Differential Revision: [D62610491](https://our.internmc.facebook.com/intern/diff/D62610491) [ghstack-poisoned]
|
@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Differential Revision: [D62610491](https://our.internmc.facebook.com/intern/diff/D62610491) [ghstack-poisoned]
Differential Revision: [D62610491](https://our.internmc.facebook.com/intern/diff/D62610491) [ghstack-poisoned]
|
@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Differential Revision: [D62610491](https://our.internmc.facebook.com/intern/diff/D62610491) [ghstack-poisoned]
|
@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Differential Revision: [D62610491](https://our.internmc.facebook.com/intern/diff/D62610491) [ghstack-poisoned]
|
@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
1 similar comment
|
@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Differential Revision: [D62610491](https://our.internmc.facebook.com/intern/diff/D62610491) [ghstack-poisoned]
Differential Revision: [D62610491](https://our.internmc.facebook.com/intern/diff/D62610491) [ghstack-poisoned]
|
@tugsbayasgalan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@pytorchbot merge -f 'Landed internally' (Initiating merge automatically since Phabricator Diff has merged, using force because this PR might not pass merge_rules.json but landed internally) |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Differential Revision: [D62610491](https://our.internmc.facebook.com/intern/diff/D62610491) Pull Request resolved: pytorch#135918 Approved by: https://github.com/avikchaudhuri ghstack dependencies: pytorch#135080, pytorch#135912
4 participants
Stack from ghstack (oldest at bottom):
Differential Revision: D62610491