[JIT] Add Support For Module Containers as Iterables #28255
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[ghstack-poisoned]
facebook-github-bot
added
the
oncall: jit
eellison
changed the title
Add support for treating Sequentials, ModuleLists, and ModuleDicts as iterables. As previously, when emitting a for loop over a Module Container we unroll the for loop over all elements. We require that any Sugared Value in an iterable with a Module Container have a statically - determinable length. Otherwise, if you zipped over a list of varying length and an nn.Sequential that alternated between returning a Tensor and a Dictionary, the output type would change based on the length of the list. Fix for #17179 And #27790 [ghstack-poisoned]
Add support for treating Sequentials, ModuleLists, and ModuleDicts as iterables. As previously, when emitting a for loop over a Module Container we unroll the for loop over all elements. We require that any Sugared Value in an iterable with a Module Container have a statically - determinable length. Otherwise, if you zipped over a list of varying length and an nn.Sequential that alternated between returning a Tensor and a Dictionary, the output type would change based on the length of the list. Fix for #17179 And #27790 [ghstack-poisoned]
Add support for treating Sequentials, ModuleLists, and ModuleDicts as iterables. As previously, when emitting a for loop over a Module Container we unroll the for loop over all elements. We require that any Sugared Value in an iterable with a Module Container have a statically - determinable length. Otherwise, if you zipped over a list of varying length and an nn.Sequential that alternated between returning a Tensor and a Dictionary, the output type would change based on the length of the list. Fix for #17179 And #27401 and #27506 [ghstack-poisoned]
Add support for treating Sequentials, ModuleLists, and ModuleDicts as iterables. As previously, when emitting a for loop over a Module Container we unroll the for loop over all elements. We require that any Sugared Value in an iterable with a Module Container have a statically - determinable length. Otherwise, if you zipped over a list of varying length and an nn.Sequential that alternated between returning a Tensor and a Dictionary, the output type would change based on the length of the list. Fix for #17179 And #27401 and #27506 [ghstack-poisoned]
zdevito
reviewed
Oct 28, 2019
Contributor
zdevito
left a comment
zdevito
left a comment
There was a problem hiding this comment.
This looks pretty good. I have one substantive comment: that we should try just putting the iteration functionality into sugaredvalue, as a way of avoiding having to introduce a new concept.
Add support for treating Sequentials, ModuleLists, and ModuleDicts as iterables. As previously, when emitting a for loop over a Module Container we unroll the for loop over all elements. We require that any Sugared Value in an iterable with a Module Container have a statically - determinable length. Otherwise, if you zipped over a list of varying length and an nn.Sequential that alternated between returning a Tensor and a Dictionary, the output type would change based on the length of the list. Fix for #17179 And #27401 and #27506 [ghstack-poisoned]
Add support for treating Sequentials, ModuleLists, and ModuleDicts as iterables. As previously, when emitting a for loop over a Module Container we unroll the for loop over all elements. We require that any Sugared Value in an iterable with a Module Container have a statically - determinable length. Otherwise, if you zipped over a list of varying length and an nn.Sequential that alternated between returning a Tensor and a Dictionary, the output type would change based on the length of the list. Fix for #17179 And #27401 and #27506 [ghstack-poisoned]
Add support for treating Sequentials, ModuleLists, and ModuleDicts as iterables. As previously, when emitting a for loop over a Module Container we unroll the for loop over all elements. We require that any Sugared Value in an iterable with a Module Container have a statically - determinable length. Otherwise, if you zipped over a list of varying length and an nn.Sequential that alternated between returning a Tensor and a Dictionary, the output type would change based on the length of the list. Fix for #17179 And #27401 and #27506 [ghstack-poisoned]
Add support for treating Sequentials, ModuleLists, and ModuleDicts as iterables. As previously, when emitting a for loop over a Module Container we unroll the for loop over all elements. We require that any Sugared Value in an iterable with a Module Container have a statically - determinable length. Otherwise, if you zipped over a list of varying length and an nn.Sequential that alternated between returning a Tensor and a Dictionary, the output type would change based on the length of the list. Fix for #17179 And #27401 and #27506 [ghstack-poisoned]
zdevito
approved these changes
Nov 1, 2019
Contributor
zdevito
left a comment
zdevito
left a comment
There was a problem hiding this comment.
This looks good. The two substantive comments are: avoid using more constant prop than we were before for ranges, and lets make statically determined lengths also the same as when we unroll loops to avoid introducing an exponential number of cases to handle.
| namespace torch { | ||
| namespace jit { | ||
|
|
||
| c10::optional<Stack> tryConstantPropNode(const Node* node) { |
Contributor
There was a problem hiding this comment.
This is missing all of constant props checks that the node itself can be constant-propagated. Doesn't seem safe to use.
Contributor
There was a problem hiding this comment.
It seems like this method is actually use as runNodeIfInputsAreConstant which has a different implicit contract. Should document what this will do. What is Print(4) expected to do here?
Add support for treating Sequentials, ModuleLists, and ModuleDicts as iterables. As previously, when emitting a for loop over a Module Container we unroll the for loop over all elements. We require that any Sugared Value in an iterable with a Module Container have a statically - determinable length. Otherwise, if you zipped over a list of varying length and an nn.Sequential that alternated between returning a Tensor and a Dictionary, the output type would change based on the length of the list. Fix for #17179 And #27401 and #27506 [ghstack-poisoned]
Add support for treating Sequentials, ModuleLists, and ModuleDicts as iterables. As previously, when emitting a for loop over a Module Container we unroll the for loop over all elements. We require that any Sugared Value in an iterable with a Module Container have a statically - determinable length. Otherwise, if you zipped over a list of varying length and an nn.Sequential that alternated between returning a Tensor and a Dictionary, the output type would change based on the length of the list. Fix for #17179 And #27401 and #27506 [ghstack-poisoned]
zdevito
pushed a commit
to zdevito/ATen
that referenced
this pull request
Nov 4, 2019
Summary: Pull Request resolved: pytorch/pytorch#28255 Add support for treating Sequentials, ModuleLists, and ModuleDicts as iterables. As previously, when emitting a for loop over a Module Container we unroll the for loop over all elements. We require that any Sugared Value in an iterable with a Module Container have a statically - determinable length. Otherwise, if you zipped over a list of varying length and an nn.Sequential that alternated between returning a Tensor and a Dictionary, the output type would change based on the length of the list. Fix for #17179 And pytorch/pytorch#27401 and pytorch/pytorch#27506 Test Plan: Imported from OSS Reviewed By: ZolotukhinM Differential Revision: D18278124 Pulled By: eellison fbshipit-source-id: aca336a5b8da89c756b1f0884883649510cbde3c
Contributor
1 similar comment
Contributor
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Add support for treating Sequentials, ModuleLists, and ModuleDicts as iterables.
As previously, when emitting a for loop over a Module Container we unroll the for loop over all elements. We require that any Sugared Value in an iterable with a Module Container have a statically - determinable length.
Otherwise, if you zipped over a list of varying length and an nn.Sequential that alternated between returning a Tensor and a Dictionary, the output type would change based on the length of the list.
Fix for #17179
And #27401
and #27506