[dynamo, nested graph breaks] move cell codegen before side effects codegen #160601
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…odegen [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/160601
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 40c3651 with merge base df640df ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This was referenced Aug 12, 2025
This was referenced Aug 14, 2025
…e effects codegen" [ghstack-poisoned]
…e effects codegen" [ghstack-poisoned]
…e effects codegen" [ghstack-poisoned]
…e effects codegen" [ghstack-poisoned]
…e effects codegen" [ghstack-poisoned]
…e effects codegen" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames Lucaskabela [ghstack-poisoned]
…e effects codegen" This is needed because if we codegen cells for nested frames AFTER side effects, then reconstruction could get messed up. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames Lucaskabela [ghstack-poisoned]
…e effects codegen" This is needed because if we codegen cells for nested frames AFTER side effects, then reconstruction could get messed up. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames Lucaskabela [ghstack-poisoned]
…e effects codegen" This is needed because if we codegen cells for nested frames AFTER side effects, then reconstruction could get messed up. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames Lucaskabela [ghstack-poisoned]
Lucaskabela
reviewed
Sep 26, 2025
| return [create_instruction("CALL_FUNCTION", arg=nargs)] | ||
|
|
||
|
|
||
| def create_call_function_ex(flags: int, push_null: bool) -> list[Instruction]: |
Contributor
There was a problem hiding this comment.
I think it is worth adding a comment on the method api that push_null is a no-op in py version 3.10
Lucaskabela
requested changes
Sep 26, 2025
Contributor
Lucaskabela
left a comment
Lucaskabela
left a comment
There was a problem hiding this comment.
Seems the flexAttention tests are failing, although from the error message I am not sure if that is from this PR or not - can you rebase?
Lucaskabela
reviewed
Sep 26, 2025
|
|
||
| from .. import graph_break_hints, polyfills, variables | ||
| from ..bytecode_transformation import create_call_function, create_instruction | ||
| from ..bytecode_transformation import ( |
Member
Author
|
I think a bunch of the failures comes from #163503, which I'm still trying to resolve. |
…e effects codegen" This is needed because if we codegen cells for nested frames AFTER side effects, then reconstruction could get messed up. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames Lucaskabela [ghstack-poisoned]
mlazos
reviewed
Sep 29, 2025
torch/_dynamo/output_graph.py
Outdated
| # see symbolic_convert.py for | ||
| # codegen stack convention after the unsupported instruction | ||
| # NOTE: cells are loaded into continuation functions directly | ||
| # NOTE: cells will loaded into continuation functions directly by symbolic_convert |
mlazos
approved these changes
Sep 29, 2025
Contributor
mlazos
left a comment
mlazos
left a comment
There was a problem hiding this comment.
Looks good, can you add an explicit example in the PR desc when reconstruction will get messed up?
Member
Author
|
The added test case demonstrates the reconstruction failure if we kept cell codegen at the original place (only happens with nested graph breaks since we reconstruct nested frame cells from VariableTracker rather than directly using LOAD_CLOSURE). At a high level, what happened before this change was that side_effects was pruning the cells (I don't recall exactly why this happens), and because cells were codegen'd after the side effects were applied, we were unable to properly reconstruct the cell. The error I was seeing was a list/tuple IndexError. |
…e effects codegen" This is needed because if we codegen cells for nested frames AFTER side effects, then reconstruction could get messed up. From below: >The added test case demonstrates the reconstruction failure if we kept cell codegen at the original place (only happens with nested graph breaks since we reconstruct nested frame cells from VariableTracker rather than directly using LOAD_CLOSURE). >At a high level, what happened before this change was that side_effects was pruning the cells (I don't recall exactly why this happens), and because cells were codegen'd after the side effects were applied, we were unable to properly reconstruct the cell. The error I was seeing was a list/tuple IndexError. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames Lucaskabela [ghstack-poisoned]
…e effects codegen" This is needed because if we codegen cells for nested frames AFTER side effects, then reconstruction could get messed up. From below: >The added test case demonstrates the reconstruction failure if we kept cell codegen at the original place (only happens with nested graph breaks since we reconstruct nested frame cells from VariableTracker rather than directly using LOAD_CLOSURE). >At a high level, what happened before this change was that side_effects was pruning the cells (I don't recall exactly why this happens), and because cells were codegen'd after the side effects were applied, we were unable to properly reconstruct the cell. The error I was seeing was a list/tuple IndexError. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames Lucaskabela [ghstack-poisoned]
Collaborator
|
Starting merge as part of PR stack under #162737 |
pytorchmergebot
pushed a commit
that referenced
this pull request
Oct 8, 2025
…ues (#162737) Turns out codegen'ing a nested step graph break is significantly more complicated than first thought. The optimized function should actually do: - call graph/load values/do side effects etc. - call into the leaf's resume function, but skipped (this essentially step graph break function for just the leaf function) - call into all the other resume functions, traced. This PR also adds `torch._dynamo.step_unsupported()`, which can be used for internal testing purposes to better test step graph break handling. Pull Request resolved: #162737 Approved by: https://github.com/Lucaskabela ghstack dependencies: #160601
Chao1Han
pushed a commit
to Chao1Han/pytorch
that referenced
this pull request
Oct 21, 2025
…odegen (pytorch#160601) This is needed because if we codegen cells for nested frames AFTER side effects, then reconstruction could get messed up. From below: >The added test case demonstrates the reconstruction failure if we kept cell codegen at the original place (only happens with nested graph breaks since we reconstruct nested frame cells from VariableTracker rather than directly using LOAD_CLOSURE). >At a high level, what happened before this change was that side_effects was pruning the cells (I don't recall exactly why this happens), and because cells were codegen'd after the side effects were applied, we were unable to properly reconstruct the cell. The error I was seeing was a list/tuple IndexError. Pull Request resolved: pytorch#160601 Approved by: https://github.com/mlazos
Chao1Han
pushed a commit
to Chao1Han/pytorch
that referenced
this pull request
Oct 21, 2025
…ues (pytorch#162737) Turns out codegen'ing a nested step graph break is significantly more complicated than first thought. The optimized function should actually do: - call graph/load values/do side effects etc. - call into the leaf's resume function, but skipped (this essentially step graph break function for just the leaf function) - call into all the other resume functions, traced. This PR also adds `torch._dynamo.step_unsupported()`, which can be used for internal testing purposes to better test step graph break handling. Pull Request resolved: pytorch#162737 Approved by: https://github.com/Lucaskabela ghstack dependencies: pytorch#160601
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
This is needed because if we codegen cells for nested frames AFTER side effects, then reconstruction could get messed up. From below:
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @amjames @Lucaskabela