Log exception_stack_trace to dynamo_compile #161096
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/161096
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit ecea5cb with merge base 24e7f3c ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@pytorchbot label "topic: not user facing" |
|
For adding a unit test, I expect if you turn on suppress errors, nothing will throw, but also we can just catch the exception that gets thrown within the unit tests? (assertRaises, or just try except). In order to generate the error, I expect unsupported exceptions should work fine -I.e. LSTMs are not supported so torch.compiling one of those should produce a log with a exception. |
The LSTM compilation didn't throw and exception. Modeled a graph break and exception throw because of a print statement in backward. |
Added. |
test/dynamo/test_utils.py
Outdated
| print("graph break!") # This should trigger a Dynamo error | ||
| return grad_output | ||
|
|
||
| CustomFuncBwdPrintGraphBreak = type( |
There was a problem hiding this comment.
Nit - This is all mostly redundant, I expect that you can either directly torch.compile the backward function and get an error, or worse comes to worse, make a nn module, and have it's forward direction directly call print.
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Summary: Note: Adding unit test for this is tricky as having errors in the specific unit test would cause test_utils.py to crash all together. Tested as follows: 1. Added x = 1/0 after guarded_code = compile_inner(code, one_graph, hooks, transform) in convert_frame.py 2. Printed exception_stack_trace and got: ['Traceback (most recent call last):\n File "/data/users/jovian/pytorch/torch/_dynamo/convert_frame.py", line 1207, in _compile\n x = 1/0\n ~^~\nZeroDivisionError: division by zero\n'] X-link: pytorch/pytorch#161096 Approved by: https://github.com/c00w Reviewed By: seemethere Differential Revision: D80776058 fbshipit-source-id: 3531dc2df17f8f18a78c13551950563ef3a46059
Note: Adding unit test for this is tricky as having errors in the specific unit test would cause test_utils.py to crash all together. Tested as follows: 1. Added x = 1/0 after guarded_code = compile_inner(code, one_graph, hooks, transform) in convert_frame.py 2. Printed exception_stack_trace and got: ['Traceback (most recent call last):\n File "/data/users/jovian/pytorch/torch/_dynamo/convert_frame.py", line 1207, in _compile\n x = 1/0\n ~^~\nZeroDivisionError: division by zero\n'] Pull Request resolved: pytorch#161096 Approved by: https://github.com/c00w
3 participants
Note: Adding unit test for this is tricky as having errors in the specific unit test would cause test_utils.py to crash all together.
Tested as follows:
^\nZeroDivisionError: division by zero\n']cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @amjames @Lucaskabela