[inductor] refactor: device dispatch inside do_bench #125736
Conversation
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/125736
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (1 Unrelated Failure)As of commit 1bff9d0 with merge base aaa2f93 ( BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
cc voznesenskym penguinwu EikanWang Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang [ghstack-poisoned]
cc voznesenskym penguinwu EikanWang Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang [ghstack-poisoned]
| return do_bench_gpu(lambda: fn(*fn_args, **fn_kwargs), **kwargs) | ||
|
|
||
|
|
||
| def do_bench_gpu(fn, **kwargs): |
There was a problem hiding this comment.
Why change *args to fn? I think the former way is more flexible.
cc voznesenskym penguinwu EikanWang Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang [ghstack-poisoned]
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
Hey @jgong5, please note that |
Summary: We are relying on some private APIs from inductor and a recent refactor: pytorch/pytorch#125736 broken the do_bench API we rely on for autoquant, maybe we should use our own do_bench or rely on triton's directly? Test Plan: regression tests python test/integration/test_integration.py -k test_autoquant_one_input_29_cuda Reviewers: Subscribers: Tasks: Tags:
Thanks for the reminder. Let me fix it with a PR. But may I know why it was not failing the CI? |
Apology for that. Is this what @jerryzh168 is fixing with pytorch/ao#242? |
Summary: We are relying on some private APIs from inductor and a recent refactor: pytorch/pytorch#125736 broken the do_bench API we rely on for autoquant, maybe we should use our own do_bench or rely on triton's directly? Test Plan: regression tests python test/integration/test_integration.py -k test_autoquant_one_input_29_cuda Reviewers: Subscribers: Tasks: Tags:
yeah this PR should fix the issue for torchao, but going forward I feel maybe we want to have a more public do_bench API that everyone can rely on |
Here is the PR to fix this: #126224. Please review. @izaitsevfb |
Summary: We are relying on some private APIs from inductor and a recent refactor: pytorch/pytorch#125736 broken the do_bench API we rely on for autoquant, maybe we should use our own do_bench or rely on triton's directly? Test Plan: regression tests python test/integration/test_integration.py -k test_autoquant_one_input_29_cuda Reviewers: Subscribers: Tasks: Tags:
Stack from ghstack (oldest at bottom):
cc @voznesenskym @penguinwu @EikanWang @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang