[inductor] autotune benchmark support for cpu #125159
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/125159
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (1 Unrelated Failure)As of commit 7111f21 with merge base 5007312 ( FLAKY - The following job failed but was likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
cc ezyang msaroufim bdhirsh anijain2305 chauhang voznesenskym penguinwu EikanWang Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire [ghstack-poisoned]
pytorch-bot
bot
added
ciflow/linux-aarch64
There was a problem hiding this comment.
Do submodule update in a different PR.
There was a problem hiding this comment.
Thanks. A wrong "git add". Fixed.
| if is_cpu_device(args): | ||
| return do_bench_cpu(lambda: algo(*args, out=out)) | ||
| else: | ||
| return do_bench(lambda: algo(*args, out=out)) |
There was a problem hiding this comment.
Can we have this device dispatch logic inside do_bench?
Basically, rename current do_bench to do_bench_gpu. And create a new do_bench function that dispatches to do_bench_cpu/do_bench_gpu based on devices.
There was a problem hiding this comment.
Thanks. I did the refactoring in a separate PR: #125736
This PR adds the autotune Infrastructure for CPU. It generalizes and extends `BenchmarkRequest` with CPU support and C++ module loader. A `do_bench_cpu` util function is added for benchmarking functions on CPU with warmups and returns the median number from multiple trials. cc gujinghui PenghuiCheng XiaobingSuper jianyuh mingfeima sanchitintel ashokei jingxu10 min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen snadampal ezyang msaroufim bdhirsh anijain2305 chauhang voznesenskym penguinwu EikanWang zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire [ghstack-poisoned]
This PR adds the autotune Infrastructure for CPU. It generalizes and extends `BenchmarkRequest` with CPU support and C++ module loader. A `do_bench_cpu` util function is added for benchmarking functions on CPU with warmups and returns the median number from multiple trials. cc gujinghui PenghuiCheng XiaobingSuper jianyuh mingfeima sanchitintel ashokei jingxu10 min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen snadampal ezyang msaroufim bdhirsh anijain2305 chauhang voznesenskym penguinwu EikanWang zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire [ghstack-poisoned]
This PR adds the autotune Infrastructure for CPU. It generalizes and extends `BenchmarkRequest` with CPU support and C++ module loader. A `do_bench_cpu` util function is added for benchmarking functions on CPU with warmups and returns the median number from multiple trials. cc gujinghui PenghuiCheng XiaobingSuper jianyuh mingfeima sanchitintel ashokei jingxu10 min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen snadampal ezyang msaroufim bdhirsh anijain2305 chauhang voznesenskym penguinwu EikanWang zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire [ghstack-poisoned]
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
we do have api depend on |
I thought it was only used by inductor internal. You may have to rename it to |
Stack from ghstack (oldest at bottom):
This PR adds the autotune Infrastructure for CPU. It generalizes and extends
BenchmarkRequestwith CPU support and C++ module loader. Ado_bench_cpuutil function is added for benchmarking functions on CPU with warmups and returns the median number from multiple trials.cc @gujinghui @PenghuiCheng @XiaobingSuper @jianyuh @mingfeima @sanchitintel @ashokei @jingxu10 @min-jean-cho @yanbing-j @Guobing-Chen @Xia-Weiwen @snadampal @ezyang @msaroufim @bdhirsh @anijain2305 @chauhang @voznesenskym @penguinwu @EikanWang @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire