[inductor] performance model interface #162017

coconutruben · 2025-09-02T22:29:20Z

Stack from ghstack (oldest at bottom):

why

enable the system to register performance model ranking functions
those functions are then used to rank th choices before benchmarking
enable more modular, faster, more expansive autotuning

what

PerformanceModelChoices, expansion of inductor choices
mechanism
- users can register functions (through decorators or registration
  function)
- functions are hardware, template, op specific
- when an input hits the performance model, if we have any functions
  registered for that template/op combo
  - we expand the config space for that template
  - we ask the function to rank all the choices
  - we forward filter down to topk
control
- topk: -1 to say "same as default search space" or > 1 for specific
- topk: 0 turns off any performance modeling
- discard unranked: controls what to do with choices that have not
  been estimated (discard, or keep)

You can check out the testing code to see examples of toy functions and
their integration into the system

This choice handler is not turned on by default

Even if the choice handler is turned on, it will be a noop if no
functions are registered, or if topk = 0

testing

python -m pytest test/inductor/test_performance_model_interface_e2e.py
python -m pytest test/inductor/test_performance_model_interface.py

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov @mlazos

Differential Revision: D81534564

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` [ghstack-poisoned]

pytorch-bot · 2025-09-02T22:29:25Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162017

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

❌ 33 New Failures

As of commit 15c0dda with merge base d25c35d ():

NEW FAILURES - The following jobs have failed:

inductor / unit-test / inductor-cpu-test / test (inductor_amx, 2, 2, linux.8xlarge.amx) (gh)
inductor/test_torchinductor.py::CpuTests::test_graph_partition_mutation_real_name_cpu
inductor / unit-test / inductor-cpu-test / test (inductor_avx2, 2, 2, linux.10xlarge.avx2) (gh)
inductor/test_torchinductor.py::CpuTests::test_graph_partition_mutation_real_name_cpu
inductor / unit-test / inductor-halide-test / test (inductor-halide, 1, 1, linux.12xlarge) (gh)
inductor/test_halide.py::HalideCpuTests::test_graph_partition_mutation_real_name_cpu_halide
inductor / unit-test / inductor-test / test (inductor, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_baddbmm_cuda_float32
inductor / unit-test / linux-jammy-cpu-py3.12-gcc11-inductor-triton-cpu / test (inductor-triton-cpu, 1, 1, linux.12xlarge) (gh)
inductor/test_triton_cpu_backend.py::CpuTritonTests::test_graph_partition_mutation_real_name_cpu
Lint / lintrunner-noclang / linux-job (gh)
>>> Lint for torch/_inductor/performance_model/choices.py:
linux-aarch64 / linux-jammy-aarch64-py3.10 / test (default, 1, 3, lf.linux.arm64.m7g.4xlarge) (gh)
inductor/test_torchinductor.py::CpuTests::test_graph_partition_mutation_real_name_cpu
linux-aarch64 / linux-jammy-aarch64-py3.10 / test (default, 1, 3, lf.linux.arm64.m8g.4xlarge) (gh)
inductor/test_torchinductor.py::CpuTests::test_graph_partition_mutation_real_name_cpu
linux-aarch64 / linux-jammy-aarch64-py3.10 / test (default, 2, 3, lf.linux.arm64.m7g.4xlarge) (gh)
inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_graph_partition_mutation_real_name_dynamic_shapes_cpu
linux-aarch64 / linux-jammy-aarch64-py3.10 / test (default, 2, 3, lf.linux.arm64.m8g.4xlarge) (gh)
inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_graph_partition_mutation_real_name_dynamic_shapes_cpu
pull / linux-jammy-py3.10-clang12 / test (default, 2, 5, lf.linux.4xlarge) (gh)
inductor/test_cpu_select_algorithm.py::TestSelectAlgorithmDynamicShapesCPU::test_linear_dynamic_shapes_batch_size_2_in_features_1_out_features_1_bias_True_input_3d_False_cpu_float16
pull / linux-jammy-py3.10-clang12 / test (default, 3, 5, lf.linux.4xlarge) (gh)
inductor/test_torchinductor.py::CpuTests::test_graph_partition_mutation_real_name_cpu
pull / linux-jammy-py3.10-clang12 / test (default, 4, 5, lf.linux.4xlarge) (gh)
inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_graph_partition_mutation_real_name_dynamic_shapes_cpu
pull / linux-jammy-py3.10-clang12 / test (default, 5, 5, lf.linux.4xlarge) (gh)
inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_graph_partition_mutation_real_name_dynamic_shapes_cpu
pull / linux-jammy-py3.10-clang18-asan / test (default, 2, 7, lf.linux.4xlarge) (gh)
inductor/test_cpu_select_algorithm.py::TestSelectAlgorithmCPU::test_linear_static_shapes_batch_size_1_in_features_1000_out_features_1_bias_True_input_3d_True_cpu_float16
pull / linux-jammy-py3.10-clang18-asan / test (default, 3, 7, lf.linux.4xlarge) (gh)
inductor/test_torchinductor.py::CpuTests::test_graph_partition_mutation_real_name_cpu
pull / linux-jammy-py3.10-clang18-asan / test (default, 4, 7, lf.linux.4xlarge) (gh)
inductor/test_cpu_select_algorithm.py::TestSelectAlgorithmCPU::test_linear_static_shapes_batch_size_1000_in_features_1000_out_features_1_bias_True_input_3d_True_cpu_float16
pull / linux-jammy-py3.10-clang18-asan / test (default, 5, 7, lf.linux.4xlarge) (gh)
inductor/test_cpu_select_algorithm.py::TestSelectAlgorithmDynamicShapesCPU::test_linear_dynamic_shapes_batch_size_1_in_features_1000_out_features_1_bias_True_input_3d_True_cpu_float16
pull / linux-jammy-py3.10-clang18-asan / test (default, 6, 7, lf.linux.4xlarge) (gh)
inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_graph_partition_mutation_real_name_dynamic_shapes_cpu
pull / linux-jammy-py3.10-gcc11 / test (default, 2, 5, lf.linux.2xlarge) (gh)
inductor/test_cpu_select_algorithm.py::TestSelectAlgorithmDynamicShapesCPU::test_linear_dynamic_shapes_batch_size_2_in_features_1000_out_features_1_bias_True_input_3d_True_cpu_float16
pull / linux-jammy-py3.10-gcc11 / test (default, 3, 5, lf.linux.2xlarge) (gh)
inductor/test_torchinductor.py::CpuTests::test_graph_partition_mutation_real_name_cpu
pull / linux-jammy-py3.10-gcc11 / test (default, 4, 5, lf.linux.2xlarge) (gh)
inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_graph_partition_mutation_real_name_dynamic_shapes_cpu
pull / linux-jammy-py3.10-gcc11 / test (default, 5, 5, lf.linux.2xlarge) (gh)
inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_graph_partition_mutation_real_name_dynamic_shapes_cpu
pull / linux-jammy-py3.13-clang12 / test (default, 2, 5, lf.linux.4xlarge) (gh)
inductor/test_cpu_select_algorithm.py::TestSelectAlgorithmCPU::test_linear_static_shapes_batch_size_1_in_features_1_out_features_1_bias_True_input_3d_True_cpu_float16
pull / linux-jammy-py3.13-clang12 / test (default, 3, 5, lf.linux.4xlarge) (gh)
inductor/test_torchinductor.py::CpuTests::test_graph_partition_mutation_real_name_cpu
pull / linux-jammy-py3.13-clang12 / test (default, 4, 5, lf.linux.4xlarge) (gh)
inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_graph_partition_mutation_real_name_dynamic_shapes_cpu
pull / linux-jammy-py3.13-clang12 / test (default, 5, 5, lf.linux.4xlarge) (gh)
inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_graph_partition_mutation_real_name_dynamic_shapes_cpu
trunk / linux-jammy-cuda12.8-py3.10-gcc11 / test (default, 3, 5, linux.g6.4xlarge.experimental.nvidia.gpu) (gh)
inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_graph_partition_mutation_real_name_dynamic_shapes_cpu
trunk / linux-jammy-cuda12.8-py3.10-gcc11 / test (default, 5, 5, linux.g6.4xlarge.experimental.nvidia.gpu) (gh)
inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_graph_partition_mutation_real_name_dynamic_shapes_cpu
trunk / macos-py3-arm64 / test (default, 2, 3, macos-m1-stable) (gh)
inductor/test_torchinductor.py::CpuTests::test_graph_partition_mutation_real_name_cpu
trunk / macos-py3-arm64 / test (default, 3, 3, macos-m1-stable) (gh)
inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_graph_partition_mutation_real_name_dynamic_shapes_cpu
trunk / macos-py3-arm64 / test (mps, 1, 1, macos-m1-14) (gh)
inductor/test_torchinductor.py::CpuTests::test_graph_partition_mutation_real_name_cpu
trunk / macos-py3-arm64 / test (mps, 1, 1, macos-m2-15) (gh)
inductor/test_torchinductor.py::CpuTests::test_graph_partition_mutation_real_name_cpu

This comment was automatically generated by Dr. CI and updates every 15 minutes.

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` ghstack-source-id: 30557be Pull Request resolved: #162017

coconutruben · 2025-09-02T22:34:39Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning # what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 # testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov mlazos Differential Revision: [D81534564](https://our.internmc.facebook.com/intern/diff/D81534564) [ghstack-poisoned]

coconutruben · 2025-09-05T22:24:10Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning # what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 # testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov mlazos Differential Revision: [D81534564](https://our.internmc.facebook.com/intern/diff/D81534564) [ghstack-poisoned]

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` ghstack-source-id: d1b6f9b Pull Request resolved: #162017

coconutruben · 2025-09-05T22:31:42Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning # what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 # testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov mlazos Differential Revision: [D81534564](https://our.internmc.facebook.com/intern/diff/D81534564) [ghstack-poisoned]

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` ghstack-source-id: a313b98 Pull Request resolved: #162017

# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning # what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 # testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov mlazos Differential Revision: [D81534564](https://our.internmc.facebook.com/intern/diff/D81534564) [ghstack-poisoned]

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` ghstack-source-id: 1e45804 Pull Request resolved: #162017

coconutruben · 2025-09-09T16:13:21Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning # what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 # testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov mlazos Differential Revision: [D81534564](https://our.internmc.facebook.com/intern/diff/D81534564) [ghstack-poisoned]

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` ghstack-source-id: 07f44dc Pull Request resolved: #162017

# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning # what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 # testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov mlazos Differential Revision: [D81534564](https://our.internmc.facebook.com/intern/diff/D81534564) [ghstack-poisoned]

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` ghstack-source-id: 1e2e67b Pull Request resolved: #162017

# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning # what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 # testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov mlazos Differential Revision: [D81534564](https://our.internmc.facebook.com/intern/diff/D81534564) [ghstack-poisoned]

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` ghstack-source-id: d99e756 Pull Request resolved: #162017

coconutruben · 2025-09-10T18:01:52Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning # what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 # testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov mlazos Differential Revision: [D81534564](https://our.internmc.facebook.com/intern/diff/D81534564) [ghstack-poisoned]

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` ghstack-source-id: e70c6a5 Pull Request resolved: #162017

# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning # what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 # testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov mlazos Differential Revision: [D81534564](https://our.internmc.facebook.com/intern/diff/D81534564) [ghstack-poisoned]

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` ghstack-source-id: c784cfc Pull Request resolved: #162017

coconutruben · 2025-09-11T22:53:14Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning # what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 # testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov mlazos Differential Revision: [D81534564](https://our.internmc.facebook.com/intern/diff/D81534564) [ghstack-poisoned]

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` ghstack-source-id: 11630fd Pull Request resolved: #162017

# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning # what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 # testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov mlazos Differential Revision: [D81534564](https://our.internmc.facebook.com/intern/diff/D81534564) [ghstack-poisoned]

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` ghstack-source-id: 79dc51d Pull Request resolved: #162017

# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning # what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 # testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov mlazos Differential Revision: [D81534564](https://our.internmc.facebook.com/intern/diff/D81534564) [ghstack-poisoned]

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` ghstack-source-id: 3a9b498 Pull Request resolved: #162017

\# why - enable the system to register performance model ranking functions - those functions are then used to rank th choices before benchmarking - enable more modular, faster, more expansive autotuning \# what - PerformanceModelChoices, expansion of inductor choices - mechanism - users can register functions (through decorators or registration function) - functions are hardware, template, op specific - when an input hits the performance model, if we have any functions registered for that template/op combo - we expand the config space for that template - we ask the function to rank all the choices - we forward filter down to topk - control - topk: -1 to say "same as default search space" or > 1 for specific - topk: 0 turns off any performance modeling - discard unranked: controls what to do with choices that have not been estimated (discard, or keep) You can check out the testing code to see examples of toy functions and their integration into the system This choice handler is not turned on by default Even if the choice handler is turned on, it will be a noop if no functions are registered, or if topk = 0 \# testing ``` python -m pytest test/inductor/test_performance_model_interface_e2e.py python -m pytest test/inductor/test_performance_model_interface.py ``` ghstack-source-id: ddc0529 Pull Request resolved: pytorch/pytorch#162017

This was referenced Sep 2, 2025

[inductor] add notion of extra_kwargs for mm_configs #161123

Closed

[inductor] move tma workspace in heuristics #161124

Closed

[inductor] move addmm/baddbmm template args into heuristics #161125

Closed

pytorch-bot bot added ciflow/inductor module: inductor labels Sep 2, 2025

coconutruben mentioned this pull request Sep 2, 2025

[inductor] move scaled_mm template args into heuristics #161126

Closed

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 2, 2025

coconutruben added the release notes: inductor label Sep 2, 2025

coconutruben mentioned this pull request Sep 3, 2025

[inductor][contigous mm] mild refactor #162075

Closed

coconutruben closed this Sep 18, 2025

github-actions bot deleted the gh/coconutruben/62/head branch October 19, 2025 02:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[inductor] performance model interface #162017

[inductor] performance model interface #162017

Uh oh!

coconutruben commented Sep 2, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Sep 2, 2025 •

edited

Loading

Uh oh!

coconutruben commented Sep 2, 2025

Uh oh!

coconutruben commented Sep 5, 2025

Uh oh!

coconutruben commented Sep 5, 2025

Uh oh!

coconutruben commented Sep 9, 2025

Uh oh!

coconutruben commented Sep 10, 2025

Uh oh!

coconutruben commented Sep 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

[inductor] performance model interface #162017

[inductor] performance model interface #162017

Uh oh!

Conversation

coconutruben commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

why

what

testing

Uh oh!

pytorch-bot bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162017

❌ 33 New Failures

Uh oh!

coconutruben commented Sep 2, 2025

Uh oh!

coconutruben commented Sep 5, 2025

Uh oh!

coconutruben commented Sep 5, 2025

Uh oh!

coconutruben commented Sep 9, 2025

Uh oh!

coconutruben commented Sep 10, 2025

Uh oh!

coconutruben commented Sep 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

coconutruben commented Sep 2, 2025 •

edited

Loading

pytorch-bot bot commented Sep 2, 2025 •

edited

Loading