always mark 1st dimension dynamic #137612

bobrenjc93 · 2024-10-09T18:04:22Z

Stack from ghstack (oldest at bottom):

-> always mark 1st dimension dynamic #137612

cc @XilunWu @H-Huang @awgu @kwen2501 @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @c-p-i-o @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang @rec

[ghstack-poisoned]

pytorch-bot · 2024-10-09T18:04:26Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/137612

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 85 New Failures, 1 Unrelated Failure

As of commit 1698bd0 with merge base 8321eec ():

NEW FAILURES - The following jobs have failed:

inductor / cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
hf_T5_generate
inductor / cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
vision_maskrcnn
inductor / cuda12.1-py3.10-gcc9-sm86 / test (inductor_cpp_wrapper_abi_compatible, 1, 1, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_cpu_cpp_wrapper.py::TestCppWrapper::test_linear_with_pointwise_batch_size_384_in_features_196_out_features_384_bias_False_epilogue_add_cpu_float32_cpp_wrapper
inductor / cuda12.1-py3.10-gcc9-sm86 / test (inductor_distributed, 1, 1, linux.g5.12xlarge.nvidia.gpu) (gh)
distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_gather_into_tensor_single
inductor / cuda12.1-py3.10-gcc9-sm86 / test (inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
hf_T5_generate
inductor / cuda12.1-py3.10-gcc9-sm86 / test (inductor_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
vision_maskrcnn
inductor / cuda12.1-py3.10-gcc9-sm86 / test (inductor, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_torchinductor.py::CpuTests::test_mutable_custom_op_fixed_layout2_cpu
inductor / cuda12.1-py3.10-gcc9-sm86 / test (inductor, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_euclidean_large_cuda
inductor / cuda12.1-py3.12-gcc9-sm86 / test (inductor, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_torchinductor.py::CpuTests::test_mutable_custom_op_fixed_layout2_cpu
inductor / cuda12.1-py3.12-gcc9-sm86 / test (inductor, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_euclidean_large_cuda
inductor / linux-jammy-cpu-py3.12-gcc11-inductor-halide / test (inductor-halide, 1, 1, linux.12xlarge) (gh)
inductor/test_halide.py::CpuHalideTests::test_kwargs_cpu
inductor / linux-jammy-cpu-py3.12-gcc11-inductor-triton-cpu / test (inductor-triton-cpu, 1, 1, linux.12xlarge) (gh)
inductor/test_triton_cpu_backend.py::CpuTritonTests::test_kwargs_cpu
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (cpu_inductor_amp_freezing_torchbench, 1, 2, linux.16xlarge.spr) (gh)
detectron2_maskrcnn_r_50_fpn
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (cpu_inductor_amp_freezing_torchbench, 2, 2, linux.16xlarge.spr) (gh)
vision_maskrcnn
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (cpu_inductor_freezing_avx2_torchbench, 1, 2, linux.10xlarge.avx2) (gh)
detectron2_maskrcnn_r_50_fpn
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (cpu_inductor_freezing_avx2_torchbench, 2, 2, linux.10xlarge.avx2) (gh)
vision_maskrcnn
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (cpu_inductor_freezing_torchbench, 1, 2, linux.12xlarge) (gh)
detectron2_maskrcnn_r_50_fpn
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (cpu_inductor_freezing_torchbench, 2, 2, linux.12xlarge) (gh)
vision_maskrcnn
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (cpu_inductor_torchbench, 1, 2, linux.12xlarge) (gh)
detectron2_maskrcnn_r_50_fpn
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (cpu_inductor_torchbench, 2, 2, linux.12xlarge) (gh)
vision_maskrcnn
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (dynamic_cpu_inductor_torchbench, 1, 2, linux.12xlarge) (gh)
detectron2_fcos_r_50_fpn
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (dynamic_cpu_inductor_torchbench, 2, 2, linux.12xlarge) (gh)
vision_maskrcnn
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (inductor_avx2, 1, 2, linux.10xlarge.avx2) (gh)
inductor/test_torchinductor.py::CpuTests::test_kwargs_cpu
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (inductor_avx2, 2, 2, linux.10xlarge.avx2) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_combinations_cpu_int32
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (inductor_avx512, 1, 2, linux.12xlarge) (gh)
inductor/test_torchinductor.py::CpuTests::test_kwargs_cpu
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (inductor_avx512, 2, 2, linux.12xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_fft_ihfft_cpu_int64
inductor-periodic / cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) (gh)
hf_T5
inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
vision_maskrcnn
inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
torchrec_dlrm
inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
vision_maskrcnn
inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
vision_maskrcnn
pull / cuda12.1-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) (gh)
REGRESSION: benchmark ('add_loop_inductor', 'compile_time_instruction_count') failed, actual result 24525121051 is 1.57% higher than expected 24146845503 ±+1.50% if this is an expected regression, please update the expected results.
pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 1, 5, linux.4xlarge.nvidia.gpu) (gh)
dynamo/test_input_attr_tracking.py::TestInputAttrTracking::test_const_property_on_tensor
pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 2, 5, linux.4xlarge.nvidia.gpu) (gh)
dynamo/test_functions.py::FunctionTests::test_partials_graph_break_reconstruct
pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) (gh)
dynamo/test_aot_autograd.py::AotAutogradFallbackTests::test_multiple_aot_autograd_calls_dupe_args
pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 4, 5, linux.4xlarge.nvidia.gpu) (gh)
dynamo/test_backward_higher_order_ops.py::BackwardHigherOrderOpTests::test_invoke_in_pt2_compiled_autograd_side_effect
pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 5, 5, linux.4xlarge.nvidia.gpu) (gh)
dynamo/test_comptime.py::ComptimeTests::test_get_local
pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 1, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
dynamo/test_minifier.py::MinifierTests::test_after_dynamo_non_leaf_compile_error
pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 2, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
dynamo/test_higher_order_ops.py::HigherOrderOpTests::test_cond_branches_no_arguments
pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 3, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
dynamo/test_aot_autograd.py::AotAutogradFallbackTests::test_multiple_aot_autograd_calls_dupe_args
pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 4, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
dynamo/test_activation_checkpointing.py::ActivationCheckpointingViaTagsTests::test_compile_selective_checkpoint_random_op
pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 5, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
dynamo/test_comptime.py::ComptimeTests::test_get_local
pull / linux-focal-py3_9-clang9-xla / test (xla, 1, 1, linux.12xlarge) (gh)
test_manual_buffer_donation_for_non_inplce_op
pull / linux-focal-py3.11-clang10 / test (crossref, 1, 2, linux.2xlarge) (gh)
export/test_torchbind.py::TestCompileTorchbind::test_compile_obj_graph_breaks
pull / linux-focal-py3.11-clang10 / test (crossref, 2, 2, linux.2xlarge) (gh)
export/test_sparse.py::TestSparseProp::test_add
pull / linux-focal-py3.11-clang10 / test (default, 1, 4, linux.4xlarge) (gh)
dynamo/test_higher_order_ops.py::HigherOrderOpTests::test_cond_branches_no_arguments
pull / linux-focal-py3.11-clang10 / test (default, 2, 4, linux.4xlarge) (gh)
dynamo/test_config.py::ConfigTests::test_no_automatic_dynamic
pull / linux-focal-py3.11-clang10 / test (default, 3, 4, linux.4xlarge) (gh)
export/test_torchbind.py::TestCompileTorchbind::test_compile_obj_graph_breaks
pull / linux-focal-py3.11-clang10 / test (default, 4, 4, linux.4xlarge) (gh)
dynamo/test_backward_higher_order_ops.py::BackwardHigherOrderOpTests::test_invoke_in_pt2_compiled_autograd_side_effect
pull / linux-focal-py3.11-clang10 / test (dynamo, 1, 3, linux.2xlarge) (gh)
test_fake_tensor.py::FakeTensorTest::test_jagged_fake_to_fake_preserved
pull / linux-focal-py3.11-clang10 / test (dynamo, 2, 3, linux.2xlarge) (gh)
test_binary_ufuncs.py::TestBinaryUfuncsCPU::test_sub_typing_cpu
pull / linux-focal-py3.11-clang10 / test (dynamo, 3, 3, linux.2xlarge) (gh)
functorch/test_control_flow.py::TestControlFlow::test_scan_simple_graph
pull / linux-focal-py3.12-clang10 / test (default, 1, 4, linux.4xlarge) (gh)
dynamo/test_higher_order_ops.py::HigherOrderOpTests::test_cond_branches_no_arguments
pull / linux-focal-py3.12-clang10 / test (default, 2, 4, linux.4xlarge) (gh)
dynamo/test_aot_autograd.py::AotAutogradFallbackTests::test_multiple_aot_autograd_calls_dupe_args
pull / linux-focal-py3.12-clang10 / test (default, 3, 4, linux.4xlarge) (gh)
functorch/test_control_flow.py::TestControlFlow::test_scan_simple_graph
pull / linux-focal-py3.12-clang10 / test (default, 4, 4, linux.4xlarge) (gh)
dynamo/test_backward_higher_order_ops.py::BackwardHigherOrderOpTests::test_invoke_in_pt2_compiled_autograd_side_effect
pull / linux-focal-py3.12-clang10 / test (dynamo, 1, 3, linux.2xlarge) (gh)
test_fake_tensor.py::FakeTensorTest::test_jagged_fake_to_fake_preserved
pull / linux-focal-py3.12-clang10 / test (dynamo, 2, 3, linux.2xlarge) (gh)
test_binary_ufuncs.py::TestBinaryUfuncsCPU::test_sub_typing_cpu
pull / linux-focal-py3.12-clang10 / test (dynamo, 3, 3, linux.2xlarge) (gh)
functorch/test_control_flow.py::TestControlFlow::test_scan_simple_graph
pull / linux-focal-py3.12-clang10-experimental-split-build / test (default, 1, 3, linux.4xlarge) (gh)
functorch/test_aotdispatch.py::TestAOTModuleSimplified::test_grads_no_force_contiguous_nested_tensor_tangent
pull / linux-focal-py3.12-clang10-experimental-split-build / test (default, 2, 3, linux.4xlarge) (gh)
dynamo/test_backward_higher_order_ops.py::BackwardHigherOrderOpTests::test_invoke_in_pt2_compiled_autograd_side_effect
pull / linux-focal-py3.12-clang10-experimental-split-build / test (default, 3, 3, linux.4xlarge) (gh)
dynamo/test_higher_order_ops.py::HigherOrderOpTests::test_cond_branches_no_arguments
pull / linux-focal-py3.12-clang10-experimental-split-build / test (dynamo, 1, 3, linux.2xlarge) (gh)
test_fake_tensor.py::FakeTensorTest::test_jagged_fake_to_fake_preserved
pull / linux-focal-py3.12-clang10-experimental-split-build / test (dynamo, 2, 3, linux.2xlarge) (gh)
functorch/test_vmap.py::TestRandomnessCPU::test_alpha_dropout_randomness_error_batched_input_none_cpu
pull / linux-focal-py3.12-clang10-experimental-split-build / test (dynamo, 3, 3, linux.2xlarge) (gh)
functorch/test_aotdispatch.py::TestAOTAutograd::test_input_mutation_aliases_and_none_require_gradients
pull / linux-focal-py3.9-clang10 / test (crossref, 1, 2, linux.2xlarge) (gh)
export/test_sparse.py::TestSparseProp::test_eltwisenet_bfloat16_int32_SparseBSC
pull / linux-focal-py3.9-clang10 / test (crossref, 2, 2, linux.2xlarge) (gh)
export/test_export.py::TestExport::test_export_as_backend
pull / linux-focal-py3.9-clang10 / test (default, 1, 4, linux.4xlarge) (gh)
dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_do_paste_mask_dynamic_shapes
pull / linux-focal-py3.9-clang10 / test (default, 2, 4, linux.4xlarge) (gh)
dynamo/test_config.py::ConfigTests::test_no_automatic_dynamic
pull / linux-focal-py3.9-clang10 / test (default, 3, 4, linux.4xlarge) (gh)
dynamo/test_higher_order_ops.py::HigherOrderOpTests::test_cond_branches_no_arguments
pull / linux-focal-py3.9-clang10 / test (default, 4, 4, linux.4xlarge) (gh)
dynamo/test_backward_higher_order_ops.py::BackwardHigherOrderOpTests::test_invoke_in_pt2_compiled_autograd_side_effect
pull / linux-focal-py3.9-clang10 / test (dynamo, 1, 3, linux.2xlarge) (gh)
test_fake_tensor.py::FakeTensorTest::test_jagged_fake_to_fake_preserved
pull / linux-focal-py3.9-clang10 / test (dynamo, 2, 3, linux.2xlarge) (gh)
functorch/test_eager_transforms.py::TestExamplesCorrectnessCPU::test_find_learning_rate_ensembling_AlphaDropout_mechanism_functional_call_cpu
pull / linux-focal-py3.9-clang10 / test (dynamo, 3, 3, linux.2xlarge) (gh)
functorch/test_aotdispatch.py::TestAOTAutograd::test_input_mutation_aliases_and_none_require_gradients
pull / linux-jammy-py3.10-clang15-asan / test (default, 1, 6, linux.4xlarge) (gh)
dynamo/test_minifier.py::MinifierTests::test_after_dynamo_non_leaf_compile_error
pull / linux-jammy-py3.10-clang15-asan / test (default, 2, 6, linux.4xlarge) (gh)
dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_donated_buffer1_dynamic_shapes
pull / linux-jammy-py3.10-clang15-asan / test (default, 3, 6, linux.4xlarge) (gh)
dynamo/test_higher_order_ops.py::HigherOrderOpTests::test_cond_branches_no_arguments
pull / linux-jammy-py3.10-clang15-asan / test (default, 4, 6, linux.4xlarge) (gh)
export/test_torchbind.py::TestCompileTorchbind::test_compile_obj_graph_breaks
pull / linux-jammy-py3.10-clang15-asan / test (default, 5, 6, linux.4xlarge) (gh)
dynamo/test_config.py::ConfigTests::test_no_automatic_dynamic
pull / linux-jammy-py3.10-clang15-asan / test (default, 6, 6, linux.4xlarge) (gh)
dynamo/test_backward_higher_order_ops.py::BackwardHigherOrderOpTests::test_invoke_in_pt2_compiled_autograd_side_effect
pull / linux-jammy-py3.9-gcc11 / test (default, 1, 4, linux.2xlarge) (gh)
functorch/test_control_flow.py::TestControlFlow::test_scan_simple_graph
pull / linux-jammy-py3.9-gcc11 / test (default, 2, 4, linux.2xlarge) (gh)
dynamo/test_config.py::ConfigTests::test_no_automatic_dynamic
pull / linux-jammy-py3.9-gcc11 / test (default, 3, 4, linux.2xlarge) (gh)
dynamo/test_functions.py::FunctionTests::test_partials_graph_break_reconstruct
pull / linux-jammy-py3.9-gcc11 / test (default, 4, 4, linux.2xlarge) (gh)
dynamo/test_backward_higher_order_ops.py::BackwardHigherOrderOpTests::test_invoke_in_pt2_compiled_autograd_side_effect
pull / linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, linux.2xlarge) (gh)
distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_noncontiguous_output

FLAKY - The following job failed but was likely due to flakiness present on trunk:

inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (inductor_torchbench_cpu_smoketest_perf, 1, 1, linux.24xl.spr-metal) (gh) (similar failure)
WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu]

This comment was automatically generated by Dr. CI and updates every 15 minutes.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames rec [ghstack-poisoned]

torch/_dynamo/variables/builder.py

ezyang

some ordering jiggering

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames rec [ghstack-poisoned]

ghstack-source-id: e53b77f Pull Request resolved: #137612

bobrenjc93 · 2024-10-09T18:49:09Z

benchmark suite run: https://github.com/pytorch/pytorch/actions/runs/11261191341

fixed

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames rec [ghstack-poisoned]

bobrenjc93 · 2024-10-10T21:34:57Z

I didn't rebase on top of an existing benchmark commit for the first run. Just kicked off second run. For posterity posting all of the steps I took here:

Main commit (2024/10/08): a02093e
My commit: a04e0e1

git checkout a02093e824707a38bc94303568fb9490f50bf28a
git cherry-pick a04e0e151cd3140ec06ec48a33ee650c222b5d6e
ghstack --draft

Kicked off https://github.com/pytorch/pytorch/actions/runs/11282346820

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames rec [ghstack-poisoned]

ghstack-source-id: fd7e6e2 Pull Request resolved: #137612

torch/_dynamo/variables/builder.py

torch/_dynamo/config.py

ezyang · 2024-10-11T15:14:59Z

I did a quick spot check on failures and it kind of looks like just brittle tests wobbling.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames rec [ghstack-poisoned]