Skip to content

add opaque unary sin and cos to SYMPY_INTERP #139569

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
bobrenjc93 wants to merge 5 commits into from
Closed

add opaque unary sin and cos to SYMPY_INTERP #139569

bobrenjc93 wants to merge 5 commits into from

Conversation

@bobrenjc93
Copy link
Contributor

@bobrenjc93 bobrenjc93 commented Nov 3, 2024 •
edited
Loading

Stack from ghstack (oldest at bottom):

Fixes PYTORCH_TEST_WITH_DYNAMO=1 python test/test_nn.py TestNNDeviceTypeCPU.test_affine_3d_rotateRandom_cpu when specialize_float = False

cc @ezyang @SherlockNoMad @EikanWang @jgong5 @wenzhe-nrv

@pytorch-bot
Copy link

pytorch-bot bot commented Nov 3, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/139569

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 9b26acb with merge base e6ff07f (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@bobrenjc93 bobrenjc93 mentioned this pull request Nov 3, 2024
Closed
@pytorch-bot pytorch-bot bot added ciflow/inductor release notes: fx release notes category labels Nov 3, 2024
This was referenced Nov 3, 2024
Closed
Closed
Closed
Closed
This was referenced Nov 3, 2024
Closed
Closed
Closed
Closed
Closed
@bobrenjc93 bobrenjc93 added the topic: not user facing topic category label Nov 3, 2024
@bobrenjc93 bobrenjc93 requested a review from ezyang November 3, 2024 16:30
@bobrenjc93 bobrenjc93 marked this pull request as ready for review November 3, 2024 16:30
@bobrenjc93 bobrenjc93 mentioned this pull request Nov 3, 2024
Closed
@bobrenjc93
Update on "add opaque unary sin and cos to SYMPY_INTERP"
5f86b20 


Fixes `PYTORCH_TEST_WITH_DYNAMO=1 python test/test_nn.py TestNNDeviceTypeCPU.test_affine_3d_rotateRandom_cpu` when specialize_float = False

cc ezyang SherlockNoMad EikanWang jgong5 wenzhe-nrv

[ghstack-poisoned]
ezyang
ezyang approved these changes Nov 3, 2024
View reviewed changes
Copy link
Contributor

@ezyang ezyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might as well add the rest of them while you are at it?

@bobrenjc93 bobrenjc93 mentioned this pull request Nov 4, 2024
Closed
@bobrenjc93
Update on "add opaque unary sin and cos to SYMPY_INTERP"
6a5126d 


Fixes `PYTORCH_TEST_WITH_DYNAMO=1 python test/test_nn.py TestNNDeviceTypeCPU.test_affine_3d_rotateRandom_cpu` when specialize_float = False

cc ezyang SherlockNoMad EikanWang jgong5 wenzhe-nrv

[ghstack-poisoned]
@bobrenjc93
Copy link
Contributor Author

bobrenjc93 commented Nov 4, 2024

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Nov 4, 2024
@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 4, 2024

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Nov 4, 2024

Merge failed

Reason: 1 mandatory check(s) failed. The first few are:

Dig deeper by viewing the failures on hud

Details for Dev Infra team Raised by workflow job

Failing merge rule: Core Maintainers

@bobrenjc93
Update on "add opaque unary sin and cos to SYMPY_INTERP"
3fd470d 


Fixes `PYTORCH_TEST_WITH_DYNAMO=1 python test/test_nn.py TestNNDeviceTypeCPU.test_affine_3d_rotateRandom_cpu` when specialize_float = False

cc ezyang SherlockNoMad EikanWang jgong5 wenzhe-nrv

[ghstack-poisoned]
pytorch-bot bot pushed a commit that referenced this pull request Nov 11, 2024
@bobrenjc93 @atalman
add special case for __round__ constant variables (#139583)
d3ea69c 
Fixes `PYTORCH_TEST_WITH_INDUCTOR=1 tlp python test/test_torch.py TestTorchDeviceTypeCUDA.test_cauchy_cuda_float64` when specialize_float=False

Pull Request resolved: #139583
Approved by: https://github.com/ezyang
ghstack dependencies: #139569, #139457, #139568, #139572, #139846, #139454, #139896, #139935, #139587
zero000064 pushed a commit to zero000064/pytorch that referenced this pull request Nov 14, 2024
@bobrenjc93 @zero000064
refuse to generate a symbolic variable if a float input is inf (pytor…
abf827d 
…ch#139846)

Fixes `PYTORCH_TEST_WITH_INDUCTOR=1 tlp python test/test_torch.py TestTorchDeviceTypeCPU.test_cauchy_cpu_float64` when `specialize_float=False`

Pull Request resolved: pytorch#139846
Approved by: https://github.com/ruidazeng, https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572
zero000064 pushed a commit to zero000064/pytorch that referenced this pull request Nov 14, 2024
@bobrenjc93 @zero000064
specialize symfloats for wrapped_gradient in get_fake_value (pytorch#…
6a371a0 
…139935)

Fixes `PYTORCH_TEST_WITH_DYNAMO=1 python test/test_torch.py TestTorchDeviceTypeCPU.test_gradient_type_promotion_cpu` when `specialize_float=False`

Reviewers might wonder why we need to have this whitelist. Can't we rely on python_arg_parser.h to do the specialization generically? Alas this path doesn't actually FFI to C++ so we do need to do the specialization in pythonland.

Pull Request resolved: pytorch#139935
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846, pytorch#139454, pytorch#139896
zero000064 pushed a commit to zero000064/pytorch that referenced this pull request Nov 14, 2024
@bobrenjc93 @zero000064
Fix another item memo loss location + bool specialization bug (pytorc…
51491dc 
…h#139587)

This fix was a bit more involved:
1) It fixes a item_memo loss place.
2) It updates a test to be eager instead of aot_eager since it reveals a very obscure bug related to replacements that's not worth solving since in practice inductor will regenerate the runtime asserts anyways
3) It updates tensorify to specialize more places now that the aforementioned bug is fixed.

Fixes `PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=6 python test/inductor/test_torchinductor_opinfo.py TestInductorOpInfoCPU.test_comprehensive_linalg_norm_cpu_float16` when `specialize_float=False`

while ensuring `python test/dynamo/test_dynamic_shapes.py DynamicShapesMiscTests.test_runtime_assert_replacement_dynamic_shapes` doesn't regress

Pull Request resolved: pytorch#139587
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846, pytorch#139454, pytorch#139896, pytorch#139935
zero000064 pushed a commit to zero000064/pytorch that referenced this pull request Nov 14, 2024
@bobrenjc93 @zero000064
add special case for __round__ constant variables (pytorch#139583)
54376d4 
Fixes `PYTORCH_TEST_WITH_INDUCTOR=1 tlp python test/test_torch.py TestTorchDeviceTypeCUDA.test_cauchy_cuda_float64` when specialize_float=False

Pull Request resolved: pytorch#139583
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846, pytorch#139454, pytorch#139896, pytorch#139935, pytorch#139587
Ryo-not-rio pushed a commit to Ryo-not-rio/pytorch that referenced this pull request Dec 2, 2024
@bobrenjc93 @Ryo-not-rio
don't run z3 analysis on backed symfloat nodes (pytorch#139568)
ea666d9 
Pull Request resolved: pytorch#139568
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457
Ryo-not-rio pushed a commit to Ryo-not-rio/pytorch that referenced this pull request Dec 2, 2024
@bobrenjc93 @Ryo-not-rio
Specialize symfloats that flow through is_integer (pytorch#139572)
643e28d 
Fixes `python test/dynamo/test_dynamic_shapes.py DynamicShapesFunctionTests.test_number_method_method_is_integer_num_type6_dynamic_shapes` when specialize_float = False

Pull Request resolved: pytorch#139572
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568
Ryo-not-rio pushed a commit to Ryo-not-rio/pytorch that referenced this pull request Dec 2, 2024
@bobrenjc93 @Ryo-not-rio
Don't wrap inf values as symfloat (pytorch#139896)
e0c2f53 
Fixes `PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 python test/inductor/test_torchinductor_opinfo.py TestInductorOpInfoCPU.test_comprehensive_linalg_norm_cpu_float16` when `specialize_float=False`

Pull Request resolved: pytorch#139896
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846, pytorch#139454
Ryo-not-rio pushed a commit to Ryo-not-rio/pytorch that referenced this pull request Dec 2, 2024
@bobrenjc93 @Ryo-not-rio
Fix another item memo loss location + bool specialization bug (pytorc…
cbdf326 
…h#139587)

This fix was a bit more involved:
1) It fixes a item_memo loss place.
2) It updates a test to be eager instead of aot_eager since it reveals a very obscure bug related to replacements that's not worth solving since in practice inductor will regenerate the runtime asserts anyways
3) It updates tensorify to specialize more places now that the aforementioned bug is fixed.

Fixes `PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=6 python test/inductor/test_torchinductor_opinfo.py TestInductorOpInfoCPU.test_comprehensive_linalg_norm_cpu_float16` when `specialize_float=False`

while ensuring `python test/dynamo/test_dynamic_shapes.py DynamicShapesMiscTests.test_runtime_assert_replacement_dynamic_shapes` doesn't regress

Pull Request resolved: pytorch#139587
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846, pytorch#139454, pytorch#139896, pytorch#139935
@github-actions github-actions bot deleted the gh/bobrenjc93/106/head branch December 5, 2024 02:16
pobin6 pushed a commit to pobin6/pytorch that referenced this pull request Dec 5, 2024
@bobrenjc93
refuse to generate a symbolic variable if a float input is inf (pytor…
ffa65c0 
…ch#139846)

Fixes `PYTORCH_TEST_WITH_INDUCTOR=1 tlp python test/test_torch.py TestTorchDeviceTypeCPU.test_cauchy_cpu_float64` when `specialize_float=False`

Pull Request resolved: pytorch#139846
Approved by: https://github.com/ruidazeng, https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572
pobin6 pushed a commit to pobin6/pytorch that referenced this pull request Dec 5, 2024
@bobrenjc93
Trigger symfloat specialization in argument binding code (pytorch#139454
1048c78 
)

Fixes the test `python test/inductor/test_torchinductor.py CpuTests.test_upsample_cat_conv_cpu` when `specialize_float=False`

Pull Request resolved: pytorch#139454
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846
pobin6 pushed a commit to pobin6/pytorch that referenced this pull request Dec 5, 2024
@bobrenjc93
Don't wrap inf values as symfloat (pytorch#139896)
a09c905 
Fixes `PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 python test/inductor/test_torchinductor_opinfo.py TestInductorOpInfoCPU.test_comprehensive_linalg_norm_cpu_float16` when `specialize_float=False`

Pull Request resolved: pytorch#139896
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846, pytorch#139454
pobin6 pushed a commit to pobin6/pytorch that referenced this pull request Dec 5, 2024
@bobrenjc93
specialize symfloats for wrapped_gradient in get_fake_value (pytorch#…
b5d04fc 
…139935)

Fixes `PYTORCH_TEST_WITH_DYNAMO=1 python test/test_torch.py TestTorchDeviceTypeCPU.test_gradient_type_promotion_cpu` when `specialize_float=False`

Reviewers might wonder why we need to have this whitelist. Can't we rely on python_arg_parser.h to do the specialization generically? Alas this path doesn't actually FFI to C++ so we do need to do the specialization in pythonland.

Pull Request resolved: pytorch#139935
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846, pytorch#139454, pytorch#139896
pobin6 pushed a commit to pobin6/pytorch that referenced this pull request Dec 5, 2024
@bobrenjc93
Fix another item memo loss location + bool specialization bug (pytorc…
14d09fb 
…h#139587)

This fix was a bit more involved:
1) It fixes a item_memo loss place.
2) It updates a test to be eager instead of aot_eager since it reveals a very obscure bug related to replacements that's not worth solving since in practice inductor will regenerate the runtime asserts anyways
3) It updates tensorify to specialize more places now that the aforementioned bug is fixed.

Fixes `PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=6 python test/inductor/test_torchinductor_opinfo.py TestInductorOpInfoCPU.test_comprehensive_linalg_norm_cpu_float16` when `specialize_float=False`

while ensuring `python test/dynamo/test_dynamic_shapes.py DynamicShapesMiscTests.test_runtime_assert_replacement_dynamic_shapes` doesn't regress

Pull Request resolved: pytorch#139587
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846, pytorch#139454, pytorch#139896, pytorch#139935
pobin6 pushed a commit to pobin6/pytorch that referenced this pull request Dec 5, 2024
@bobrenjc93
add special case for __round__ constant variables (pytorch#139583)
c107733 
Fixes `PYTORCH_TEST_WITH_INDUCTOR=1 tlp python test/test_torch.py TestTorchDeviceTypeCUDA.test_cauchy_cuda_float64` when specialize_float=False

Pull Request resolved: pytorch#139583
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846, pytorch#139454, pytorch#139896, pytorch#139935, pytorch#139587
fmo-mt pushed a commit to fmo-mt/pytorch that referenced this pull request Dec 11, 2024
@bobrenjc93 @fmo-mt
Trigger symfloat specialization in argument binding code (pytorch#139454
44c9fe6 
)

Fixes the test `python test/inductor/test_torchinductor.py CpuTests.test_upsample_cat_conv_cpu` when `specialize_float=False`

Pull Request resolved: pytorch#139454
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846
fmo-mt pushed a commit to fmo-mt/pytorch that referenced this pull request Dec 11, 2024
@bobrenjc93 @fmo-mt
Don't wrap inf values as symfloat (pytorch#139896)
ee95ee5 
Fixes `PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 python test/inductor/test_torchinductor_opinfo.py TestInductorOpInfoCPU.test_comprehensive_linalg_norm_cpu_float16` when `specialize_float=False`

Pull Request resolved: pytorch#139896
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846, pytorch#139454
fmo-mt pushed a commit to fmo-mt/pytorch that referenced this pull request Dec 11, 2024
@bobrenjc93 @fmo-mt
specialize symfloats for wrapped_gradient in get_fake_value (pytorch#…
f20e53e 
…139935)

Fixes `PYTORCH_TEST_WITH_DYNAMO=1 python test/test_torch.py TestTorchDeviceTypeCPU.test_gradient_type_promotion_cpu` when `specialize_float=False`

Reviewers might wonder why we need to have this whitelist. Can't we rely on python_arg_parser.h to do the specialization generically? Alas this path doesn't actually FFI to C++ so we do need to do the specialization in pythonland.

Pull Request resolved: pytorch#139935
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846, pytorch#139454, pytorch#139896
fmo-mt pushed a commit to fmo-mt/pytorch that referenced this pull request Dec 11, 2024
@bobrenjc93 @fmo-mt
Fix another item memo loss location + bool specialization bug (pytorc…
78fed4b 
…h#139587)

This fix was a bit more involved:
1) It fixes a item_memo loss place.
2) It updates a test to be eager instead of aot_eager since it reveals a very obscure bug related to replacements that's not worth solving since in practice inductor will regenerate the runtime asserts anyways
3) It updates tensorify to specialize more places now that the aforementioned bug is fixed.

Fixes `PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=6 python test/inductor/test_torchinductor_opinfo.py TestInductorOpInfoCPU.test_comprehensive_linalg_norm_cpu_float16` when `specialize_float=False`

while ensuring `python test/dynamo/test_dynamic_shapes.py DynamicShapesMiscTests.test_runtime_assert_replacement_dynamic_shapes` doesn't regress

Pull Request resolved: pytorch#139587
Approved by: https://github.com/ezyang
ghstack dependencies: pytorch#139569, pytorch#139457, pytorch#139568, pytorch#139572, pytorch#139846, pytorch#139454, pytorch#139896, pytorch#139935
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ezyang ezyang ezyang approved these changes

Assignees

No one assigned

Labels

ciflow/inductor ciflow/trunk Trigger trunk jobs on your pull request fx Merged release notes: fx release notes category topic: not user facing topic category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@bobrenjc93 @pytorchmergebot @ezyang @facebook-github-bot