Skip to content

Conversation

@Aidyn-A
Copy link
Collaborator

@Aidyn-A Aidyn-A commented Jun 10, 2025

Since its introduction ~4 years ago, the test test_sort_large has always been deselected because it requires 200GB of CUDA memory. Now, as we do have GPUs this big, it gets selected, but fails with var_mean not being a member if torch.Tensor and var_mean accepting only floating point tensors.

@pytorch-bot
Copy link

pytorch-bot bot commented Jun 10, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/155546

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 1 Pending, 2 Unrelated Failures

As of commit fea7174 with merge base 3863bbb (image):

FLAKY - The following job failed but was likely due to flakiness present on trunk:

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label Jun 10, 2025
@Aidyn-A Aidyn-A requested a review from malfet June 10, 2025 07:34
@zou3519 zou3519 added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Jun 10, 2025
Comment on lines +231 to +232
self.assertEqual(vm, torch.arange(8192, dtype=dtype, device=device))
self.assertEqual(im, t0.sort().indices, exact_dtype=False)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we this test in CI?

Copy link
Collaborator Author

@Aidyn-A Aidyn-A Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, CI machines are not that big, but I have tested it on GB300 which has 288GB of memory.

@Aidyn-A
Copy link
Collaborator Author

Aidyn-A commented Jun 10, 2025

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jun 10, 2025
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

jithunnair-amd pushed a commit to ROCm/pytorch that referenced this pull request Aug 25, 2025
Currently std::min -> ::min did not work as expected on ROCm when input
values >= 2147483648

Replace std::min to ternary statement
Also std::min can be replaced by explicit typing std::min<int64_t>

fixes on ROCm:

test_sort_and_select.py::TestSortAndSelectCUDA::test_sort_large_cuda_float16
error:
RuntimeError: Cannot sort dimension of length 8192

Combines upstream PRs:
- pytorch#161054 to fix std::min on ROCm
- pytorch#155546 fix python test
- pytorch#159939 change test dtype from
int8 to float16

Fixes: SWDEV-526432
dhonnappa-amd pushed a commit to ROCm/pytorch that referenced this pull request Aug 25, 2025
Currently std::min -> ::min did not work as expected on ROCm when input
values >= 2147483648

Replace std::min to ternary statement
Also std::min can be replaced by explicit typing std::min<int64_t>

fixes on ROCm:

test_sort_and_select.py::TestSortAndSelectCUDA::test_sort_large_cuda_float16
error:
RuntimeError: Cannot sort dimension of length 8192

Combines upstream PRs:
- pytorch#161054 to fix std::min on ROCm
- pytorch#155546 fix python test
- pytorch#159939 change test dtype from
int8 to float16

Fixes: SWDEV-526432
dhonnappa-amd pushed a commit to ROCm/pytorch that referenced this pull request Aug 25, 2025
Currently std::min -> ::min did not work as expected on ROCm when input
values >= 2147483648

Replace std::min to ternary statement
Also std::min can be replaced by explicit typing std::min<int64_t>

fixes on ROCm:

test_sort_and_select.py::TestSortAndSelectCUDA::test_sort_large_cuda_float16
error:
RuntimeError: Cannot sort dimension of length 8192

Combines upstream PRs:
- pytorch#161054 to fix std::min on ROCm
- pytorch#155546 fix python test
- pytorch#159939 change test dtype from
int8 to float16

Fixes: SWDEV-526432
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk Trigger trunk jobs on your pull request Merged open source topic: not user facing topic category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants