Skip to content

Conversation

@clee2000
Copy link
Contributor

@clee2000 clee2000 commented Jun 24, 2025

test_torch.py::TestTorchDeviceTypeCUDA::test_storage_use_count_cuda:
Added in #155451
Fails in debug mode GH job link HUD commit link

inductor/test_inductor_freezing.py::FreezingGpuTests::test_cpp_wrapper_cuda:
GH job link HUD commit link
started failing after moving to new cuda version #155234

I'll ping people if this gets merged

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov

@pytorch-bot
Copy link

pytorch-bot bot commented Jun 24, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/156731

Note: Links to docs will display an error until the docs builds have been completed.

❌ 10 New Failures, 1 Pending

As of commit ba0f580 with merge base 5ad2bee (image):

NEW FAILURES - The following jobs have failed:

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label Jun 24, 2025
@clee2000 clee2000 added the ciflow/periodic Trigger jobs ran periodically on master (periodic.yml) on the PR label Jun 24, 2025
@clee2000 clee2000 marked this pull request as ready for review June 24, 2025 20:45
@clee2000 clee2000 requested a review from a team June 24, 2025 21:38
@clee2000
Copy link
Contributor Author

@pytorchbot merge -f "s390x failures not related"

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@atalman
Copy link
Contributor

atalman commented Jul 3, 2025

@pytorchbot cherry-pick --onto release/2.8 -c release

@pytorchbot
Copy link
Collaborator

Cherry picking #156731

The cherry pick PR is at #157560 The following tracker issues are updated:

Details for Dev Infra team Raised by workflow job

@guangyey guangyey mentioned this pull request Jul 7, 2025
pytorchmergebot pushed a commit that referenced this pull request Jul 8, 2025
# Motivation
#155451 decoupled `torch._C._storage_Use_Count` from CUDA and introduced a corresponding unit test:
https://github.com/pytorch/pytorch/blob/815545f2dd6ade563cb1263f8bb7813f355edb2e/test/test_torch.py#L257-L262
However, this test fails when PyTorch is built with debug assertions enabled. @clee2000 disabled this UT in #156731. The root cause is that `_cdata` is obtained from an `intrusive_ptr`, not a `weak_intrusive_ptr`. As a result, calling `c10::weak_intrusive_ptr::use_count` on it triggers the internal assertion:
https://github.com/pytorch/pytorch/blob/815545f2dd6ade563cb1263f8bb7813f355edb2e/c10/util/intrusive_ptr.h#L912-L917
For example:
```python
a = torch.randn(10, device=device) # refcount=1, weakcount=1
prev_cf = torch._C._storage_Use_Count(a.untyped_storage()._cdata) # violate the assertation
```
This violates the expected invariant inside `weak_intrusive_ptr::use_count`, which assumes the pointer was originally constructed from a valid `weak_intrusive_ptr`. Actually, `storage_impl` is obtained from an `intrusive_ptr`.
https://github.com/pytorch/pytorch/blob/815545f2dd6ade563cb1263f8bb7813f355edb2e/torch/csrc/Module.cpp#L2105-L2109

# Solution
Use `c10::intrusive_ptr::use_count` instead.

Pull Request resolved: #157694
Approved by: https://github.com/albanD
@github-actions github-actions bot deleted the csl/disable_periodic_tests branch August 3, 2025 02:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/inductor ciflow/periodic Trigger jobs ran periodically on master (periodic.yml) on the PR Merged module: inductor topic: not user facing topic category

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants