Skip to content

Conversation

@zou3519
Copy link
Contributor

@zou3519 zou3519 commented Sep 6, 2018

Not a lot changed

@zou3519 zou3519 added 0.3 oncall: jit Add this issue/PR to JIT oncall triage queue and removed 0.3 labels Sep 6, 2018
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

zou3519 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Copy link
Contributor

@zdevito zdevito left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have we actually checked that the memory test is working? e.g. introduced a leak and watch a test fail?

@zou3519
Copy link
Contributor Author

zou3519 commented Sep 6, 2018

@zdevito yes, I checked that back when I was fixing the memory leak in #10222 but I'll check again.

edit: verified by leaking a tensor in c++ and running the tests. There is also a general pytorch unit test for the cuda memory checker somewhere

@ssnl
Copy link
Collaborator

ssnl commented Sep 6, 2018

The checker test is at

pytorch/test/test_cuda.py

Lines 1930 to 1952 in 0ef2b31

def test_cuda_memory_leak_detection(self):
l = []
@self.wrap_with_cuda_memory_check
def no_leak():
pass
@self.wrap_with_cuda_memory_check
def leak_gpu0():
l.append(torch.tensor(10, device=torch.device("cuda:0")))
no_leak()
with self.assertRaisesRegex(AssertionError, r"leaked \d+ bytes CUDA memory on device 0"):
leak_gpu0()
if TEST_MULTIGPU:
@self.wrap_with_cuda_memory_check
def leak_gpu1():
l.append(torch.tensor(10, device=torch.device("cuda:1")))
with self.assertRaisesRegex(AssertionError, r"leaked \d+ bytes CUDA memory on device 1"):
leak_gpu1()

@zou3519 zou3519 deleted the jit-cuda-tests branch September 6, 2018 23:08
@zou3519 zou3519 restored the jit-cuda-tests branch September 6, 2018 23:08
PenghuiCheng pushed a commit to PenghuiCheng/pytorch that referenced this pull request Sep 11, 2018
Summary:
Not a lot changed
Pull Request resolved: pytorch#11332

Differential Revision: D9683680

Pulled By: zou3519

fbshipit-source-id: 95f444e54049dd268fc10effe425ef2df79c6467
@zou3519 zou3519 deleted the jit-cuda-tests branch September 13, 2018 15:35
@ezyang ezyang added the merged label Jun 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

oncall: jit Add this issue/PR to JIT oncall triage queue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants