Revert "Enable test_distributed for ROCm but only with nccl backend" #29736
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 7073ee2.
Contributor
facebook-github-bot
left a comment
facebook-github-bot
left a comment
There was a problem hiding this comment.
@bddppq has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
iotamudelta
approved these changes
Nov 13, 2019
Contributor
iotamudelta
left a comment
iotamudelta
left a comment
There was a problem hiding this comment.
OK for me. @jithunnair-amd will try to reproduce locally.
Collaborator
|
Thanks for mentioning the failing builds @bddppq. I'll look into the flaky tests. |
Closed
Contributor
facebook-github-bot
pushed a commit
that referenced
this pull request
Feb 10, 2020
…32551) Summary: This is a redux of the original PR #28814 which was reverted in PR #29736 due to test_DistributedDataParallel being suspected as being flaky. Further investigation revealed it wasn't flakiness, but a bug in the PyTorch source code which has been now fixed in PR #32356. This PR is another attempt at enabling the test_distributed unit test suite only for the nccl backend. Pull Request resolved: #32551 Differential Revision: D19729966 Pulled By: bddppq fbshipit-source-id: 12a0d850991a903cc7723d63693b6157071d7115
ttumiel
pushed a commit
to ttumiel/pytorch
that referenced
this pull request
Mar 4, 2020
…ytorch#32551) Summary: This is a redux of the original PR pytorch#28814 which was reverted in PR pytorch#29736 due to test_DistributedDataParallel being suspected as being flaky. Further investigation revealed it wasn't flakiness, but a bug in the PyTorch source code which has been now fixed in PR pytorch#32356. This PR is another attempt at enabling the test_distributed unit test suite only for the nccl backend. Pull Request resolved: pytorch#32551 Differential Revision: D19729966 Pulled By: bddppq fbshipit-source-id: 12a0d850991a903cc7723d63693b6157071d7115
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This reverts commit 7073ee2.
They are flaky on master:
https://ci.pytorch.org/jenkins/job/pytorch-builds/job/py3.6-clang7-rocmdeb-ubuntu16.04-test2/6830//console
https://ci.pytorch.org/jenkins/job/pytorch-builds/job/py3.6-clang7-rocmdeb-ubuntu16.04-test2/6824//console
https://ci.pytorch.org/jenkins/job/pytorch-builds/job/py3.6-clang7-rocmdeb-ubuntu16.04-test2/6802//console
cc @jithunnair-amd