Remove rpc fork and dist autograd fork tests from PyTorch repo #29827
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There are known issues for "fork tests + OMP" in Pytorch, rpc and dist autograd tests use OMP thread pools, this caused rpc fork and dist autograd fork tests to be flaky. So remove these fork tests from PyTorch repo. rpc spawn and dist autograd spawn tests are still running. Differential Revision: [D18507384](https://our.internmc.facebook.com/intern/diff/D18507384/) [ghstack-poisoned]
zhaojuanmao
added a commit
that referenced
this pull request
Nov 14, 2019
There are known issues for "fork tests + OMP" in Pytorch, rpc and dist autograd tests use OMP thread pools, this caused rpc fork and dist autograd fork tests to be flaky. So remove these fork tests from PyTorch repo. rpc spawn and dist autograd spawn tests are still running. Differential Revision: [D18507384](https://our.internmc.facebook.com/intern/diff/D18507384/) ghstack-source-id: 93919578 Pull Request resolved: #29827
xush6528
reviewed
Nov 14, 2019
| TESTS.extend([ | ||
| 'rpc_fork', | ||
| 'rpc_spawn', | ||
| 'dist_autograd_fork', |
Contributor
There was a problem hiding this comment.
Can you do the same to test_dist_optimizer_fork.py. It depends on RpcAgent as well.
mrshenli
reviewed
Nov 14, 2019
| if PY33: | ||
| TESTS.extend([ | ||
| 'rpc_fork', | ||
| 'rpc_spawn', |
Contributor
There was a problem hiding this comment.
since we only have spawn mode on the OSS side, do we still need to explicitly have a "spawn" in the file name?
Contributor
Author
There was a problem hiding this comment.
we can keep it for now? once we completely get rid of fork, we can merge rpcSpawnTest with RpcTest
…repo" There are known issues for "fork tests + OMP" in Pytorch, rpc and dist autograd tests use OMP thread pools, this caused rpc fork and dist autograd fork tests to be flaky. So remove these fork tests from PyTorch repo. rpc spawn and dist autograd spawn tests are still running. Differential Revision: [D18507384](https://our.internmc.facebook.com/intern/diff/D18507384/) [ghstack-poisoned]
zhaojuanmao
added a commit
that referenced
this pull request
Nov 15, 2019
Pull Request resolved: #29827 There are known issues for "fork tests + OMP" in Pytorch, rpc and dist autograd tests use OMP thread pools, this caused rpc fork and dist autograd fork tests to be flaky. So remove these fork tests from PyTorch repo. rpc spawn and dist autograd spawn tests are still running. Differential Revision: [D18507384](https://our.internmc.facebook.com/intern/diff/D18507384/) ghstack-source-id: 94040608
pritamdamania87
approved these changes
Nov 16, 2019
mrshenli
approved these changes
Nov 18, 2019
Contributor
mrshenli
left a comment
mrshenli
left a comment
There was a problem hiding this comment.
lint and test failures are irrelevant.
Curious, why dist_optimizer_spawn is not in windows and rocm blacklist?
Contributor
Author
…repo" There are known issues for "fork tests + OMP" in Pytorch, rpc and dist autograd tests use OMP thread pools, this caused rpc fork and dist autograd fork tests to be flaky. So remove these fork tests from PyTorch repo. rpc spawn and dist autograd spawn tests are still running. Differential Revision: [D18507384](https://our.internmc.facebook.com/intern/diff/D18507384/) [ghstack-poisoned]
…repo" There are known issues for "fork tests + OMP" in Pytorch, rpc and dist autograd tests use OMP thread pools, this caused rpc fork and dist autograd fork tests to be flaky. So remove these fork tests from PyTorch repo. rpc spawn and dist autograd spawn tests are still running. Differential Revision: [D18507384](https://our.internmc.facebook.com/intern/diff/D18507384/) [ghstack-poisoned]
zhaojuanmao
added a commit
that referenced
this pull request
Nov 18, 2019
Pull Request resolved: #29827 There are known issues for "fork tests + OMP" in Pytorch, rpc and dist autograd tests use OMP thread pools, this caused rpc fork and dist autograd fork tests to be flaky. So remove these fork tests from PyTorch repo. rpc spawn and dist autograd spawn tests are still running. Differential Revision: [D18507384](https://our.internmc.facebook.com/intern/diff/D18507384/) ghstack-source-id: 94139812
Contributor
Author
|
checked "rpc_fork", "dist_autograd_fork" and "dist_optimizer_fork" did not run but "rpc_spawn", "dist_autograd_spawn" and "dist_optimizer_spawn" ran in https://circleci.com/api/v1.1/project/github/pytorch/pytorch/3645637/output/105/0?file=true&allocation-id=5dd30d9f0d6a0b62d1d857f9-0-build%2F10D9680D |
Contributor
Author
|
rocm test failure is not relevant FAILED (skipped=12, unexpected successes=1) |
Contributor
|
This pull request has been merged in 861ef05. |
This was referenced Nov 19, 2019
mrshenli
added a commit
that referenced
this pull request
Nov 19, 2019
As after #29827 we only test RPC using spawn, the multi-thread/fork error should disappear. [ghstack-poisoned]
mrshenli
added a commit
that referenced
this pull request
Nov 19, 2019
As after #29827 we only test RPC using spawn, the multi-thread/fork error should disappear. [ghstack-poisoned]
mrshenli
added a commit
that referenced
this pull request
Nov 19, 2019
As after #29827 we only test RPC using spawn, the multi-thread/fork error should disappear. [ghstack-poisoned]
mrshenli
added a commit
that referenced
this pull request
Nov 21, 2019
With #29827, the flakiness should disappear for test_call_method_on_rref [ghstack-poisoned]
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
There are known issues for "fork tests + OMP" in Pytorch, rpc and dist autograd tests use OMP thread pools, this caused rpc fork and dist autograd fork tests to be flaky. So remove these fork tests from PyTorch repo. rpc spawn and dist autograd spawn tests are still running.
Differential Revision: D18507384