In @fbhuba PR #21474 we switch the stream before running tests. But in this PR, a lot of tests failed and we had to disable the stream switching behavior for large portions of the test suite. There is probably something very important in PyTorch that is not stream safe, we need to figure it out.

Currently known reasons why you might be on wrong stream:

• Calling thrust functions (e.g., thrust::tabulate) without specifying a policy that puts things on the right stream
• Backwards runs on default stream