💊 CI failures summary and remediations

As of commit 5ef2baf (more details on the Dr. CI page and at hud.pytorch.org/pr/55210):

• 2/2 failures possibly* introduced in this PR
  • 1/2 non-scanned failure(s)

1 failure not recognized by patterns:

Job	Step	Action
Windows CI (pytorch-win-vs2019-cuda10-cudnn7-py3) / test (2)	Install Visual Studio 2019 toolchain	🔁 rerun

ci.pytorch.org: 1 failed

• Failed: pr/pytorch-linux-bionic-rocm4.2-py3.6

Preview docs built from this PR

Since this PR is not related to parallelization feature, only single core perf is tested:
NB: LayerNorm CPU doesn't have BFloat16 support previously, the before perf refers to simple impl as following:

-  AT_DISPATCH_FLOATING_TYPES(X.scalar_type(), "LayerNormKernelImpl", [&]() {
+  AT_DISPATCH_FLOATING_TYPES_AND(kBFloat16, X.scalar_type(), "LayerNormKernelImpl", [&]() {

• performance update on avx512 machine: Xeon(R) Gold 6248 CPU @ 2.50GHz

before: LayerNorm: 32x128x1024: fp32: 2.806 ms; bf16: 9.901 ms
after:  LayerNorm: 32x128x1024: fp32: 2.813 ms; bf16: 2.306 ms

• performance update on avx2 machine: Xeon(R) CPU E5-2680 v3 @ 2.50GHz

before: LayerNorm: 32x128x1024: fp32: 5.286 ms; bf16: 15.186 ms
after:  LayerNorm: 32x128x1024: fp32: 5.258 ms; bf16: 3.469 ms

@VitalyFedyunin has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

rebased and clear test cases failures from test_ops.py

@VitalyFedyunin has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Please rebase

@VitalyFedyunin rebased, please check!

@VitalyFedyunin has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@VitalyFedyunin merged this pull request in 652d911.