🐛 Describe the bug

This issue tracks the failures in test_modules.py for the MPS backend.

• BatchNorm1d/BatchNorm2d: fails due to unsupported float64 passed to MPS's batch_norm implementation with error:
  TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.

• These tests fail due to functional issues (mismatching elements):

1. SELU
2. Softplus
3. PReLU
4. ReLU6
5. Threshold

I'll update this issue whenever each of the above failures get fixed.
These issues were discovered in the PR #95334

Versions

PyTorch version: 2.1.0a0+gite975f83
Is debug build: True

CMake version: version 3.26.3

Python version: 3.9.16 (main, Mar 8 2023, 04:29:24) [Clang 14.0.6 ] (64-bit runtime)

CPU:
Apple processor

cc @bdhirsh @kulinseth @albanD @malfet @DenisVieriu97 @abhudev