masked_fill with FloatTensor mask will never mask but fails silently. #89320
Description
🐛 Describe the bug
Passing bool or int tensors as a mask input to masked_fill returns the excepted result, but a float tensor will not mask and returns input tensor silently.
import torch
x = torch.rand(4, 4)
m = torch.rand(4, 4) > 0.5
print(m, '\n')
print(x.masked_fill(m , 0.), '\n')
print(x.masked_fill(m.int() , 0.), '\n')
print(x.masked_fill(m.float() , 0.), '\n')
Output:
tensor([[False, False, True, True],
[ True, False, False, False],
[ True, True, False, False],
[ True, True, False, False]])
tensor([[0.8455, 0.7714, 0.0000, 0.0000],
[0.0000, 0.1767, 0.9352, 0.1114],
[0.0000, 0.0000, 0.7616, 0.2102],
[0.0000, 0.0000, 0.4796, 0.1690]])
tensor([[0.8455, 0.7714, 0.0000, 0.0000],
[0.0000, 0.1767, 0.9352, 0.1114],
[0.0000, 0.0000, 0.7616, 0.2102],
[0.0000, 0.0000, 0.4796, 0.1690]])
tensor([[0.8455, 0.7714, 0.7397, 0.3867],
[0.8923, 0.1767, 0.9352, 0.1114],
[0.9112, 0.1297, 0.7616, 0.2102],
[0.9136, 0.0941, 0.4796, 0.1690]])
I feel this behaviour is inconsistent with python's behaviour: bool(0.0) == False and bool(1.0) == True and should return the same as the other two examples. In any case, it should at least fail when a FloatTensor is passed as a mask.
Versions
$ python collect_env.py
Collecting environment information...
PyTorch version: 1.13.0
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A
OS: Arch Linux (x86_64)
GCC version: (GCC) 12.2.0
Clang version: 14.0.6
CMake version: version 3.24.3
Libc version: glibc-2.36
Python version: 3.10.8 (main, Nov 1 2022, 14:18:21) [GCC 12.2.0] (64-bit runtime)
Python platform: Linux-6.0.8-arch1-1-x86_64-with-glibc2.36
Is CUDA available: True
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce GTX 1080 Ti
Nvidia driver version: 520.56.06
cuDNN version: Probably one of the following:
/usr/lib/libcudnn.so.8.6.0
/usr/lib/libcudnn_adv_infer.so.8.6.0
/usr/lib/libcudnn_adv_train.so.8.6.0
/usr/lib/libcudnn_cnn_infer.so.8.6.0
/usr/lib/libcudnn_cnn_train.so.8.6.0
/usr/lib/libcudnn_ops_infer.so.8.6.0
/usr/lib/libcudnn_ops_train.so.8.6.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
Versions of relevant libraries:
[pip3] mypy==0.982
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.4
[pip3] rotary-embedding-torch==0.1.5
[pip3] torch==1.13.0
[pip3] torchtext==0.10.0a0+0d670e0
[conda] nomkl 3.0 0
I also reproduced this on a Ubuntu 20 internal node, but don't have access to the env right now.