RuntimeError: quantile() input tensor is too large #4854
Description
Discussed in #4851
Originally posted by AAttarpour August 7, 2022
...
I use ScaleIntensityRangePercentilesd transform. In the new MONAI, it gives me this error from torch when it wants to use this transform:
RuntimeError: quantile() input tensor is too large I firstly thought sth went wrong with my virtualenv when I uninstalled and installed MONAI again, but I tested in google colab and it gave me the same error. Here is how I tested it:
!pip install -q monai-weekly
from monai.transforms import ScaleIntensityRangePercentilesd
from monai.networks.nets import DynUnet
import numpy as np
from monai.config import print_config
print_config()
MONAI version: 0.10.dev2232
Numpy version: 1.21.6
Pytorch version: 1.12.0+cu113
MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
MONAI rev id: bedd7dde553c107e5b8f633b830e3e7e0a70b5e3
MONAI __file__: /usr/local/lib/python3.7/dist-packages/monai/__init__.py
Optional dependencies:
Pytorch Ignite version: NOT INSTALLED or UNKNOWN VERSION.
Nibabel version: 3.0.2
scikit-image version: 0.18.3
Pillow version: 7.1.2
Tensorboard version: 2.8.0
gdown version: 4.4.0
TorchVision version: 0.13.0+cu113
tqdm version: 4.64.0
lmdb version: 0.99
psutil version: 5.4.8
pandas version: 1.3.5
einops version: NOT INSTALLED or UNKNOWN VERSION.
transformers version: NOT INSTALLED or UNKNOWN VERSION.
mlflow version: NOT INSTALLED or UNKNOWN VERSION.
pynrrd version: NOT INSTALLED or UNKNOWN VERSION.
img = np.ones((512,512,512), dtype='float32')
print(isinstance(img, np.ndarray))
img[100:200, 100:200, 100:200] = 10
seg = np.ones((512,512,512), dtype='float32')
transform = ScaleIntensityRangePercentilesd(keys=["image"], lower=0.05,
upper=99.95, b_min=0, b_max=1,
clip=True, relative=False)
img_dict = {'image': img, 'label': seg}
img_dict_transformed = transform(img_dict)
RuntimeError Traceback (most recent call last)
[<ipython-input-4-e0f9f6c40a55>](https://localhost:8080/#) in <module>()
7 clip=True, relative=False)
8 img_dict = {'image': img, 'label': seg}
----> 9 img_dict_transformed = transform(img_dict)
3 frames
[/usr/local/lib/python3.7/dist-packages/monai/transforms/utils_pytorch_numpy_unification.py](https://localhost:8080/#) in percentile(x, q, dim, keepdim, **kwargs)
109 else:
110 q = convert_to_dst_type(q / 100.0, x)[0]
--> 111 result = torch.quantile(x, q, dim=dim, keepdim=keepdim)
112 return result
113
RuntimeError: quantile() input tensor is too large
This isn't a problem with the previous MONAI; in the google colab, I installed monai-weekly==0.9.dev2214 again and there was no error.
(edits: previous versions with no error probably because the percentile was done with the numpy backend, related issue for torch.quantile pytorch/pytorch#64947)