Skip to content

Inconsistency networks.layer.AffineTransform and transforms.Affine #4281

@Constantin-Jehn

Description

@Constantin-Jehn

Hello,

Describe the bug
Applying the same affine transformation using networks.layer.AffineTransform and transforms.Affine seems to lead to different result. The layer seems to introduce an unwanted shear.

To Reproduce
Steps to reproduce the behavior:

  1. Download 10_3T_nody_001.nii.gz.zip
  2. Run the following notebook
import monai
from monai.transforms import (
    AddChanneld,
    LoadImaged,
    ToTensord,
)
import torch as t
from copy import deepcopy

Load images and copy for comparison

path = "10_3T_nody_001.nii.gz"
add_channel = AddChanneld(keys=["image"])
loader = LoadImaged(keys=["image"])
to_tensor = ToTensord(keys=["image"])
monai_dict = {"image": path}
monai_image_layer = loader(monai_dict)
monai_image_layer = add_channel(monai_image_layer)
monai_image_layer = to_tensor(monai_image_layer)
monai_image_transform = deepcopy(monai_image_layer)

Set up example affine

translations = [0,0,0]
rotations = [t.pi/8, 0 ,0]

rotation_tensor = monai.transforms.utils.create_rotate(3, rotations,backend='torch')
translation_tensor = monai.transforms.utils.create_translate(3, translations,backend='torch')
affine =  (t.matmul(translation_tensor,rotation_tensor))

Define affine layer and transform

monai_aff_layer = monai.networks.layers.AffineTransform(mode = "bilinear",  normalized = True, align_corners= True, padding_mode = "zeros")
monai_aff_trans = monai.transforms.Affine(affine=affine, norm_coords = True, image_only = True)

Do transform using affine layer, add another dimension for batch

input_batched = monai_image_layer["image"].unsqueeze(0)

monai_image_layer["image"] = monai_aff_layer(input_batched ,t.tensor(affine).unsqueeze(0))

monai_image_layer["image"] = monai_image_layer["image"].squeeze().unsqueeze(0)

nifti_saver = monai.data.NiftiSaver(output_dir="tests", output_postfix = "layer",
                                            resample=False, padding_mode="zeros",
                                            separate_folder=False)
nifti_saver.save(monai_image_layer["image"],monai_image_layer["image_meta_dict"])
file written: tests/10_3T_nody_001_layer.nii.gz.


/var/folders/ll/btjq1lpn2msbmbn34rst5wz80000gn/T/ipykernel_64711/3194165265.py:3: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  monai_image_layer["image"] = monai_aff_layer(input_batched ,t.tensor(affine).unsqueeze(0))

Do transform using transform.Affine

monai_image_transform["image"] = monai_aff_trans(monai_image_transform["image"], padding_mode="zeros")

nifti_saver = monai.data.NiftiSaver(output_dir="tests", output_postfix = "trans",
                                            resample=False, padding_mode="zeros",
                                            separate_folder=False)
nifti_saver.save(monai_image_transform["image"], monai_image_transform["image_meta_dict"])
file written: tests/10_3T_nody_001_trans.nii.gz.

Expected behavior
Unambigious behaviour of the network and transforms affine transform.

Screenshots
original file:
original_file
using networks.layer:
networks_layer_Affine
using transforms.Affine
transforms_Affine

Environment

================================
Printing MONAI config...

MONAI version: 0.8.1+181.ga676e387
Numpy version: 1.20.3
Pytorch version: 1.10.2
MONAI flags: HAS_EXT = False, USE_COMPILED = False
MONAI rev id: a676e38
MONAI file: /Users/constantin/opt/anaconda3/envs/SVR_pytorch/lib/python3.9/site-packages/monai/init.py

Optional dependencies:
Pytorch Ignite version: NOT INSTALLED or UNKNOWN VERSION.
Nibabel version: 3.2.2
scikit-image version: 0.18.3
Pillow version: 8.4.0
Tensorboard version: 2.9.0
gdown version: NOT INSTALLED or UNKNOWN VERSION.
TorchVision version: 0.11.3
tqdm version: 4.62.3
lmdb version: NOT INSTALLED or UNKNOWN VERSION.
psutil version: 5.8.0
pandas version: 1.3.4
einops version: NOT INSTALLED or UNKNOWN VERSION.
transformers version: NOT INSTALLED or UNKNOWN VERSION.
mlflow version: NOT INSTALLED or UNKNOWN VERSION.

For details about installing the optional dependencies, please visit:
https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies

================================
Printing system config...

System: Darwin
Mac version: 10.16
Platform: macOS-10.16-x86_64-i386-64bit
Processor: i386
Machine: x86_64
Python version: 3.9.7
Process name: python3.9
Command: ['python', '-c', 'import monai; monai.config.print_debug_info()']
Open files: []
Num physical CPUs: 8
Num logical CPUs: 8
Num usable CPUs: UNKNOWN for given OS
CPU usage (%): [32.3, 32.3, 27.3, 25.2, 10.3, 57.9, 2.4, 1.6]
CPU freq. (MHz): 2400
Load avg. in last 1, 5, 15 mins (%): [23.4, 24.4, 22.5]
Disk usage (%): 54.7
Avg. sensor temp. (Celsius): UNKNOWN for given OS
Total physical memory (GB): 16.0
Available memory (GB): 0.7
Used memory (GB): 1.2

================================
Printing GPU config...

Num GPUs: 0
Has CUDA: False
cuDNN enabled: False

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions