Skip to content

2097 CoordConv transform#2162

Merged
wyli merged 11 commits intoProject-MONAI:devfrom
marksgraham:2097_CoordConv_transform
May 15, 2021
Merged

2097 CoordConv transform#2162
wyli merged 11 commits intoProject-MONAI:devfrom
marksgraham:2097_CoordConv_transform

Conversation

@marksgraham
Copy link
Copy Markdown
Contributor

@marksgraham marksgraham commented May 8, 2021

Fixes #2097 .

Description

Implements a transformation that appends spatial coordinates to the input as new channels, inspired by the CoordConv paper. Have found this improves the performance of my segmentation networks.

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage. I get a pytype error that I can't resolve - help appreciated
  • Quick tests passed locally by running ./runtests.sh --quick --unittests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@wyli
Copy link
Copy Markdown
Contributor

wyli commented May 12, 2021

/black
thanks, for the typing error, a simple workaround would be removing -> Union[np.ndarray, torch.Tensor] in the newly added classes

@marksgraham
Copy link
Copy Markdown
Contributor Author

Thanks for comments. I have:

  • Fixed the typing error (I think!)
  • Used @rijobro's cleaner code for generating the channels. I've decided to stick with the spatial_channels argument starting from 1 rather than 0 but can change if you feel stronlg they should start from 0. I looked for some precedent in other transforms but couldn't find any.
  • Finally, I've renamed the transform to AddCoordinateChannels, and provided a bit of motivation for when you might want to use it in the docs. It isn't correct to call it CoordConv because CoordConv changes the convolution layer itself to append spatial channels for each convolution, while my code only appends these channels to the input data as a transform. I think my version has particular use for medical imaging examples where e.g. you might train on patches of 3D volumes that are all aligned to a common space. Appending the coordinate frame to the patch might allow the network to understand where the patch is in the image - CoordConv only appends the local reference coordinates of the input feature map.

@rijobro
Copy link
Copy Markdown
Contributor

rijobro commented May 14, 2021

Thanks, looks great. Since the output is a concatenation of the channels and the original image, in the unit test it might be worth checking that the first part of the image matches the original image?

I suggested another update to the calculation of the coords (sorry for playing a bit of code golf!) if you want to add that too.

@marksgraham
Copy link
Copy Markdown
Contributor Author

Thanks - I've taken your suggestion for meshgrid (had to add indexing='ij' to get the first two dimensions right), and added that unit test.

@rijobro
Copy link
Copy Markdown
Contributor

rijobro commented May 14, 2021

Looks good to me, happy to merge once the checks are passing!

@wyli wyli enabled auto-merge (squash) May 15, 2021 07:29
Copy link
Copy Markdown
Contributor

@wyli wyli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wyli wyli merged commit 96cb3cb into Project-MONAI:dev May 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CoordConv transform

4 participants