Skip to content

Implement SlidingWindowHoVerNetInferer#5531

Merged
Nic-Ma merged 34 commits intoProject-MONAI:devfrom
bhashemian:update-slidingwindowinfere
Nov 18, 2022
Merged

Implement SlidingWindowHoVerNetInferer#5531
Nic-Ma merged 34 commits intoProject-MONAI:devfrom
bhashemian:update-slidingwindowinfere

Conversation

@bhashemian
Copy link
Copy Markdown
Member

@bhashemian bhashemian commented Nov 15, 2022

Fixes #5521

Description

This PR implement SlidingWindowHoVerNetInferer that with HoVerNet model, where the output size is different than the input but cannot be scaled and should be padded and then cropped.

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.

@myron myron self-requested a review November 15, 2022 23:24
Copy link
Copy Markdown
Collaborator

@myron myron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@drbeh thanks for the PR, but this adds a lot of extra code to cover some special edge case. I don't think it should be inside of SlidingWindowInferrer (which job should be just to slide over image, and return the result)

I suggest to either a) create a subclass/another-class for hovernet b) or add an options to SlidingWindowInferrer e.g. callback: callable = None. and if that function is provided, it'll be applied to every patch, then from hover net you can define that callback to do your padding.

@bhashemian
Copy link
Copy Markdown
Member Author

@drbeh thanks for the PR, but this adds a lot of extra code to cover some special edge case. I don't think it should be inside of SlidingWindowInferrer (which job should be just to slide over image, and return the result)

I suggest to either a) create a subclass/another-class for hovernet b) or add an options to SlidingWindowInferrer e.g. callback: callable = None. and if that function is provided, it'll be applied to every patch, then from hover net you can define that callback to do your padding.

@myron thanks for your comment. I see your point but the current SlidingWindowInference also handles the situation when the output is not the same as input but can be corrected by scaling. This one adds the correction by padding.

The weird design of this class (implementing everything in one single function and then calling it) does not allow any subclassing or modular use of it, so all the code should be copies for any extension, which will create a lot of duplicity and make the maintenance hard.

Regarding the use of handlers, as we want to use this in a bundle, it seems that they way that bundles are consumed by MONAI Label [and MOANI Deploy] does not like use of handlers and they will be ignored: https://github.com/Project-MONAI/MONAILabel/blob/809b8d20168ef5043490d55b53f284c15aeadedb/monailabel/tasks/infer/bundle.py

Signed-off-by: Behrooz <[email protected]>
@myron
Copy link
Copy Markdown
Collaborator

myron commented Nov 16, 2022

e same as input but can be corrected by scaling.

@drbeh I understand what you mean, thank you. that case, when the output is corrected by scaling is also in the wrong place IMHO, and should be removed. it seems everyone just keep adding special new options (to satisfy their edge case). I agree we should refactor this class SlidingWindowInference to simplify it and to allow subclasses to add functionality without copying code)

for your PR, if sub-classing/refactor is not possible, I recommend to use callback/handlers for your edge case. You can minimally modify the SlidingWindowInference to accept a callback function (as an optional input), then create another new small class SlidingForHovernet (could be inside of HoVerNet file actually) - which will call SlidingWindowInference with provided callback.

Then you can provide this SlidingForHovernet for inference for MONAI Label. Let me know if you think of a better way

@bhashemian
Copy link
Copy Markdown
Member Author

bhashemian commented Nov 16, 2022

@myron I totally agree with you that we should separate the functionalities and a refactoring is definitely needed.

I made some changes that I think it is aligned with that you said. I have created another class SlidingWindowHoVerNetInferer (in apps/pathology) which calls sliding_window_inferer (which handles window outputs) but takes care of some input paddings and cropping in the class. Thus SlidingWindowInferer is almost intact. What do you think?

@bhashemian bhashemian changed the title Enhance SlidingWindowInference Implement SlidingWindowHoVerNetInference Nov 16, 2022
@bhashemian bhashemian changed the title Implement SlidingWindowHoVerNetInference Implement SlidingWindowHoVerNetInferer Nov 16, 2022
@bhashemian bhashemian marked this pull request as ready for review November 16, 2022 16:09
@bhashemian bhashemian requested review from Nic-Ma and myron November 16, 2022 16:09
@bhashemian bhashemian requested a review from myron November 16, 2022 21:27
@Nic-Ma Nic-Ma requested review from KumoLiu and wyli November 17, 2022 10:05
Copy link
Copy Markdown
Contributor

@wyli wyli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, I put some comments inline, I haven't checked this PR as part of any inference pipeline.

@bhashemian bhashemian enabled auto-merge (squash) November 17, 2022 15:18
Signed-off-by: Behrooz <[email protected]>
Signed-off-by: Behrooz <[email protected]>
@bhashemian bhashemian requested review from Nic-Ma and wyli November 17, 2022 17:36
Copy link
Copy Markdown
Contributor

@wyli wyli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, this looks good to me, @KumoLiu please let me know if you are testing this otherwise I'll merge this before next week.

@bhashemian
Copy link
Copy Markdown
Member Author

@Nic-Ma Nic-Ma disabled auto-merge November 18, 2022 02:06
@Nic-Ma
Copy link
Copy Markdown
Contributor

Nic-Ma commented Nov 18, 2022

/build

@Nic-Ma Nic-Ma enabled auto-merge (squash) November 18, 2022 02:17
@Nic-Ma Nic-Ma merged commit e4b99e1 into Project-MONAI:dev Nov 18, 2022
bhashemian added a commit to bhashemian/MONAI that referenced this pull request Nov 23, 2022
Fixes Project-MONAI#5521 

### Description

This PR implement `SlidingWindowHoVerNetInferer` that with HoVerNet
model, where the output size is different than the input but cannot be
scaled and should be padded and then cropped.

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [x] New tests added to cover the changes.
- [x] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [x] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [x] In-line docstrings updated.

Signed-off-by: Behrooz <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Behrooz <[email protected]>
@bhashemian bhashemian deleted the update-slidingwindowinfere branch February 22, 2023 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HoVerNet ROI-based Inference

5 participants