Skip to content

Sliding Patch WSI Dataset#4239

Merged
wyli merged 49 commits intoProject-MONAI:devfrom
bhashemian:sliding-patch
May 26, 2022
Merged

Sliding Patch WSI Dataset#4239
wyli merged 49 commits intoProject-MONAI:devfrom
bhashemian:sliding-patch

Conversation

@bhashemian
Copy link
Copy Markdown
Member

@bhashemian bhashemian commented May 6, 2022 •
edited
Loading

Fixes #4179

Description

This PR implements SlidingPatchWSIDataset that extract patches from one or multiple whole slide images in a sliding-window manner. This is useful for the inference of the pathology pipelines.

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@bhashemian bhashemian requested review from Nic-Ma and wyli May 6, 2022 19:51
@bhashemian
Remove unused import
577b097 
Signed-off-by: Behrooz <[email protected]>
wyli
wyli reviewed May 9, 2022
View reviewed changes
@bhashemian bhashemian requested a review from wyli May 9, 2022 18:47
@bhashemian
Copy link
Copy Markdown
Member Author

bhashemian commented May 10, 2022

Hi @Nic-Ma can you too help to review this PR? Thanks

Nic-Ma
Nic-Ma reviewed May 10, 2022
View reviewed changes
bhashemian added 4 commits May 16, 2022 16:45
@bhashemian
Merge dev
09a5e1a 
Signed-off-by: Behrooz <[email protected]>
@bhashemian
Update docstring
f206ecd 
Signed-off-by: Behrooz <[email protected]>
@bhashemian
Implement iter_wsi_patch_location
cbd477a 
Signed-off-by: Behrooz <[email protected]>
@bhashemian
Update docstring
aba82b3 
Signed-off-by: Behrooz <[email protected]>
wyli
wyli reviewed May 17, 2022
View reviewed changes
Copy link
Copy Markdown
Contributor

@wyli wyli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, please see some comments inline

ericspod
ericspod reviewed May 24, 2022
View reviewed changes
wyli
wyli reviewed May 25, 2022
View reviewed changes
wyli
wyli reviewed May 25, 2022
View reviewed changes
wyli
wyli approved these changes May 25, 2022
View reviewed changes
Copy link
Copy Markdown
Contributor

@wyli wyli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, this enhances the sliding window functions and adds wsi sliding window support.

@wyli
Copy link
Copy Markdown
Contributor

wyli commented May 25, 2022

/build

@wyli
Copy link
Copy Markdown
Contributor

wyli commented May 25, 2022 •
edited
Loading

there's a test error...

[2022-05-25T13:08:02.322Z] ======================================================================
[2022-05-25T13:08:02.322Z] FAIL: test_read_patches_9 (tests.test_sliding_patch_wsi_dataset.TestSlidingPatchWSIDatasetCuCIM)
[2022-05-25T13:08:02.322Z] ----------------------------------------------------------------------
[2022-05-25T13:08:02.322Z] Traceback (most recent call last):
[2022-05-25T13:08:02.322Z]   File "/opt/conda/lib/python3.8/site-packages/parameterized/parameterized.py", line 533, in standalone_func
[2022-05-25T13:08:02.322Z]     return func(*(a + p.args), **p.kwargs)
[2022-05-25T13:08:02.322Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_sliding_patch_wsi_dataset.py", line 227, in test_read_patches
[2022-05-25T13:08:02.322Z]     self.assertEqual(len(dataset), len(expected))
[2022-05-25T13:08:02.322Z] AssertionError: 1 != 2
[2022-05-25T13:08:02.322Z] 
[2022-05-25T13:08:02.322Z] ----------------------------------------------------------------------

with nvcr.io/nvidia/pytorch:22.04-py3 V100, please let me know if you need more info to debug

I think the class could inherit Randomizable to ensure the deterministic behaviour e.g.

MONAI/monai/apps/datasets.py

Line 33 in eb60b9c

class MedNISTDataset(Randomizable, CacheDataset):

@bhashemian
Copy link
Copy Markdown
Member Author

bhashemian commented May 25, 2022

there's a test error...

[2022-05-25T13:08:02.322Z] ======================================================================
[2022-05-25T13:08:02.322Z] FAIL: test_read_patches_9 (tests.test_sliding_patch_wsi_dataset.TestSlidingPatchWSIDatasetCuCIM)
[2022-05-25T13:08:02.322Z] ----------------------------------------------------------------------
[2022-05-25T13:08:02.322Z] Traceback (most recent call last):
[2022-05-25T13:08:02.322Z]   File "/opt/conda/lib/python3.8/site-packages/parameterized/parameterized.py", line 533, in standalone_func
[2022-05-25T13:08:02.322Z]     return func(*(a + p.args), **p.kwargs)
[2022-05-25T13:08:02.322Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_sliding_patch_wsi_dataset.py", line 227, in test_read_patches
[2022-05-25T13:08:02.322Z]     self.assertEqual(len(dataset), len(expected))
[2022-05-25T13:08:02.322Z] AssertionError: 1 != 2
[2022-05-25T13:08:02.322Z] 
[2022-05-25T13:08:02.322Z] ----------------------------------------------------------------------

with nvcr.io/nvidia/pytorch:22.04-py3 V100, please let me know if you need more info to debug

I think the class could inherit Randomizable to ensure the deterministic behaviour e.g.

MONAI/monai/apps/datasets.py

Line 33 in eb60b9c

class MedNISTDataset(Randomizable, CacheDataset):

@wyli
I'll add the Randomizable but with set_determinisim (and using numpy random), I'm wondering why it should produce different results! Do you have any idea?

@wyli
Copy link
Copy Markdown
Contributor

wyli commented May 25, 2022

I guess because there are multiple skipUnless in the tests, depending on the skipping conditions the np.random.randint is called various times:

MONAI/monai/data/wsi_datasets.py

Line 232 in b520d68

return tuple(np.random.randint(low, high) for low, high in offset_limits)

with the randomizable API, it's possible to use self.R.randint(low, high), so that it controls the local random state. pls see the seed usage: (

MONAI/tests/test_mednistdataset.py

Line 59 in db61a08

data = MedNISTDataset(root_dir=testing_dir, transform=transform, section="test", download=False, seed=42)
)

@bhashemian
Fixed randomness
e219e58 
Signed-off-by: Behrooz <[email protected]>
@wyli
Copy link
Copy Markdown
Contributor

wyli commented May 26, 2022

/build

@wyli wyli disabled auto-merge May 26, 2022 12:39
@wyli wyli enabled auto-merge (squash) May 26, 2022 12:39
@wyli
Copy link
Copy Markdown
Contributor

wyli commented May 26, 2022

/build

1 similar comment
@wyli
Copy link
Copy Markdown
Contributor

wyli commented May 26, 2022

/build

@bhashemian
Copy link
Copy Markdown
Member Author

bhashemian commented May 26, 2022

@wyli, it seems that blossom-ci has failed. Can you provide details about it? Thanks

@wyli
Copy link
Copy Markdown
Contributor

wyli commented May 26, 2022

I think the current errors are irrelevant to the PR, I'm working on it

bhashemian added a commit to bhashemian/MONAI that referenced this pull request May 26, 2022
@bhashemian
Add required files form Project-MONAI#4239
148243f 
Signed-off-by: Behrooz <[email protected]>
@wyli wyli merged commit 95b7c4f into Project-MONAI:dev May 26, 2022
@bhashemian bhashemian deleted the sliding-patch branch May 26, 2022 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ericspod ericspod ericspod left review comments

@Nic-Ma Nic-Ma Nic-Ma left review comments

@wyli wyli wyli approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Sliding Patch WSIReader for Inference

4 participants

@bhashemian @wyli @ericspod @Nic-Ma