Skip to content

merge master#63

Merged
Nic-Ma merged 135 commits intoNic-Ma:1900-save-batchfrom
Project-MONAI:master
Apr 2, 2021
Merged

merge master#63
Nic-Ma merged 135 commits intoNic-Ma:1900-save-batchfrom
Project-MONAI:master

Conversation

@Nic-Ma
Copy link
Copy Markdown
Owner

@Nic-Ma Nic-Ma commented Apr 2, 2021

Fixes # .

Description

A few sentences describing the changes proposed in this pull request.

Status

Ready/Work in progress/Hold

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh --codeformat --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

wyli and others added 30 commits February 24, 2021 15:02
* update openmp flag

Signed-off-by: Wenqi Li <[email protected]>

* improves boundtype docs

Signed-off-by: Wenqi Li <[email protected]>

* update setup.py

Signed-off-by: Wenqi Li <[email protected]>

* input validation 1d

Signed-off-by: Wenqi Li <[email protected]>

* fixes typos

Signed-off-by: Wenqi Li <[email protected]>

* fixes typos

Signed-off-by: Wenqi Li <[email protected]>

* merge upstream changes

Signed-off-by: Wenqi Li <[email protected]>

* tests enums

Signed-off-by: Wenqi Li <[email protected]>

* init. test grid pull

Signed-off-by: Wenqi Li <[email protected]>

* update

Signed-off-by: Wenqi Li <[email protected]>

* test grid_pull

Signed-off-by: Wenqi Li <[email protected]>

* fixes min test

Signed-off-by: Wenqi Li <[email protected]>

* adds device tests

Signed-off-by: Wenqi Li <[email protected]>

* bwd tests

Signed-off-by: Wenqi Li <[email protected]>
move transforms out of compose file. this is backwards compatible.
* pad_collation

Signed-off-by: Richard Brown <[email protected]>

* increase number of test cases to ensure required testing errors

Signed-off-by: Richard Brown <[email protected]>

* determinism in setUp

Signed-off-by: Richard Brown <[email protected]>

* pad collate for list of lists

Signed-off-by: Richard Brown <[email protected]>

* code format

Signed-off-by: Richard Brown <[email protected]>

* allow padding options

Signed-off-by: Richard Brown <[email protected]>
decollate batch
enhance random range parameters

If element `i` is iterable, then `uniform[-rotate_range[i][0], rotate_range[i][1])` will be used to generate the rotation parameter for the ith dimension. If not, `uniform[-rotate_range[i], rotate_range[i])` will be used. This can be altered on a per-dimension basis. E.g., `((0,3), 1, ...)`: for dim0, rotation will be in range `[0, 3]`, and for dim1 `[-1, 1]` will be used. Setting a single value will use `[-x, x]` for dim0 and nothing for the remaining dimensions.

Backwards compatibility is ensured because the old behaviour used to do `uniform[-rotate_range[i], rotate_range[i])`, and this PR just adds the possibility to do `uniform[-rotate_range[i][0], rotate_range[i][1])`.
* Fixing issues preventing loss functions from being compatible with Torchscript

Signed-off-by: Eric Kerfoot <[email protected]>

* Updates

Signed-off-by: Eric Kerfoot <[email protected]>

* Updates

Signed-off-by: Eric Kerfoot <[email protected]>

* [MONAI] python code formatting

Signed-off-by: monai-bot <[email protected]>

* Updates

Signed-off-by: Eric Kerfoot <[email protected]>

* Adding conditional skip to Torchscript tests

Signed-off-by: Eric Kerfoot <[email protected]>

Co-authored-by: monai-bot <[email protected]>
* [DLMED] fix length > 1024 issue in string list all gather

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update according to comments

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update according to comments

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] add more test

Signed-off-by: Nic Ma <[email protected]>

Co-authored-by: Yiheng Wang <[email protected]>
* DeleteChannel array, dictionary, tests

Signed-off-by: Leo Tam <[email protected]>

* formatting

Signed-off-by: Leo Tam <[email protected]>

* DeleteChannel array, dictionary, tests

Signed-off-by: Leo Tam <[email protected]>

* Register in transform init

Signed-off-by: Leo Tam <[email protected]>

* clean comment

Signed-off-by: Leo Tam <[email protected]>

* Dict init

Signed-off-by: Leo Tam <[email protected]>

* Register D and Dict

Signed-off-by: Leo Tam <[email protected]>

* Style fix

Signed-off-by: Leo Tam <[email protected]>

* Safety cast

Signed-off-by: Leo Tam <[email protected]>

* Changinging names from DeleteChannel to RemoveRepeatedChannel

Signed-off-by: Leo Tam <[email protected]>

* Lint fixes

Signed-off-by: Leo Tam <[email protected]>

* Test import fix, autofix again

Signed-off-by: Leo Tam <[email protected]>
Signed-off-by: Nic Ma <[email protected]>

Co-authored-by: Eric Kerfoot <[email protected]>
* update distcall

Signed-off-by: Wenqi Li <[email protected]>

* update final closing

Signed-off-by: Wenqi Li <[email protected]>

Co-authored-by: Eric Kerfoot <[email protected]>
* Add ToPIL transformation

Signed-off-by: Behrooz <[email protected]>

* Add ToPILd, ToPILD, ToPILDict

Signed-off-by: Behrooz <[email protected]>

* Remove has_pil

Signed-off-by: Behrooz <[email protected]>

* Include ToPIL, ToPILd, ToPILD, and ToPILDict

Also include ToNumpyD, ToNumpyDict, TorchVisionD, and TorchVisionDict

Signed-off-by: Behrooz <[email protected]>

* Fix a typing issue

Signed-off-by: Behrooz <[email protected]>

* Fix PIL optional import

Signed-off-by: Behrooz <[email protected]>

* Add unittests for ToPIL and ToPILD

Signed-off-by: Behrooz <[email protected]>

* Fix formatting

Signed-off-by: Behrooz <[email protected]>

* Fix formatting

Signed-off-by: Behrooz <[email protected]>

* Add PILImage.Image as the input for ToNumpy and ToTensor

Signed-off-by: Behrooz <[email protected]>

* Fix type checking issue for PIL.Image.Image

Signed-off-by: Behrooz <[email protected]>

* Change PILImage_fromarray to lower case

Signed-off-by: Behrooz <[email protected]>
add RandomizableTransform
* Add more tests to test_deepgrow_dataset

Signed-off-by: YuanTingHsieh <[email protected]>
* Add inference transforms

Signed-off-by: YuanTingHsieh <[email protected]>

* Remove unused import

Signed-off-by: YuanTingHsieh <[email protected]>

* Fix review comments

Signed-off-by: YuanTingHsieh <[email protected]>

Co-authored-by: SACHIDANAND ALLE <[email protected]>
* [DLMED] add RandAxisFlip transforms

Signed-off-by: Nic Ma <[email protected]>

* [MONAI] python code formatting

Signed-off-by: monai-bot <[email protected]>

* [DLMED] fix flake8 issues

Signed-off-by: Nic Ma <[email protected]>

Co-authored-by: monai-bot <[email protected]>
fix big endianness problem for dictionary reading. Also allow dictionary image_only
* [DLMED] add AutoAdjustChannel transform

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] add dict version transform

Signed-off-by: Nic Ma <[email protected]>

* [MONAI] python code formatting

Signed-off-by: monai-bot <[email protected]>

* [DLMED] fix doc-build issue

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update according to Wenqi's comments

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update doc-strings

Signed-off-by: Nic Ma <[email protected]>

* [MONAI] python code formatting

Signed-off-by: monai-bot <[email protected]>

Co-authored-by: monai-bot <[email protected]>
Signed-off-by: Richard Brown <[email protected]>
* Update load pretrain for densenet

Signed-off-by: Yiheng Wang <[email protected]>

* Fix isort issue

Signed-off-by: Yiheng Wang <[email protected]>
* [DLMED] add suqeeze to handler

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update according to comments

Signed-off-by: Nic Ma <[email protected]>
* fixes #1285

Signed-off-by: Wenqi Li <[email protected]>

* adds test

Signed-off-by: Wenqi Li <[email protected]>
* fixes #1685

Signed-off-by: Wenqi Li <[email protected]>

* add temp test

Signed-off-by: Wenqi Li <[email protected]>

* adds docstring

Signed-off-by: Wenqi Li <[email protected]>

* fixes dist sampler

Signed-off-by: Wenqi Li <[email protected]>

* remove temp tests

Signed-off-by: Wenqi Li <[email protected]>

* fixes type hint issue

Signed-off-by: Wenqi Li <[email protected]>
* fixes data type in switching

Signed-off-by: Wenqi Li <[email protected]>

* [MONAI] python code formatting

Signed-off-by: monai-bot <[email protected]>

Co-authored-by: monai-bot <[email protected]>
* Implement CuImageReader and OpenSlideReader

Signed-off-by: Behrooz <[email protected]>

* Add unittests for CuImageReader

Signed-off-by: Behrooz <[email protected]>

* Add unittests for OpenSlideReader

Signed-off-by: Behrooz <[email protected]>

* Sort imports

Signed-off-by: Behrooz <[email protected]>

* Add correct boundaries

Signed-off-by: Behrooz <[email protected]>

* Add test cases for reading patches on a grid for CuImage

Signed-off-by: Behrooz <[email protected]>

* Add patch whole slide imaging dataset for pathology

Signed-off-by: Behrooz <[email protected]>

* Add test case for read patches for OpenSlide

Signed-off-by: Behrooz <[email protected]>

* flake8 and few minor changes

Signed-off-by: Behrooz <[email protected]>

* black

Signed-off-by: Behrooz <[email protected]>

* flake8

Signed-off-by: Behrooz <[email protected]>

* Add kwargs to CuImageReader and OpenSlideReader's read method

Signed-off-by: Behrooz <[email protected]>

* Change the type hint from np.dtype to DTypeLike

Signed-off-by: Behrooz <[email protected]>

* Fix a bug

Signed-off-by: Behrooz <[email protected]>

* Implement WSIReader and unittests

Signed-off-by: Behrooz <[email protected]>

* Minor updates

Signed-off-by: Behrooz <[email protected]>

* Fix few typing issues

Signed-off-by: Behrooz <[email protected]>

* Revert datasets

Signed-off-by: Behrooz <[email protected]>

* Add shape property to openslide image object
Reverse size to be compatible with output size (hxw)

Signed-off-by: Behrooz <[email protected]>

* Add untittest for loading the whole image
Reverse the size accroding to the WSIReader

Signed-off-by: Behrooz <[email protected]>

* Update the whole image size

Signed-off-by: Behrooz <[email protected]>

* Remove optional size

Signed-off-by: Behrooz <[email protected]>

* Remove optional dtype

Signed-off-by: Behrooz <[email protected]>

* Remove _get_spatial_shape return type

Signed-off-by: Behrooz <[email protected]>

* Reverse the orders of dimensions of `location`
to be compatible with image shape

Signed-off-by: Behrooz <[email protected]>

* Change test cases to use smaller image and revese location's dimensions

Signed-off-by: Behrooz <[email protected]>

* Replace the test TIFF and some upgrades

Signed-off-by: Behrooz <[email protected]>

* Update dependencies for OpenSlide

Signed-off-by: Behrooz <[email protected]>

* Update unittests for OpenSlide and CuImage

Signed-off-by: Behrooz <[email protected]>

* Fix openslide dependency

Signed-off-by: Behrooz <[email protected]>

* Fix doc dependencies

Signed-off-by: Behrooz <[email protected]>

* Minor changes

Signed-off-by: Behrooz <[email protected]>

* Few variable name changes

Signed-off-by: Behrooz <[email protected]>

* Add EnsureChannelFirst

Signed-off-by: Behrooz <[email protected]>

* Add metadata to WSIReader

Signed-off-by: Behrooz <[email protected]>
rijobro and others added 29 commits March 26, 2021 19:53
* [DLMED] add MapLabelValue

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] add unit tests

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] add missing doc-string

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update according to comments

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <[email protected]>
* Add unittests for SmartCachePatchWSIDataset

Signed-off-by: Behrooz <[email protected]>

* Turn off shuffle for SmartCachePatchWSIDataset

Signed-off-by: Behrooz <[email protected]>
* fixes #1848

Signed-off-by: Wenqi Li <[email protected]>

* temp tests

Signed-off-by: Wenqi Li <[email protected]>

* Revert "temp tests"

This reverts commit 563e7ea.

Signed-off-by: Wenqi Li <[email protected]>
Allowing the choice of epoch and iteration interval.

Signed-off-by: Petru-Daniel Tudosiu <[email protected]>
* [DLMED] enhance list_collate

Signed-off-by: Nic Ma <[email protected]>

* [MONAI] python code formatting

Signed-off-by: monai-bot <[email protected]>

* [DLMED] update based on comments

Co-authored-by: Wenqi Li <[email protected]>
Signed-off-by: Nic Ma <[email protected]>

Co-authored-by: monai-bot <[email protected]>
Co-authored-by: Wenqi Li <[email protected]>
* [DLMED] add slicing support in datasets

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] add unit tests

Signed-off-by: Nic Ma <[email protected]>

* [MONAI] python code formatting

Signed-off-by: monai-bot <[email protected]>

* [DLMED] fix typo

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update according to comments

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update doc-strings and typehints

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <[email protected]>

Co-authored-by: monai-bot <[email protected]>
* download_if_not_exist

Signed-off-by: Behrooz <[email protected]>

* Update cuimage reader tests

Signed-off-by: Behrooz <[email protected]>

* Update openslide reader tests

Signed-off-by: Behrooz <[email protected]>

* Update the file path to be relative to the test file

Signed-off-by: Behrooz <[email protected]>

* Update patch wsi dataset tests

Signed-off-by: Behrooz <[email protected]>

* Update smartcache patch wsi dataset tests

Signed-off-by: Behrooz <[email protected]>

* Revert "download_if_not_exist"

This reverts commit a4c65f3.

Signed-off-by: Behrooz <[email protected]>

* Update with download_url

Signed-off-by: Behrooz <[email protected]>

* [MONAI] python code formatting
* Implement random bias field transform

Signed-off-by: Yiheng Wang <[email protected]>
* Fixing error in saving non-pytorch-variable parameters for backwards pass

Signed-off-by: chaliebudd <[email protected]>

* Adding unit tests to check bilateral filter backwards runs

Signed-off-by: chaliebudd <[email protected]>

* fixing typo introduced by merge

Signed-off-by: chaliebudd <[email protected]>

* code reformatting

Signed-off-by: chaliebudd <[email protected]>
* fixes warping

Signed-off-by: Wenqi Li <[email protected]>

* fixes types

Signed-off-by: Wenqi Li <[email protected]>

* update based on comments

Signed-off-by: Wenqi Li <[email protected]>

* fixes warnings in setup.cfg, config, deprecated assertexecption

Signed-off-by: Wenqi Li <[email protected]>
* Remove assert statement from non-test files

* Remove redundant `None` default

* Use `is` to compare type of objects

* Refactor unnecessary `else` / `elif` when `if` block has a `raise` statement

* Refactor unnecessary `else` / `elif` when `if` block has a `return` statement

* runtests.sh --autofix

Signed-off-by: Wenqi Li <[email protected]>

* autofix/7d8e8b86-113e-4a5a-8d0d-c621aab7b5f2

Signed-off-by: Wenqi Li <[email protected]>

* fixes assert types

Signed-off-by: Wenqi Li <[email protected]>

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Co-authored-by: Wenqi Li <[email protected]>
* Implement MaskedInferenceWSIDataset for pathology inference

Signed-off-by: Behrooz <[email protected]>

* Update pathology init

Signed-off-by: Behrooz <[email protected]>

* Update docs

Signed-off-by: Behrooz <[email protected]>

* Remove last elemnt of cum_num_patches

Signed-off-by: Behrooz <[email protected]>

* Add unittest with multiple cases for MaskedInferenceWSIDataset

Signed-off-by: Behrooz <[email protected]>

* sort imports in init

Signed-off-by: Behrooz <[email protected]>

* Remove list dataset

Signed-off-by: Behrooz <[email protected]>

* Remove try/except and add type hint

Signed-off-by: Behrooz <[email protected]>

* Convert the sample output to a list

Signed-off-by: Behrooz <[email protected]>

* Remove some type hints

Signed-off-by: Behrooz <[email protected]>

* Remove prints

Signed-off-by: Behrooz <[email protected]>

* Update based on commnets

Signed-off-by: Behrooz <[email protected]>

* Update patch_size

Signed-off-by: Behrooz <[email protected]>

* Update unittests

Signed-off-by: Behrooz <[email protected]>

* Add more type hints

Signed-off-by: Behrooz <[email protected]>
* remove unused in_channels in LNCC

Signed-off-by: Wenqi Li <[email protected]>

* update integration tests

Signed-off-by: Wenqi Li <[email protected]>

* extending warp options

Signed-off-by: Wenqi Li <[email protected]>
* [DLMED] enhance brats transform

Signed-off-by: Nic Ma <[email protected]>

* [MONAI] python code formatting

Signed-off-by: monai-bot <[email protected]>

Co-authored-by: monai-bot <[email protected]>
* Fix stdshift type error

Signed-off-by: Yiheng Wang <[email protected]>

* Add dtype testcases

Signed-off-by: Yiheng Wang <[email protected]>
* Implement ProbMapGenerator handler

Signed-off-by: Behrooz <[email protected]>

* Update doc and init

Signed-off-by: Behrooz <[email protected]>

* Sort init imports

Signed-off-by: Behrooz <[email protected]>

* Add unittest for ProbMapGenerator

Signed-off-by: Behrooz <[email protected]>

* Ignore if ignite is not available

Signed-off-by: Behrooz <[email protected]>

* Update Engine import

Signed-off-by: Behrooz <[email protected]>

* Exclude from min-test

Signed-off-by: Behrooz <[email protected]>

* Address all the comments

Signed-off-by: Behrooz <[email protected]>

* Fix file path and dtype

Signed-off-by: Behrooz <[email protected]>
* Implement MaskedInferenceWSIDataset for pathology inference

Signed-off-by: Behrooz <[email protected]>

* Update pathology init

Signed-off-by: Behrooz <[email protected]>

* Update docs

Signed-off-by: Behrooz <[email protected]>

* Remove last elemnt of cum_num_patches

Signed-off-by: Behrooz <[email protected]>

* Add unittest with multiple cases for MaskedInferenceWSIDataset

Signed-off-by: Behrooz <[email protected]>

* sort imports in init

Signed-off-by: Behrooz <[email protected]>

* Remove list dataset

Signed-off-by: Behrooz <[email protected]>

* Remove try/except and add type hint

Signed-off-by: Behrooz <[email protected]>

* Convert the sample output to a list

Signed-off-by: Behrooz <[email protected]>

* Remove some type hints

Signed-off-by: Behrooz <[email protected]>

* Implement FROC calcualtion for pathology

Signed-off-by: Behrooz <[email protected]>

* Update ProbNMS doctring

Signed-off-by: Behrooz <[email protected]>

* Update docs and change namings

Signed-off-by: Behrooz <[email protected]>

* Fix a bug and minor changes

Signed-off-by: Behrooz <[email protected]>

* Minor changes

Signed-off-by: Behrooz <[email protected]>

* Fix docstring formatting

Signed-off-by: Behrooz <[email protected]>

* Add a type hint

Signed-off-by: Behrooz <[email protected]>

* Implement unittests for EvaluateTumorFROC

Signed-off-by: Behrooz <[email protected]>

* Ignore type for np.amax

Signed-off-by: Behrooz <[email protected]>

* Remove space

Signed-off-by: Behrooz <[email protected]>

* Ignore type for range instead of np.amax

Signed-off-by: Behrooz <[email protected]>

* Skip test if PIL is not available

Signed-off-by: Behrooz <[email protected]>

* Update docstring

Signed-off-by: Behrooz <[email protected]>

* Skip ground truth generating if PIL is not available

Signed-off-by: Behrooz <[email protected]>

* Update unittest

Signed-off-by: Behrooz <[email protected]>

* Remove print

Signed-off-by: Behrooz <[email protected]>

* Rename TumorFROC and add few type hints

Signed-off-by: Behrooz <[email protected]>

* Rename evaluators to metrics

Signed-off-by: Behrooz <[email protected]>

* Remove non-relevant files

Signed-off-by: Behrooz <[email protected]>

* Rename to LesionFROC and minor changes

Signed-off-by: Behrooz <[email protected]>

* Update test

Signed-off-by: Behrooz <[email protected]>
Signed-off-by: Behrooz <[email protected]>
* Rename the prob map producer unittest to match the module

Signed-off-by: Behrooz <[email protected]>

* Change probs_map to prob_map

Signed-off-by: Behrooz <[email protected]>

* Prepend image_inference_outputs with temp to be ignored

Signed-off-by: Behrooz <[email protected]>
* followup of #1878, fixes tests, remove json loading

Signed-off-by: Wenqi Li <[email protected]>

* Update test ordinal numbers

Signed-off-by: Behrooz <[email protected]>

Co-authored-by: Behrooz <[email protected]>
* clip variance to be >= 0

Signed-off-by: kate-sann5100 <[email protected]>

* max for torch1.6

Signed-off-by: Wenqi Li <[email protected]>

Co-authored-by: Wenqi Li <[email protected]>
* update to use pytorch2103

Signed-off-by: Wenqi Li <[email protected]>
@Nic-Ma Nic-Ma merged commit f60772b into Nic-Ma:1900-save-batch Apr 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.