5308 Add runtime cache support to CacheDataset#5365
Merged
wyli merged 102 commits intoProject-MONAI:devfrom Nov 9, 2022
Merged
5308 Add runtime cache support to CacheDataset#5365wyli merged 102 commits intoProject-MONAI:devfrom
CacheDataset#5365wyli merged 102 commits intoProject-MONAI:devfrom
Conversation
merge master
merge master
merge master
merge master
merge master
merge master
Signed-off-by: Wenqi Li <[email protected]> Fixes Project-MONAI#5090 ### Description update version numbers ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Wenqi Li <[email protected]>
…ONAI#5182) Signed-off-by: KumoLiu <[email protected]> Fixes Project-MONAI#5176. ### Description Remove batch dim in `SobelGradients` and `SobelGradientsd` to make it consistent with other post transforms. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: KumoLiu <[email protected]>
…t-MONAI#5183) Signed-off-by: KumoLiu <[email protected]> Fixes Project-MONAI#5177. ### Description Add channel dim in `ComputeHoVerMaps` and `ComputeHoVerMapsd` to make it consistent with other transforms. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: KumoLiu <[email protected]> Co-authored-by: Behrooz Hashemian <[email protected]>
Fixes Project-MONAI#5031. ### Description Update federated learning figure. Using SVG format. Also removes from svg from gitignore. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [x] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Holger Roth <[email protected]>
Signed-off-by: KumoLiu <[email protected]> Fixes Project-MONAI#5209. ### Description Fix typo in `RandScaleIntensityd` ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [x] In-line docstrings updated. Signed-off-by: KumoLiu <[email protected]>
fixes Project-MONAI#5193 fixes Project-MONAI#5204 ### Description revise the import error messages and ignore a torchaudio warning. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Wenqi Li <[email protected]>
…ey()` default (Project-MONAI#5213) Signed-off-by: Wenqi Li <[email protected]> Fixes Project-MONAI#5211 Fixes Project-MONAI#5214 ### Description - set device based on the first item: ```py device = self.rand_2d_elastic.device if device is None and isinstance(d[first_key], torch.Tensor): device = d[first_key].device # type: ignore self.rand_2d_elastic.set_device(device) ``` - simplied transform `first_key()` default to a tuple ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [x] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Wenqi Li <[email protected]>
Signed-off-by: Wenqi Li <[email protected]> Fixes Project-MONAI#3844 ### Description adding the flexibility for skipping the transform if the pixdim is in the specified range ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [x] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. - [x] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Wenqi Li <[email protected]>
Signed-off-by: Yiheng Wang <[email protected]> Fixes Project-MONAI#4264 . ### Description This PR adds the antialiasing option for `Resized` transform. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Yiheng Wang <[email protected]>
…les (Project-MONAI#5224) ### Description Some users may have models that take adjoint information. This PR allows the adjoint information to be passed through the visualisation classes to the model via `**kwargs`. Also combined the test files of GradCAM and GradCAM++ as they were the same. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [x] New tests added to cover the changes. - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. Signed-off-by: Richard Brown <[email protected]>
### Description We should be using `PY_EXE` in runtests as opposed to raw `pip`. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). Signed-off-by: Richard Brown <[email protected]>
Fixes Project-MONAI#5232 . ### Description According to user's feedback, this PR enhanced the MONAI bundle doc page to include tutorial examples and model-zoo info. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Nic Ma <[email protected]>
Signed-off-by: Wenqi Li <[email protected]> Fixes Project-MONAI#5163 ### Description enhance metatensor compatibility for `OneOf` ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [x] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Wenqi Li <[email protected]>
Fixes Project-MONAI#5233 . ### Description This PR added file existing check for the logging config, requested by @holgerroth . ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Nic Ma <[email protected]> Co-authored-by: Wenqi Li <[email protected]>
Signed-off-by: KumoLiu <[email protected]> Fixes Project-MONAI#5225 . ### Description 1. fix 1d data error in `VarAutoEncoder` 2. add `use_sigmoid` flag ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [x] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: KumoLiu <[email protected]> Co-authored-by: Wenqi Li <[email protected]>
1. Fix the issue Project-MONAI#5227 2. fix a small code flaw Signed-off-by: David <[email protected]>
Signed-off-by: Holger Roth <[email protected]> Fixes Project-MONAI#5195. ### Description Add support for MonaiAlgo to be run with torchrun for multi-gpu training. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Holger Roth <[email protected]>
Fixes Project-MONAI#5218 ### Description This PR moves `HoVerNet` Mode and Branch outside of the module and make them independent `StrEnums`. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [x] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. Signed-off-by: Behrooz <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…#5230) ### Description Occlusion sensitivity currently works by setting the logits at the center of the occluded region. Even though neighbouring voxels are occluded at the same time, this is not recorded. A better way to do it would be to do an average each time a voxel is occluded. This is taken care of by switching to using `sliding_window_inference`. We had to do a bit of hacking to get this to work. `sliding_window_inference` normally takes a subset of the whole image and infers that. We want `sliding_window_inference` to tell us which part of the image we should occlude, then we occlude it and infer the **whole** image. To that end, we use a meshgrid to tell us the coordinates of the region `sliding_window_inference` wants us to occlude. We occlude, infer and then crop back to the size of the region that was requested by `sliding_window_inference`. This PR also allows for different occlusion kernels. We currently have: - gaussian (actually inverse gaussian): the center of the occluded region is zero, and towards the edge of the image is unchanged. This doesn't introduce hard edges into the image, which might undermine the visualisation process. - mean_patch: the occluded region is replaced with the mean of the patch it is occluding. - mean_img: the occluded region is replaced with the mean of the whole image (current implementation). ## Changes to input arguments This PR is backwards incompatible, as using `sliding_window_inference` means changing the API significantly. - pad_val: now determined by `mode` - stride: `overlap` used instead - per_channel: all channels are done simultaneously - upsampler: image is no longer downsampled ## Changes to output Output previously had the shape `B,C,H,W,[D],N` where `C` and `N` were the number of input and output channels of the network, respectively. Now, we output the shape `B,N,H,W,[D]` as the `per_channel` feature is no longer present. Columns 2-4 are occlusion sensitivity done with Gaussian, mean of patch and mean of image:  ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [x] Breaking change (fix or new feature that would cause existing functionality to change). - [x] New tests added to cover the changes. - [x] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. - [x] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Richard Brown <[email protected]>
### Description Adds (optional) kernel_size parameter to UpSample, used for deconv (convolution transpose up-sampling). This allows to upsample, e.g to upscale to 2x with a kernel_size 3. (currently the default is to upscale to 2x with a kernel size 2) if this parameter is not set, the behavior is the same as before ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [x] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [x] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: myron <[email protected]>
Signed-off-by: Wenqi Li <[email protected]> Fixes Project-MONAI#5251 ### Description replace `None` with `"none"` in the readers ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [x] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Wenqi Li <[email protected]>
Signed-off-by: Can Zhao <[email protected]> Fixes Project-MONAI#5250. ### Description Fixed bug that allow_low_quality_matches in set_regular_matcher was always set to True and it's not using value from the argument, in retinanet. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Can Zhao <[email protected]> Signed-off-by: monai-bot <[email protected]> Co-authored-by: monai-bot <[email protected]>
…-MONAI#5256) <!--pre-commit.ci start--> updates: - [github.com/asottile/pyupgrade: v2.34.0 → v2.38.2](asottile/pyupgrade@v2.34.0...v2.38.2) - [github.com/asottile/yesqa: v1.3.0 → v1.4.0](asottile/yesqa@v1.3.0...v1.4.0) - [github.com/hadialqattan/pycln: v1.3.5 → v2.1.1](hadialqattan/pycln@v1.3.5...v2.1.1) <!--pre-commit.ci end--> Signed-off-by: Wenqi Li <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> Co-authored-by: Wenqi Li <[email protected]>
Signed-off-by: Wenqi Li <[email protected]> Fixes Project-MONAI#5253 ### Description enhance computing offset for spacing ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [x] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Wenqi Li <[email protected]>
Contributor
Author
|
/build |
Signed-off-by: monai-bot <[email protected]>
CacheDatasetCacheDataset
Contributor
Author
|
/build |
myron
approved these changes
Nov 8, 2022
Collaborator
|
@Nic-Ma the code looks good. Visually, it seems correct. I'd only add a small information for users that |
Signed-off-by: Nic Ma <[email protected]>
Contributor
Author
|
/black |
Contributor
Author
|
/build |
CacheDatasetCacheDataset
Contributor
Author
|
I can't reproduce the CI error locally, let me change to a multi-gpu V100 server and test again. Thanks. |
Collaborator
|
looks great! thank you |
Contributor
Author
|
/black |
Contributor
Author
|
/build |
Signed-off-by: Nic Ma <[email protected]>
10051dc to
18a651a
Compare
Contributor
Author
|
/build |
1 similar comment
Contributor
|
/build |
CacheDatasetCacheDataset
Contributor
Author
|
/build |
bhashemian
pushed a commit
to bhashemian/MONAI
that referenced
this pull request
Nov 23, 2022
…#5365) Fixes Project-MONAI#5308 . ### Description This PR added runtime cache support in the `CacheDataset`. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Wenqi Li <[email protected]> Signed-off-by: Behrooz <[email protected]>
This was referenced Dec 2, 2022
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5308 .
Description
This PR added runtime cache support in the
CacheDataset.Types of changes
./runtests.sh -f -u --net --coverage../runtests.sh --quick --unittests --disttests.make htmlcommand in thedocs/folder.