Implement NrrdReader by kbressem · Pull Request #4259 · Project-MONAI/MONAI

kbressem · 2022-05-11T17:26:16Z

Signed-off-by: kbressem [email protected]

Description

This PR proposes the NrrdReader class to correctly read .nrrd and .seg.nrrd files.

Status

Ready for review

Files changes/created

monai/data/image_reader.py --> add NrrdReader and NrrdImage
monai/data/__init__.py --> add NrrdReader to imports
monai/transforms/io/array.py --> add NrrdReader to imports and "nrrdreader" to SUPPORTED_READERS
monai/config/deviceconfig.py --> add pynrrd to get_optional_config_values

tests/test_init_reader.py  --> add test for NrrdReader
tests/min_tests.py  --> exclude tests for NrrdReader
requirements-dev.txt --> add pynrrd

FIles created

tests/test_nrrd_reader.py --> tests for NrrdReader

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).
New tests added to cover the changes.
Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
In-line docstrings updated.
Documentation updated, tested make html command in the docs/ folder.

* add swin_unetr model Signed-off-by: kbressem <[email protected]>

* [DLMED] update to 22.04 Signed-off-by: Nic Ma <[email protected]> * fixes unit test tests.test_lr_finder Signed-off-by: Wenqi Li <[email protected]> * test new_empty Signed-off-by: Wenqi Li <[email protected]> Co-authored-by: Wenqi Li <[email protected]> Co-authored-by: Wenqi Li <[email protected]> Signed-off-by: kbressem <[email protected]>

* implement the base class Signed-off-by: Yiheng Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add unittest Signed-off-by: Yiheng Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * autofix Signed-off-by: Yiheng Wang <[email protected]> * switch to call apex directly Signed-off-by: Yiheng Wang <[email protected]> * uncomment unittest Signed-off-by: Yiheng Wang <[email protected]> * add apex install link in docstring Signed-off-by: Yiheng Wang <[email protected]> * add channels_last_3d test case Signed-off-by: Yiheng Wang <[email protected]> * rewrite types Signed-off-by: Yiheng Wang <[email protected]> * change types Signed-off-by: Yiheng Wang <[email protected]> * add docstrings Signed-off-by: Yiheng Wang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Wenqi Li <[email protected]> Signed-off-by: kbressem <[email protected]>

* Update dice.py reduce redundant operations in DiceFocalLoss, initially caused oom Signed-off-by: Ryan Clanton <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Ryan Clanton <[email protected]> * [MONAI] python code formatting Signed-off-by: monai-bot <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: monai-bot <[email protected]> Signed-off-by: kbressem <[email protected]>

* Make all transforms optional Signed-off-by: Behrooz <[email protected]> * Update wsireader tests Signed-off-by: Behrooz <[email protected]> * Remove optional from PersistentDataset and its derivatives Signed-off-by: Behrooz <[email protected]> * Add unittests for cache without transform Signed-off-by: Behrooz <[email protected]> * Add default replace_rate Signed-off-by: Behrooz <[email protected]> * Add default value Signed-off-by: Behrooz <[email protected]> * Set default replace_rate to 0.1 Signed-off-by: Behrooz <[email protected]> * Update metadata to include path Signed-off-by: Behrooz <[email protected]> * Adds SmartCachePatchWSIDataset Signed-off-by: Behrooz <[email protected]> * Add unittests for SmartCachePatchWSIDataset Signed-off-by: Behrooz <[email protected]> * Update references Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * Remove smart cache Signed-off-by: Behrooz <[email protected]> * Remove unused imports Signed-off-by: Behrooz <[email protected]> * Add path metadata for OpenSlide Signed-off-by: Behrooz <[email protected]> * Update metadata to be unified across different backends Signed-off-by: Behrooz <[email protected]> * Update wsi metadata for multi wsi objects Signed-off-by: Behrooz <[email protected]> * Add unittests for wsi metadata Signed-off-by: Behrooz <[email protected]> Signed-off-by: kbressem <[email protected]>

* replace modules Signed-off-by: Richard Brown <[email protected]> * fix Signed-off-by: Richard Brown <[email protected]> * replace_module -> replace_modules Signed-off-by: Richard Brown <[email protected]> * fix Signed-off-by: Richard Brown <[email protected]> Signed-off-by: kbressem <[email protected]>

Signed-off-by: Can Zhao <[email protected]> Signed-off-by: kbressem <[email protected]>

* reproduce issue Signed-off-by: Yiheng Wang <[email protected]> * remove 22.01 02 Signed-off-by: Yiheng Wang <[email protected]> * remove other workflows Signed-off-by: Yiheng Wang <[email protected]> * run on pull request Signed-off-by: Yiheng Wang <[email protected]> * remove sleep Signed-off-by: Yiheng Wang <[email protected]> * test single layer forward Signed-off-by: Yiheng Wang <[email protected]> * add has_nvfuser Signed-off-by: Yiheng Wang <[email protected]> * add check within factory Signed-off-by: Yiheng Wang <[email protected]> * revert to original cron.yml Signed-off-by: Yiheng Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix old pt issue Signed-off-by: Yiheng Wang <[email protected]> * change to return directly if no cuda Signed-off-by: Yiheng Wang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: kbressem <[email protected]>

* Update to bundle specifiation Signed-off-by: Eric Kerfoot <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding description in spec discussing the saved Torchscript object's file storage behaviour, and tweaking ckpt_export to add .json extension Signed-off-by: Eric Kerfoot <[email protected]> * Annotating optional bundle files Signed-off-by: Eric Kerfoot <[email protected]> * Adjusted ckpt_export test Signed-off-by: Eric Kerfoot <[email protected]> * Fix Signed-off-by: Eric Kerfoot <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: kbressem <[email protected]>

Signed-off-by: kbressem <[email protected]>

for more information, see https://pre-commit.ci Signed-off-by: kbressem <[email protected]>

Signed-off-by: kbressem <[email protected]>

make flake8 happy Signed-off-by: kbressem <[email protected]>

Changed NrrdImage to dataclass Signed-off-by: kbressem <[email protected]>

Nic-Ma · 2022-05-13T10:19:46Z

Hi @kbressem ,

Thanks very much for your contribution.
There are several places need to add the optional package, you can search pandas as the reference.

Thanks in advance.

Signed-off-by: kbressem <[email protected]>

…nto 4238-nrrd-reader

kbressem · 2022-05-13T11:01:39Z

Hi @Nic-Ma

I've searched for optional_import("pandas"), pandas, 'optional_import("nibabel")', nibabel and nib in all scripts. Then I went through the scripts, to check if adding nrrd as optional import would be usefull.

`pandas` was imported in:

monai/data/iterable_dataset.py No image reader is called. I believe nrrd would not be used
monai/data/utils.py pandas is used to convert tables to dicts. nrrd would not be used
monai/data/dataset.py No image reader is called. I believe nrrd would not be used
monai/config/deviceconfig.py I've added nrrd to get_optional_config_values

`nibabel` was imported in:

monai/transforms/spatial/dictionary.py nibabel is used in Orientationd only. nrrd would not be used.
monai/transforms/spatial/array.py nibabel is used in SpatialResample only, nrrd would not be used.
monai/transforms/io/array.py added nrrd as optional import and "nrrdreader" to SUPPORTED_READERS
monai/data/image_reader.py obviously added nrrd :)
monai/data/utils.py nibabel is used to reorient spatial axes. nrrd would not be used
monai/data/nifti_writer.py nrrd would not be used
monai/data/image_writer.py Currently there is no NrrdWriter. We are working on one in MONAI-Label and would like to contribute in a future PR.
monai/utils/module.py adding nrrd here would not make sense
monai/config/deviceconfig.py I've added nrrd to get_optional_config_values

I did not find any new instance where importing nrrd would be useful. Did I miss something?

Nic-Ma · 2022-05-13T11:06:09Z

Hi @kbressem ,

Sorry maybe I didn't make it clear.
I mean you need to add new optional import into some project config files, referring to pandas. For example:
https://github.com/Project-MONAI/MONAI/blob/dev/setup.cfg
https://github.com/Project-MONAI/MONAI/blob/dev/docs/requirements.txt
https://github.com/Project-MONAI/MONAI/blob/dev/requirements-dev.txt
https://github.com/Project-MONAI/MONAI/blob/dev/environment-dev.yml
https://github.com/Project-MONAI/MONAI/blob/dev/docs/source/installation.md
https://github.com/Project-MONAI/MONAI/blob/dev/monai/config/deviceconfig.py

Thanks.

Signed-off-by: kbressem <[email protected]>

for more information, see https://pre-commit.ci

kbressem · 2022-05-13T11:24:10Z

Hi @kbressem ,

Sorry maybe I didn't make it clear. I mean you need to add new optional import into some project config files, referring to pandas. For example: https://github.com/Project-MONAI/MONAI/blob/dev/setup.cfg https://github.com/Project-MONAI/MONAI/blob/dev/docs/requirements.txt https://github.com/Project-MONAI/MONAI/blob/dev/requirements-dev.txt https://github.com/Project-MONAI/MONAI/blob/dev/environment-dev.yml https://github.com/Project-MONAI/MONAI/blob/dev/docs/source/installation.md https://github.com/Project-MONAI/MONAI/blob/dev/monai/config/deviceconfig.py

Thanks.

Thank you for the clarification. I have added pynrrd to all required files.

wyli · 2022-05-13T11:26:10Z

this looks nice, one additional test that is worth adding is to check the consistency of the writer/reader...

ideally we should have a TestLoadSaveNrrd, structured like this one:

MONAI/tests/test_image_rw.py

Lines 82 to 117 in 812275a

    
           class TestLoadSavePNG(unittest.TestCase): 
        
               def setUp(self): 
        
                   self.test_dir = tempfile.mkdtemp() 
        
               def tearDown(self): 
        
                   shutil.rmtree(self.test_dir, ignore_errors=True) 
        
               def png_rw(self, test_data, reader, writer, dtype, resample=True): 
        
                   test_data = test_data.astype(dtype) 
        
                   ndim = len(test_data.shape) - 1 
        
                   for p in TEST_NDARRAYS: 
        
                       output_ext = ".png" 
        
                       filepath = f"testfile_{ndim}d" 
        
                       saver = SaveImage( 
        
                           output_dir=self.test_dir, output_ext=output_ext, resample=resample, separate_folder=False, writer=writer 
        
                       ) 
        
                       saver(p(test_data), {"filename_or_obj": f"{filepath}.png", "spatial_shape": (6, 8)}) 
        
                       saved_path = os.path.join(self.test_dir, filepath + "_trans" + output_ext) 
        
                       self.assertTrue(os.path.exists(saved_path)) 
        
                       loader = LoadImage(reader=reader) 
        
                       data, meta = loader(saved_path) 
        
                       if meta["original_channel_dim"] == -1: 
        
                           _test_data = moveaxis(test_data, 0, -1) 
        
                       else: 
        
                           _test_data = test_data[0] 
        
                       assert_allclose(data, _test_data) 
        
               @parameterized.expand(itertools.product([PILReader, ITKReader], [PILWriter, ITKWriter])) 
        
               def test_2d(self, reader, writer): 
        
                   test_data = np.arange(48, dtype=np.uint8).reshape(1, 6, 8) 
        
                   self.png_rw(test_data, reader, writer, np.uint8) 
        
               @parameterized.expand(itertools.product([PILReader, ITKReader], ["monai.data.PILWriter", ITKWriter])) 
        
               def test_rgb(self, reader, writer): 
        
                   test_data = np.arange(48, dtype=np.uint8).reshape(3, 2, 8) 
        
                   self.png_rw(test_data, reader, writer, np.uint8, False)

the goal is to test whether the nrrd file created by monai's ITKWriter is readable by monai's NrrdReader for some basic (non multichannel?)cases.

…n header, it is assumed to be LPS and converted to RAS. If space is defined and not LPS, nothing is done to prevent wrong conversions. Signed-off-by: kbressem <[email protected]>

…ITKWriter can be loaded again. 2D and 3D files with no channels are tested Signed-off-by: kbressem <[email protected]>

Signed-off-by: kbressem <[email protected]>

…nto 4238-nrrd-reader

kbressem · 2022-05-13T15:03:28Z

the goal is to test whether the nrrd file created by monai's ITKWriter is readable by monai's NrrdReader for some basic (non multichannel?)cases.

I have added a TestLoadSaveNrrd class to test_image_rw.py which tests writing and reading 2d and 3d images. At the moment this only writes using ITKWriter, in the future an NrrdWriter, which is currently developed at MONAILabel might be used: Project-MONAI/MONAILabel#779

wyli · 2022-05-13T16:55:08Z

/build

jlvahldiek mentioned this pull request May 12, 2022

Changes to NRRD-Writer rAIdiance/MONAILabel#2

Merged

ahatamiz and others added 13 commits May 12, 2022 11:41

add swin_unetr model (#4074)

3495958

* add swin_unetr model Signed-off-by: kbressem <[email protected]>

Add GaussianSmooth as antialiasing filter in Resize (#4249)

a0b4100

Signed-off-by: Can Zhao <[email protected]> Signed-off-by: kbressem <[email protected]>

Implement NrrdReader and NrrdImage classes

90e2ac9

Signed-off-by: kbressem <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

9535e36

for more information, see https://pre-commit.ci Signed-off-by: kbressem <[email protected]>

run auto style fixes on image_reader.py

f4139f2

Signed-off-by: kbressem <[email protected]>

add NrrdReader to monai/data/__init__.py

6a7b9e8

Signed-off-by: kbressem <[email protected]>

markus-hinsche mentioned this pull request May 12, 2022

Add NrrdWriter for multi-channel arrays Project-MONAI/MONAILabel#779

Closed

kbressem added 9 commits May 12, 2022 19:54

Change the way spatial information is handled in NrrdReader

59aed3e

Signed-off-by: kbressem <[email protected]>

add tests for NrrdReader

9c59784

Signed-off-by: kbressem <[email protected]>

Add NrrdReader to list of possible readers for LoadImage

76329cb

Signed-off-by: kbressem <[email protected]>

autofix formating

46169c0

Signed-off-by: kbressem <[email protected]>

autofix formating

01e495a

Signed-off-by: kbressem <[email protected]>

change NrrdImage class to namedtuple and make flake8 happy

c051993

Signed-off-by: kbressem <[email protected]>

Add pynrrd to requirements

b7c3efb

Signed-off-by: kbressem <[email protected]>

correct typing for namedtumple

03b59ea

make flake8 happy Signed-off-by: kbressem <[email protected]>

Add pynrrd info to get_optional_config_values

7f8a4c1

Changed NrrdImage to dataclass Signed-off-by: kbressem <[email protected]>

kbressem changed the title ~~[WIP] Implement NrrdReader~~ Implement NrrdReader May 13, 2022

kbressem marked this pull request as ready for review May 13, 2022 09:57

Merge branch 'dev' into 4238-nrrd-reader

8f46bc5

kbressem added 2 commits May 13, 2022 12:21

exclude test_nrrd_reader.py from min tests

b885c23

Signed-off-by: kbressem <[email protected]>

Merge branch '4238-nrrd-reader' of http://github.com/kbressem/MONAI i…

6ad883b

…nto 4238-nrrd-reader

kbressem and others added 2 commits May 13, 2022 13:15

add pynrrd to config files

7294d34

Signed-off-by: kbressem <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

a5aa7e5

for more information, see https://pre-commit.ci

Nic-Ma approved these changes May 13, 2022

View reviewed changes

kbressem added 4 commits May 13, 2022 14:19

Change the way space is handled in the header. Now, if space is not i…

4330fd8

…n header, it is assumed to be LPS and converted to RAS. If space is defined and not LPS, nothing is done to prevent wrong conversions. Signed-off-by: kbressem <[email protected]>

add TestLoadSaveNrrd where it is tested if a nrrd file, created by …

ce9c883

…ITKWriter can be loaded again. 2D and 3D files with no channels are tested Signed-off-by: kbressem <[email protected]>

autofix format

7417d89

Signed-off-by: kbressem <[email protected]>

Merge branch '4238-nrrd-reader' of http://github.com/kbressem/MONAI i…

544394d

…nto 4238-nrrd-reader

Merge branch 'dev' into 4238-nrrd-reader

37a3f11

wyli approved these changes May 13, 2022

View reviewed changes

wyli enabled auto-merge (squash) May 13, 2022 16:56

wyli merged commit 0c243b9 into Project-MONAI:dev May 13, 2022

kbressem deleted the 4238-nrrd-reader branch May 13, 2022 17:49

markus-hinsche mentioned this pull request May 19, 2022

Add NrrdWriter for multi-channel arrays Project-MONAI/MONAILabel#793

Merged

Nic-Ma mentioned this pull request May 20, 2022

PR: NuClick Transforms Addition #4266

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement NrrdReader#4259

Implement NrrdReader#4259
wyli merged 32 commits intoProject-MONAI:devfrom
kbressem:4238-nrrd-reader

kbressem commented May 11, 2022 •

edited

Loading

Uh oh!

Nic-Ma commented May 13, 2022

Uh oh!

kbressem commented May 13, 2022

Uh oh!

Nic-Ma commented May 13, 2022 •

edited

Loading

Uh oh!

kbressem commented May 13, 2022

Uh oh!

wyli commented May 13, 2022

Uh oh!

kbressem commented May 13, 2022 •

edited

Loading

Uh oh!

wyli commented May 13, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

Conversation

kbressem commented May 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Status

Files changes/created

FIles created

Types of changes

Uh oh!

Nic-Ma commented May 13, 2022

Uh oh!

kbressem commented May 13, 2022

pandas was imported in:

nibabel was imported in:

Uh oh!

Nic-Ma commented May 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kbressem commented May 13, 2022

Uh oh!

wyli commented May 13, 2022

Uh oh!

kbressem commented May 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wyli commented May 13, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

kbressem commented May 11, 2022 •

edited

Loading

`pandas` was imported in:

`nibabel` was imported in:

Nic-Ma commented May 13, 2022 •

edited

Loading

kbressem commented May 13, 2022 •

edited

Loading