Skip to content

Auto3D Task Module 1 (DataAnalyzer) for #4743#4765

Merged
wyli merged 177 commits intoProject-MONAI:devfrom
mingxin-zheng:auto3d-1.0
Aug 26, 2022
Merged

Auto3D Task Module 1 (DataAnalyzer) for #4743#4765
wyli merged 177 commits intoProject-MONAI:devfrom
mingxin-zheng:auto3d-1.0

Conversation

@mingxin-zheng
Copy link
Copy Markdown
Contributor

@mingxin-zheng mingxin-zheng commented Jul 26, 2022

Description

Implemented a DataAnalyzer class to encapsulate data analysis module. As the beginning part of the auto3d/automl pipeline, the module shall find data and label from user inputs and generate a summary (dictionary) of data stats. The summary includes
- file names, list, number of files;
- dataset summary (basic information, image dimensions, number of classes, etc.);
- individual data information (spacing, image size, number and size of the regions, etc.).
The summary can be exported as a YAML file and a dictionary variable for use in Python

Example Usage:

from monai.apps.auto3d.data_analyzer import DataAnalyzer

datalist = {
    "testing": [{"image": "image_003.nii.gz"}],
    "training": [
        {"fold": 0, "image": "image_001.nii.gz", "label": "label_001.nii.gz"},
        {"fold": 0, "image": "image_002.nii.gz", "label": "label_002.nii.gz"},
        {"fold": 1, "image": "image_001.nii.gz", "label": "label_001.nii.gz"},
        {"fold": 1, "image": "image_004.nii.gz", "label": "label_004.nii.gz"},
    ],
}

dataroot = '/datasets' # the directory where you have the image files (in this example we're using nii.gz)
analyser = DataAnalyzer(datalist, dataroot)
datastat = analyser.get_all_case_stats() # it will also generate a data_stats.yaml that saves the stats

Status

Ready for Review
Reference issue #4743.

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • New tests added to cover the changes.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@wyli
Copy link
Copy Markdown
Contributor

wyli commented Aug 2, 2022

@mingxin-zheng
Copy link
Copy Markdown
Contributor Author

Thanks @wyli . Still ramping up myself on the the MONAI style coding. I will fix those.

@Nic-Ma
Copy link
Copy Markdown
Contributor

Nic-Ma commented Aug 3, 2022

Hi @mingxin-zheng @dongyang0122 @wyli ,

I think the current pipeline can only work for 3D segmentation, should we call the module "auto3d_seg" or "auto_seg_3d"?

Thanks.

Copy link
Copy Markdown
Contributor

@Nic-Ma Nic-Ma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mingxin-zheng ,

Thanks for the contribution about this highlight feature.
I put some comments for the high level review, haven't checked the computation details.
May need @dongyang0122 @wyli to also help review it.

Thanks in advance.

Signed-off-by: Mingxin Zheng <[email protected]>
@mingxin-zheng
Copy link
Copy Markdown
Contributor Author

@mingxin-zheng
Copy link
Copy Markdown
Contributor Author

Hi @mingxin-zheng I put some doc page updates here mingxin-zheng#1 please add them to the PR so that the docstrings will be part of the doc website

Thanks @wyli ! I have merged it to the PR code.

Signed-off-by: Mingxin Zheng <[email protected]>
Copy link
Copy Markdown
Contributor

@Nic-Ma Nic-Ma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick update.
Looks good to me now.
@wyli Do you have any other comments? Especially the code folder structure.

Thanks.

@wyli
Copy link
Copy Markdown
Contributor

wyli commented Aug 26, 2022

cool, it looks good to me as well.

@wyli wyli enabled auto-merge (squash) August 26, 2022 15:55
@wyli wyli disabled auto-merge August 26, 2022 15:56
@wyli
Copy link
Copy Markdown
Contributor

wyli commented Aug 26, 2022

/build

@wyli wyli enabled auto-merge (squash) August 26, 2022 15:57
@wyli wyli changed the title [WIP] Auto3D Task Module 1 (DataAnalyzer) for #4743 Auto3D Task Module 1 (DataAnalyzer) for #4743 Aug 26, 2022
Signed-off-by: Wenqi Li <[email protected]>
@wyli wyli disabled auto-merge August 26, 2022 16:22
@wyli wyli enabled auto-merge (squash) August 26, 2022 16:22
@wyli wyli disabled auto-merge August 26, 2022 16:22
@wyli wyli enabled auto-merge (squash) August 26, 2022 17:18
@wyli
Copy link
Copy Markdown
Contributor

wyli commented Aug 26, 2022

/build

@wyli wyli disabled auto-merge August 26, 2022 17:41
@wyli wyli enabled auto-merge (squash) August 26, 2022 17:41
@wyli wyli disabled auto-merge August 26, 2022 18:51
@wyli wyli enabled auto-merge (squash) August 26, 2022 18:51
@wyli
Copy link
Copy Markdown
Contributor

wyli commented Aug 26, 2022

/build

Signed-off-by: Wenqi Li <[email protected]>
@wyli
Copy link
Copy Markdown
Contributor

wyli commented Aug 26, 2022

/build

@wyli wyli disabled auto-merge August 26, 2022 19:22
@wyli wyli enabled auto-merge (squash) August 26, 2022 19:22
@wyli wyli merged commit 15ac45d into Project-MONAI:dev Aug 26, 2022
@mingxin-zheng mingxin-zheng deleted the auto3d-1.0 branch August 29, 2022 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

9 participants