Skip to content

4094 Enhance ckpt_export to save config files#4159

Merged
Nic-Ma merged 13 commits intoProject-MONAI:devfrom
Nic-Ma:4094-enhance-ckpt-export
Apr 26, 2022
Merged

4094 Enhance ckpt_export to save config files#4159
Nic-Ma merged 13 commits intoProject-MONAI:devfrom
Nic-Ma:4094-enhance-ckpt-export

Conversation

@Nic-Ma
Copy link
Copy Markdown
Contributor

@Nic-Ma Nic-Ma commented Apr 22, 2022

Fixes #4094 .

Description

This PR enhanced the ckpt_export script to save config files separately.
And also added unit tests to load the TorchScript model with meta, configs.

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@Nic-Ma Nic-Ma mentioned this pull request Apr 22, 2022
Closed
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Apr 22, 2022

/black

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Apr 22, 2022

/build

@Nic-Ma Nic-Ma requested review from ericspod and rijobro April 22, 2022 04:28
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Apr 22, 2022

/build

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Apr 22, 2022

/build

ericspod
ericspod requested changes Apr 22, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@ericspod ericspod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can preserve the extensions of files we're saving with the minor change I'd suggest unless I'm missing something about why this was there originally.

@Nic-Ma Nic-Ma requested a review from ericspod April 22, 2022 16:30
@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Apr 22, 2022

/black

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Apr 22, 2022

/build

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Apr 22, 2022

Hi @ericspod ,

Thanks for your review, I updated the PR according to your comments.
Could you please help review it again?

Thanks in advance.

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Apr 22, 2022

/build

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Apr 23, 2022

/build

1 similar comment
@pxLi
Copy link
Copy Markdown
Contributor

pxLi commented Apr 24, 2022

/build

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Apr 25, 2022

/build

@Nic-Ma Nic-Ma merged commit e688989 into Project-MONAI:dev Apr 26, 2022
wyli added a commit that referenced this pull request Apr 26, 2022
@wyli @yiheng-wang-nv
@pre-commit-ci @Nic-Ma @dongyang0122 @bhashemian @pxLi @ericspod
sync metatensor branch with dev (#4184)
724f73f 
* 4095 Add bundle download (#4114)

* draft download

Signed-off-by: Yiheng Wang <[email protected]>

* update bundle download

Signed-off-by: Yiheng Wang <[email protected]>

* add url and load

Signed-off-by: Yiheng Wang <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rename args and remove a few places

Signed-off-by: Yiheng Wang <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix flake8 issue

Signed-off-by: Yiheng Wang <[email protected]>

* enhance with reviews

Signed-off-by: Yiheng Wang <[email protected]>

* add instantiate for load

Signed-off-by: Yiheng Wang <[email protected]>

* fix black error

Signed-off-by: Yiheng Wang <[email protected]>

* add unittest

Signed-off-by: Yiheng Wang <[email protected]>

* add load to docs

Signed-off-by: Yiheng Wang <[email protected]>

* add skip

Signed-off-by: Yiheng Wang <[email protected]>

* add schemaerror

Signed-off-by: Yiheng Wang <[email protected]>

* fix partial places

Signed-off-by: Yiheng Wang <[email protected]>

* download zip bundle

Signed-off-by: Yiheng Wang <[email protected]>

* [DLMED] restore Exception for test

Signed-off-by: Nic Ma <[email protected]>

* update ts features

Signed-off-by: Yiheng Wang <[email protected]>

* add config_files test case

Signed-off-by: Yiheng Wang <[email protected]>

* enhance docstring example for args_file

Signed-off-by: Yiheng Wang <[email protected]>

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Nic Ma <[email protected]>

* Disable pylint error and fix CI tests of new tifffile (#4162)

* workaround

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] fix tifffile issue

Signed-off-by: Nic Ma <[email protected]>

* Fixed an error in DiNTS model implementation and enabled act and norm layer options (#4157)

* fixed a bug

Signed-off-by: dongy <[email protected]>

* autofix

Signed-off-by: dongy <[email protected]>

* update test case

Signed-off-by: dongy <[email protected]>

Co-authored-by: dongy <[email protected]>

* Split transform (#4153)

* Redesign whole slide image reading (#4107)

* Redesign BaseWSIReader,  WSIReader, CuCIMWSIReader

Signed-off-by: Behrooz <[email protected]>

* Add unittests for WSIReader

Signed-off-by: Behrooz <[email protected]>

* Add image mode for output validation

Signed-off-by: Behrooz <[email protected]>

* Update docs

Signed-off-by: Behrooz <[email protected]>

* Update references to new WSIReader

Signed-off-by: Behrooz <[email protected]>

* Remove legacy WSIReader

Signed-off-by: Behrooz <[email protected]>

* Update unittests

Signed-off-by: Behrooz <[email protected]>

* Update docs

Signed-off-by: Behrooz <[email protected]>

* sort imports

Signed-off-by: Behrooz <[email protected]>

* Clean up imports

Signed-off-by: Behrooz <[email protected]>

* Update docstrings

Signed-off-by: Behrooz <[email protected]>

* Update docs and docstrings

Signed-off-by: Behrooz <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix a typo

Signed-off-by: Behrooz <[email protected]>

* Remove redundant checking

Signed-off-by: Behrooz <[email protected]>

* Update read and other methods

Signed-off-by: Behrooz <[email protected]>

* Update wsireader to support multi image and update docstrings

Signed-off-by: Behrooz <[email protected]>

* Make workaround for CuImage objects

Signed-off-by: Behrooz <[email protected]>

* Add unittests for multi image reading

Signed-off-by: Behrooz <[email protected]>

* Update a note about cucim

Signed-off-by: Behrooz <[email protected]>

* Update type hints and docstrings

Signed-off-by: Behrooz <[email protected]>

* Implement Split transform

Signed-off-by: Behrooz <[email protected]>

* Add unittests

Signed-off-by: Behrooz <[email protected]>

* Update formatting

Signed-off-by: Behrooz <[email protected]>

* Implement SplitDict

Signed-off-by: Behrooz <[email protected]>

* Add unittests for SplitDict

Signed-off-by: Behrooz <[email protected]>

* Add docs

Signed-off-by: Behrooz <[email protected]>

* Remove images from docs

Signed-off-by: Behrooz <[email protected]>

* Address all comments

Signed-off-by: Behrooz <[email protected]>

* Add example and size check

Signed-off-by: Behrooz <[email protected]>

* Update docs

Signed-off-by: Behrooz <[email protected]>

* Revert references to new wsireader

Signed-off-by: Behrooz <[email protected]>

* Add missing comma

Signed-off-by: Behrooz <[email protected]>

* fix bundle download test issue (#4169)

Signed-off-by: Yiheng Wang <[email protected]>

* 4094 Enhance `ckpt_export` to save config files (#4159)

* [DLMED] enhance checkpoint export

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update according to comments

Signed-off-by: Nic Ma <[email protected]>

* Move RGB/RGBA checks to base class (#4171)

Signed-off-by: Behrooz <[email protected]>

Co-authored-by: Nic Ma <[email protected]>

* [CICD] To support temp dgx runner (#4175)

* Support new temp dgx runner

Signed-off-by: Peixin Li <[email protected]>

* atol 1e-5

Signed-off-by: Wenqi Li <[email protected]>

Co-authored-by: Wenqi Li <[email protected]>

* Test fix for AMP kwargs (#4178)

Signed-off-by: Eric Kerfoot <[email protected]>

Co-authored-by: Yiheng Wang <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Nic Ma <[email protected]>
Co-authored-by: dongyang0122 <[email protected]>
Co-authored-by: dongy <[email protected]>
Co-authored-by: Behrooz <[email protected]>
Co-authored-by: Peixin <[email protected]>
Co-authored-by: Eric Kerfoot <[email protected]>
Can-Zhao pushed a commit to Can-Zhao/MONAI that referenced this pull request May 10, 2022
@Nic-Ma @Can-Zhao
4094 Enhance ckpt_export to save config files (Project-MONAI#4159)
203be62 
* [DLMED] enhance checkpoint export

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update according to comments

Signed-off-by: Nic Ma <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ericspod ericspod ericspod approved these changes

@rijobro rijobro Awaiting requested review from rijobro

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Checkpoint Export Functionality Should Require Bundle Directory Path Only

3 participants

@Nic-Ma @pxLi @ericspod