561 497 Provide metrics base APIs#2291
Merged
Nic-Ma merged 32 commits intoProject-MONAI:feature/561-metrics-apifrom Jun 6, 2021
Merged
561 497 Provide metrics base APIs#2291Nic-Ma merged 32 commits intoProject-MONAI:feature/561-metrics-apifrom
Nic-Ma merged 32 commits intoProject-MONAI:feature/561-metrics-apifrom
Conversation
merge master
merge master
merge master
Signed-off-by: Nic Ma <[email protected]>
Signed-off-by: Nic Ma <[email protected]>
Signed-off-by: Nic Ma <[email protected]>
Signed-off-by: Nic Ma <[email protected]>
Signed-off-by: Nic Ma <[email protected]>
Signed-off-by: Nic Ma <[email protected]>
Contributor
Author
|
/black |
Nic-Ma
changed the title
Signed-off-by: monai-bot <[email protected]>
Contributor
Author
|
Hi @vfdev-5 , This PR added the metric base class and support a list of tensors. Tried to align with your new metric API design. Thanks. |
vfdev-5
reviewed
Jun 2, 2021
Signed-off-by: Nic Ma <[email protected]>
Signed-off-by: Nic Ma <[email protected]>
Signed-off-by: Nic Ma <[email protected]>
Contributor
Author
|
/black |
Signed-off-by: Nic Ma <[email protected]>
Nic-Ma
force-pushed
the
561-provide-metrics-base
branch
from
9f3945f to
1372133
Compare
Signed-off-by: Nic Ma <[email protected]>
Signed-off-by: Nic Ma <[email protected]>
Contributor
Author
|
/black |
Contributor
Author
|
/integration-test |
Contributor
Author
Signed-off-by: Nic Ma <[email protected]>
Contributor
Author
|
/black |
Contributor
yiheng-wang-nv
left a comment
yiheng-wang-nv
left a comment
There was a problem hiding this comment.
Thanks for the changes, I left some comments above.
wyli
reviewed
Jun 3, 2021
Contributor
wyli
left a comment
wyli
left a comment
There was a problem hiding this comment.
thanks, I put some initial comments, the metric base classes may need more work, let me check the evaluation working group discussions and think about this
Signed-off-by: Nic Ma <[email protected]>
Signed-off-by: Nic Ma <[email protected]>
Signed-off-by: Nic Ma <[email protected]>
Signed-off-by: Nic Ma <[email protected]>
Contributor
Author
|
/black |
Signed-off-by: Nic Ma <[email protected]>
Signed-off-by: Nic Ma <[email protected]>
Contributor
Author
|
/black |
Contributor
Author
|
As this is a very big feature update, I created a new branch: feature/561-metrics-api in the Project-MONAI repo. Thanks. |
Nic-Ma
added a commit
that referenced
this pull request
Jun 11, 2021
* 561 497 Provide metrics base APIs (#2291) * [DLMED] add metric base class Signed-off-by: Nic Ma <[email protected]> * [DLMED] update meandice and auc Signed-off-by: Nic Ma <[email protected]> * [DLMED] extract reduce API Signed-off-by: Nic Ma <[email protected]> * [DLMED] update regression metrics Signed-off-by: Nic Ma <[email protected]> * [DLMED] update all the other metrics and enhance unit tests Signed-off-by: Nic Ma <[email protected]> * [DLMED] add doc-strings and update unit tests Signed-off-by: Nic Ma <[email protected]> * [MONAI] python code formatting Signed-off-by: monai-bot <[email protected]> * [DLMED] fix flake8 issue Signed-off-by: Nic Ma <[email protected]> * [DLMED] fix pytype issue Signed-off-by: Nic Ma <[email protected]> * [DLMED] fix all the mypy issues Signed-off-by: Nic Ma <[email protected]> * [DLMED] fix integration test Signed-off-by: Nic Ma <[email protected]> * [MONAI] python code formatting Signed-off-by: monai-bot <[email protected]> * [DLMED] fix flake8 issue Signed-off-by: Nic Ma <[email protected]> * [DLMED] fix pytype issue Signed-off-by: Nic Ma <[email protected]> * [DLMED] add more sanity check Signed-off-by: Nic Ma <[email protected]> * [DLMED] update according to comments Signed-off-by: Nic Ma <[email protected]> * [DLMED] update according to Yiheng's comments Signed-off-by: Nic Ma <[email protected]> * [DLMED] update according to Wenqi's comments Signed-off-by: Nic Ma <[email protected]> * [DLMED] change to "_compute()" and "aggregate()" Signed-off-by: Nic Ma <[email protected]> * [DLMED] add compute_list() Signed-off-by: Nic Ma <[email protected]> * [DLMED] fix flake8 issue Signed-off-by: Nic Ma <[email protected]> * [DLMED] fix flake8 issue Signed-off-by: Nic Ma <[email protected]> Co-authored-by: monai-bot <[email protected]> Co-authored-by: Yiheng Wang <[email protected]> * [DLMED] update according to comments 1. re-define base classes in 3 levels to make it more clear and flexible to extend 2. added self-contained variables in metrics to provide easier API 3. added reset() and sync() logic 4. added distributed data parallel logic for all the metrics Signed-off-by: Nic Ma <[email protected]> * [MONAI] python code formatting Signed-off-by: monai-bot <[email protected]> * [DLMED] fix flake8 issue Signed-off-by: Nic Ma <[email protected]> * [MONAI] python code formatting Signed-off-by: monai-bot <[email protected]> * [DLMED] optimize base classes Signed-off-by: Nic Ma <[email protected]> * [DLMED] update according to comments Signed-off-by: Nic Ma <[email protected]> * [DLMED] simplify integration tests to use new APIs Signed-off-by: Nic Ma <[email protected]> * [DLMED] update based on comments Signed-off-by: Nic Ma <[email protected]> * [DLMED] add synced=False in add() Signed-off-by: Nic Ma <[email protected]> * [DLMED] update according to comments Signed-off-by: Nic Ma <[email protected]> * full ci/cd for feature branches Signed-off-by: Wenqi Li <[email protected]> * [DLMED] update aggregate() Signed-off-by: Nic Ma <[email protected]> Co-authored-by: monai-bot <[email protected]> Co-authored-by: Yiheng Wang <[email protected]> Co-authored-by: Wenqi Li <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR added base class for the metrics.
And add support for
list of channel-first Tensor.It's a non-breaking change.
Status
Ready
Types of changes
./runtests.sh -f -u --net --coverage../runtests.sh --quick --unittests.make htmlcommand in thedocs/folder.