Skip to content

Autorunner fixes2#5562

Merged
wyli merged 21 commits intoProject-MONAI:devfrom
myron:autorunner2
Nov 23, 2022
Merged

Autorunner fixes2#5562
wyli merged 21 commits intoProject-MONAI:devfrom
myron:autorunner2

Conversation

@myron
Copy link
Copy Markdown
Collaborator

@myron myron commented Nov 22, 2022

This is the same PR as in #5523

but removing changes to default ensemble method (AlgoEnsembleBestByFold), and keeping it as is for now, since it trigger errors with some unit tests. the default ensemble method update can be done in a separate PR.

Description

A few sentences describing the changes proposed in this pull request.

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

myron and others added 17 commits November 18, 2022 19:55
@myron
AutoRunner small updates
0e61d03 
Signed-off-by: myron <[email protected]>
@myron
flake
c9dbeca 
Signed-off-by: myron <[email protected]>
@pre-commit-ci @myron
[pre-commit.ci] auto fixes from pre-commit.com hooks
d718338 
for more information, see https://pre-commit.ci

Signed-off-by: myron <[email protected]>
@myron
small
b8ff07d 
Signed-off-by: myron <[email protected]>
@myron
small
f69e8ba 
Signed-off-by: myron <[email protected]>
@myron
save ensembled im as uint8
f4c8457 
Signed-off-by: myron <[email protected]>
@myron
saving ensemble result in uint8
dec4796 
Signed-off-by: myron <[email protected]>
@myron
fix setting infer_files
f1650d0 
Signed-off-by: myron <[email protected]>
@pre-commit-ci @myron
[pre-commit.ci] auto fixes from pre-commit.com hooks
8a65470 
for more information, see https://pre-commit.ci

Signed-off-by: myron <[email protected]>
@myron
handling missing testing
af00e5a 
Signed-off-by: myron <[email protected]>
@myron
predict
04f7643 
Signed-off-by: myron <[email protected]>
@myron
logger
0dbad8b 
Signed-off-by: myron <[email protected]>
@myron
Merge remote-tracking branch 'upstream/dev' into autorunner
7c0074c
@myron
preserve user settings
062195f 
Signed-off-by: myron <[email protected]>
@myron
Merge remote-tracking branch 'upstream/dev' into autorunner
064eaf0
@myron
Merge branch 'dev' into autorunner2
5120a2c
@myron
revert back to AlgoEnsembleBestN default ensemble for now
21a2201 
Signed-off-by: myron <[email protected]>
@myron
Copy link
Copy Markdown
Collaborator Author

myron commented Nov 22, 2022

@wyli here is the same PR as in #5523
previously you had comment if the a new parameter "algo" should be placed at the very end or not. I think it should be in the front since it will a very commonly used parameter. Can you please comment if you still want "algo" at the end or not, and I can update it. thank you

#5523 (comment)

@wyli
Copy link
Copy Markdown
Contributor

wyli commented Nov 22, 2022

sure, as long as it's not accidentally omitted...

@myron
Copy link
Copy Markdown
Collaborator Author

myron commented Nov 22, 2022

There is one integration test, I' unable to test in "test_integration_autorunner.py" -> test_autorunner_hpo(self) , test for HPO. On my local machine it runs forever, and get stuck without any progress (no CPU or GPU usage). Same behavior on old dev branch and on current (so it's unrelated to this PR). So I've test everything except for HPO test there.
Perhaps we can merge it, as I don't think the changes in this PR should affect that test.

@myron myron marked this pull request as ready for review November 22, 2022 23:40
@myron myron changed the title Autorunner2 fixes Autorunner fixes2 Nov 22, 2022
@myron myron requested review from mingxin-zheng and wyli November 22, 2022 23:43
@mingxin-zheng
Copy link
Copy Markdown
Contributor

mingxin-zheng commented Nov 23, 2022

The previous failed integration test is working okay now on a local machine.

autorunner2_log.txt

mingxin-zheng
mingxin-zheng reviewed Nov 23, 2022
View reviewed changes
@myron
format
08f9a16 
Signed-off-by: myron <[email protected]>
@myron myron requested a review from mingxin-zheng November 23, 2022 02:34
@myron
Copy link
Copy Markdown
Collaborator Author

myron commented Nov 23, 2022

I ran the integration test successfully too (test_integration_autorunner.py). The issue was that it does not run correctly on my local 8gpu machine, because a toy dateset there is too small. It runs fine on 2gpu system.

@wyli please check if you have any comments. otherwise everything seems correct.

@wyli wyli mentioned this pull request Nov 23, 2022
Closed
@wyli
Copy link
Copy Markdown
Contributor

wyli commented Nov 23, 2022

/build

@wyli wyli enabled auto-merge (squash) November 23, 2022 16:10
@wyli wyli merged commit c8cdb5f into Project-MONAI:dev Nov 23, 2022
wyli pushed a commit to bhashemian/MONAI that referenced this pull request Nov 23, 2022
@myron @wyli
Autorunner fixes2 (Project-MONAI#5562)
06b0ee8 
This is the same PR as in
Project-MONAI#5523

but removing changes to default ensemble method
(AlgoEnsembleBestByFold), and keeping it as is for now, since it triggers
errors with some unit tests. the default ensemble method update can be
done in a separate PR.


### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [ ] New tests added to cover the changes.
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [ ] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [ ] In-line docstrings updated.
- [ ] Documentation updated, tested `make html` command in the `docs/`
folder.

Signed-off-by: myron <[email protected]>
@wyli wyli mentioned this pull request Nov 24, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mingxin-zheng mingxin-zheng mingxin-zheng approved these changes

@wyli wyli Awaiting requested review from wyli

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

Auto3D Seg framework [internal ongoin...

Development

Successfully merging this pull request may close these issues.

3 participants

@myron @wyli @mingxin-zheng