Skip to content

add WarmupCosineSchedule, UNETR & ViT Docstrings#2545

Merged
wyli merged 3 commits intoProject-MONAI:devfrom
ahatamiz:2537-unetr-v3
Jul 12, 2021
Merged

add WarmupCosineSchedule, UNETR & ViT Docstrings#2545
wyli merged 3 commits intoProject-MONAI:devfrom
ahatamiz:2537-unetr-v3

Conversation

@ahatamiz
Copy link
Copy Markdown
Contributor

@ahatamiz ahatamiz commented Jul 7, 2021 •
edited by wyli
Loading

Signed-off-by: ahatamizadeh [email protected]

Fixes #2537, fixes #2548

Description

Adding support for Warmup Cosine scheduler, used mainly in training transformer-based models. Unittests added.
Enhancing UNETR and ViT docstrings with examples.

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@ahatamiz
Copy link
Copy Markdown
Contributor Author

ahatamiz commented Jul 7, 2021

@wyli

@madil90
Copy link
Copy Markdown
Contributor

madil90 commented Jul 7, 2021

/build

2 similar comments
@madil90
Copy link
Copy Markdown
Contributor

madil90 commented Jul 7, 2021

/build

@madil90
Copy link
Copy Markdown
Contributor

madil90 commented Jul 8, 2021

/build

@ahatamiz
Copy link
Copy Markdown
Contributor Author

ahatamiz commented Jul 8, 2021

@wyli I am not sure why some checks take long time. It seems like some tests fail due to this reason.

@wyli
Copy link
Copy Markdown
Contributor

wyli commented Jul 8, 2021

@wyli I am not sure why some checks take long time. It seems like some tests fail due to this reason.

sure, thanks for letting me know. could you please help add some tests for this module "monai/optimizers/lr_scheduler.py"? I think there is currently no direct unit tests for it.

@madil90
Copy link
Copy Markdown
Contributor

madil90 commented Jul 8, 2021

/build

2 similar comments
@madil90
Copy link
Copy Markdown
Contributor

madil90 commented Jul 8, 2021

/build

@madil90
Copy link
Copy Markdown
Contributor

madil90 commented Jul 8, 2021

/build

@ahatamiz
Copy link
Copy Markdown
Contributor Author

ahatamiz commented Jul 8, 2021 •
edited
Loading

@wyli I am not sure why some checks take long time. It seems like some tests fail due to this reason.

sure, thanks for letting me know. could you please help add some tests for this module "monai/optimizers/lr_scheduler.py"? I think there is currently no direct unit tests for it.

Hi @wyli . Thanks. I think originally there were no unit tests for other schedulers in "monai/optimizers/lr_scheduler.py" . So that was the main reason I did not write unit tests for WarmupCosineSchedule. This implementation is based on HuggingFace. But I can write some tests for this one.

@madil90
Copy link
Copy Markdown
Contributor

madil90 commented Jul 8, 2021

/build

@ahatamiz ahatamiz changed the title add WarmupCosineSchedule add WarmupCosineSchedule, UNETR & ViT Docstrings Jul 12, 2021
@ahatamiz
Copy link
Copy Markdown
Contributor Author

ahatamiz commented Jul 12, 2021

/black

@ahatamiz
Copy link
Copy Markdown
Contributor Author

ahatamiz commented Jul 12, 2021

/black

wyli
wyli approved these changes Jul 12, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@wyli wyli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! would be great to have the other lr schedulers tested with the same testing approach.

@ahatamiz
Copy link
Copy Markdown
Contributor Author

ahatamiz commented Jul 12, 2021

Thanks! would be great to have the other lr schedulers tested with the same testing approach.

Hi @wyli , Thanks a lot. For now, I added tests for WarmupCosineSchedule since I did not write other schedulers in lr_schedulers. But shall try it later.

@wyli wyli merged commit 3902990 into Project-MONAI:dev Jul 12, 2021
@wyli wyli mentioned this pull request Jul 23, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wyli wyli wyli approved these changes

@rijobro rijobro Awaiting requested review from rijobro

@ericspod ericspod Awaiting requested review from ericspod ericspod is a code owner

@Nic-Ma Nic-Ma Awaiting requested review from Nic-Ma Nic-Ma is a code owner

@yiheng-wang-nv yiheng-wang-nv Awaiting requested review from yiheng-wang-nv

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

enhance unetr docstring with examples Add support for Transformer Optimization Scheduler

3 participants

@ahatamiz @madil90 @wyli