Skip to content

2595 Add support for the set_to_none arg in optimizer#2596

Merged
Nic-Ma merged 10 commits intoProject-MONAI:devfrom
Nic-Ma:optim-add-set-to-none
Jul 14, 2021
Merged

2595 Add support for the set_to_none arg in optimizer#2596
Nic-Ma merged 10 commits intoProject-MONAI:devfrom
Nic-Ma:optim-add-set-to-none

Conversation

@Nic-Ma
Copy link
Copy Markdown
Contributor

@Nic-Ma Nic-Ma commented Jul 14, 2021

Fixes #2595 .

Description

This PR added support to set the set_to_none arg for optimizer in our workflows.

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Jul 14, 2021

/black

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Jul 14, 2021

/integration-test

@Nic-Ma Nic-Ma enabled auto-merge (squash) July 14, 2021 08:34
@wyli
Copy link
Copy Markdown
Contributor

wyli commented Jul 14, 2021

Hi @Nic-Ma, there's a change conflict here in case you didn't see it

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Jul 14, 2021

/black

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Jul 14, 2021

Hi @wyli ,

Seems our PyTorch 1.7 CI test is not running with official PyTorch 1.7(our version is 1.7.0a0+8deb4fe)?
There is no set_to_none arg in it.

Thanks.

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Jul 14, 2021

Hi @wyli ,

Should I change this line to use PyTorch 1.7.1 version instead?
https://github.com/Project-MONAI/MONAI/blob/dev/.github/workflows/pythonapp.yml#L224
Maybe let me change all the other items to install explicit PyTorch version instead of the PyTorch in the docker?

Thanks.

@wyli
Copy link
Copy Markdown
Contributor

wyli commented Jul 14, 2021

Hi @wyli ,

Should I change this line to use PyTorch 1.7.1 version instead?
https://github.com/Project-MONAI/MONAI/blob/dev/.github/workflows/pythonapp.yml#L224

Thanks.

that is supposed to test nvidia's pytorch docker image, changing that is a separate topic.
Now this PR doesn't work with pytorch:20.09, maybe we can change the code to fix it... perhaps get_torch_version_tuple() <= (1, 7)?

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Jul 14, 2021

Hi @wyli ,

I want to double confirm the root cause is that the docker customized PyTorch 1.6 to make a 1.7a version in it, right?
If we change to <=1.7, users with PyTorch 1.7.0 can't use this feature anymore:
https://github.com/pytorch/pytorch/blob/v1.7.0/torch/optim/optimizer.py#L167
But maybe not a big problem?

Thanks.

@wyli
Copy link
Copy Markdown
Contributor

wyli commented Jul 14, 2021

Hi @wyli ,

I want to double confirm the root cause is that the docker customized PyTorch 1.6 to make a 1.7a version in it, right?
If we change to <=1.7, users with PyTorch 1.7.0 can't use this feature anymore:
https://github.com/pytorch/pytorch/blob/v1.7.0/torch/optim/optimizer.py#L167
But maybe not a big problem?

Thanks.

yes, 1.7a is actually some 1.6.x, btw looks like this flag is accurate:

PT_BEFORE_1_7 = torch.__version__ != "1.7.0" and version_leq(torch.__version__, "1.7.0")

if it doesn't work we change to "<=1.7"

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Jul 14, 2021

/black

@wyli
Copy link
Copy Markdown
Contributor

wyli commented Jul 14, 2021

I think if there's no cyclic import, we can from monai.utils.module import PT_BEFORE_1_7 ?

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Jul 14, 2021

I think if there's no cyclic import, we can from monai.utils.module import PT_BEFORE_1_7 ?

Sure, thanks for the suggestion, I didn't notice this global variable.

Thanks.

@Nic-Ma
Copy link
Copy Markdown
Contributor Author

Nic-Ma commented Jul 14, 2021

/black

@Nic-Ma Nic-Ma merged commit 66b226b into Project-MONAI:dev Jul 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add set_to_none=False for optimizer in workflow

4 participants