Skip to content

Allow None for checkpoint #17969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
tunz wants to merge 1 commit into from
Closed

Allow None for checkpoint #17969

tunz wants to merge 1 commit into from

Conversation

@tunz
Copy link
Contributor

@tunz tunz commented Mar 13, 2019 •
edited
Loading

Currently, we cannot run a checkpointed function with None argument.

out = torch.utils.checkpoint.checkpoint(run_fn, input_var, None)
  File "/home/tunz/anaconda3/envs/torchdev/lib/python3.7/site-packages/torch/utils/checkpoint.py", line 14, in detach_variable
    x = inp.detach()
AttributeError: 'NoneType' object has no attribute 'detach'

This PR makes checkpoint function to safely handle None argument.

ezyang
ezyang approved these changes Mar 15, 2019
View reviewed changes
Copy link
Contributor

@ezyang ezyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okey dokey

facebook-github-bot
facebook-github-bot reviewed Mar 15, 2019
View reviewed changes
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ezyang is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

oliland
oliland reviewed Jul 27, 2021
View reviewed changes
torch/utils/checkpoint.py

def check_backward_validity(inputs):
if not any(inp.requires_grad for inp in inputs):
if not any(inp.requires_grad for inp in inputs if isinstance(inp, torch.Tensor)):
Copy link

@oliland oliland Jul 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm receiving the warning when I pass in a list of tensors which have gradients. Propose:

if not any(inp.requires_grad for inp in inputs if inputs is not None):

Copy link
Contributor

@ezyang ezyang Jul 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oliland send us a PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ezyang ezyang ezyang approved these changes

@facebook-github-bot facebook-github-bot facebook-github-bot left review comments

+1 more reviewer

@oliland oliland oliland left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

open source

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@tunz @ezyang @oliland @facebook-github-bot @pytorchbot