Skip to content

Autograd bugfixes and improvements #1506

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
apaszke merged 8 commits into from
May 10, 2017
Merged

Autograd bugfixes and improvements #1506

apaszke merged 8 commits into from
May 10, 2017

Conversation

@apaszke
Copy link
Contributor

@apaszke apaszke commented May 7, 2017 •
edited
Loading

  • Turns out we can't really take a short path for volatile inputs. The flags aren't propagated correctly in case a non-volatile input is modified in an in-place op with a volatile Variable -- e.g. x[2] = y, where x is not volatile, and y is. Also, volatile ops didn't mark inputs as dirty.
  • Replace deprecated retain_variables with retain_graph
  • Add new flags to Variable.backward (and delegate to torch.autograd.backward to simplify the code)
  • Minor fix for Prod backward (grad_input wasn't volatile, even if grad_output was)
  • If a Variable has non-volatile gradient (i.e. when higher order grads are used), model.zero_grad() now replaces the grad with a new zero-filled Variable. This is useful if one wants to zero grad but used the old Variable to compute sth. Additionally assignments to .grad are now allowed (new grad is checked for its device, type and size).
  • Exposed .variable attribute from AccumulateGrad nodes (cc @szagoruyko)

@apaszke apaszke force-pushed the autograd_fixes branch 3 times, most recently from 6207eba to a0052ec Compare May 8, 2017 21:56
@apaszke apaszke closed this May 8, 2017
@apaszke apaszke reopened this May 8, 2017
@szagoruyko szagoruyko mentioned this pull request May 9, 2017
Merged
@caogang caogang mentioned this pull request May 9, 2017
Merged
4 tasks
@apaszke apaszke merged commit 5bb1348 into master May 10, 2017
@apaszke apaszke deleted the autograd_fixes branch May 10, 2017 14:43
@caogang caogang mentioned this pull request May 11, 2017
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@colesbury colesbury colesbury approved these changes

Assignees

No one assigned

Labels

open source

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@apaszke @colesbury @ezyang