Make dropout condition on training. #29436
Conversation
facebook-github-bot
added
the
oncall: jit
ssnl
left a comment
ssnl
left a comment
There was a problem hiding this comment.
wow nice catch! pretty big bug I'd say though.....
test/test_jit.py
Outdated
There was a problem hiding this comment.
did you mean to use self.dropout here?
ailzhang
force-pushed
the
fix_dropout_grad
branch
from
b6b5ec1 to
f784dba
Compare
|
Thanks for fixing |
ailzhang
force-pushed
the
fix_dropout_grad
branch
from
f784dba to
5ead372
Compare
test/test_jit.py
Outdated
| FileCheck().check("aten::bernoulli_").run(scripted_module.graph_for(X)) | ||
| FileCheck().check("aten::bernoulli_").run(scripted_func.graph_for(X)) | ||
|
|
||
| X = torch.randn(M, M, requires_grad=False) |
There was a problem hiding this comment.
The condition here is a bit weird. It should not depend on whether x is requiring grad, but on whether m is in training mode, right?
There was a problem hiding this comment.
The torchscript executor only bothers to split out symbolic gradients when the inputs require_grad, which is probably why these flags are getting set here.
There was a problem hiding this comment.
My question was more about why there is not a .eval() here to switch the scripted module to eval mode. Unless .graph_for auto selected train / eval mode based on input requires_grad which sounds wrong to me.
There was a problem hiding this comment.
@ssnl @zdevito @bddppq Thanks for the comments! To make the testcase a bit more clear, I've added a comment here to explain the behavior. https://github.com/pytorch/pytorch/pull/29436/files#diff-78aab5b4217aeaed920efab0c5f5f5c8R2202
Please let me know if that looks good! Thanks!
ailzhang
force-pushed
the
fix_dropout_grad
branch
from
11c11f3 to
3f760a0
Compare
facebook-github-bot
left a comment
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ailzhang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
facebook-github-bot
left a comment
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ailzhang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
6 participants
No description provided.