-
Notifications
You must be signed in to change notification settings - Fork 26.3k
Open
Labels
module: docsRelated to our documentation, both in docs/ and docblocksRelated to our documentation, both in docs/ and docblocksmodule: nnRelated to torch.nnRelated to torch.nntriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Description
📚 Documentation
It likely should be mentioned that the attn_mask argument of MHA is an additive mask (-inf masks values), rather than the standard multiplicative mask (0 masks values). Perhaps even enforce a value check (all values should be 0 / -inf ?, otherwise print warning?)
Metadata
Metadata
Assignees
Labels
module: docsRelated to our documentation, both in docs/ and docblocksRelated to our documentation, both in docs/ and docblocksmodule: nnRelated to torch.nnRelated to torch.nntriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module