-
Notifications
You must be signed in to change notification settings - Fork 26.3k
Closed
Labels
featureA request for a proper, new feature.A request for a proper, new feature.triagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Description
feature request description
The NLP community is shifting from LSTMs to Transformers for a number of NLP tasks.
This would be great if we could have a packed standard Transformer implementation in the 'nn' package, i.e. a nn.Transformer, just like we have a nn.LSTM.
Code example
fairseq-py has a tested implementation of the Transformer encoder and decoder (for LM and NMT) https://github.com/pytorch/fairseq/blob/master/fairseq/models/transformer.py. Wrapping this up in 'nn' should be the most straightforward approach to reimplementing this model.
Thanks!
sksq96, nyxjemk, TheShadow29, skurzhanskyi, tshrjn and 9 more
Metadata
Metadata
Assignees
Labels
featureA request for a proper, new feature.A request for a proper, new feature.triagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module