Skip to content

[feature request] nn.Transformer #10459

@aconneau

Description

@aconneau

feature request description

The NLP community is shifting from LSTMs to Transformers for a number of NLP tasks.
This would be great if we could have a packed standard Transformer implementation in the 'nn' package, i.e. a nn.Transformer, just like we have a nn.LSTM.

Code example

fairseq-py has a tested implementation of the Transformer encoder and decoder (for LM and NMT) https://github.com/pytorch/fairseq/blob/master/fairseq/models/transformer.py. Wrapping this up in 'nn' should be the most straightforward approach to reimplementing this model.

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureA request for a proper, new feature.triagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions