Right now embedding's padding_idx is poorly documented.
This is how it's supposed to work:
padding_idx = 0
Embedding(10, 3, padding_idx)
Now, whenever you pass in index 0, it will be returning a zeroed embedding.
Right now one has to go set the weight to do this as well.