DebertaForMaskedLM cannot load the parameters in the MLM head #15673
Description
Hello,
When I tried to use Deberta for MLM and I didn't succeed. It seems that it is a known issue. The checkpoint provided by microsoft/deberta-base and other similar ones don't have the pre-trained weights of the Masked LM head.
Thank you!