Environment info

• transformers version: I have had this issue with both 4.3.0 and 4.4.2 (and probably other versions as well)
• Python version: 3.7.6
• PyTorch version (GPU?): 1.7.0
• Using GPU in script?: No, I just tested it on the CPU, but it would probably also happen on the GPU
• Using distributed or parallel set-up in script?: No

Who can help

• gpt2: @patrickvonplaten, @LysandreJik
• Documentation: @sgugger

Information

Model I am using (Bert, XLNet ...): gpt2

The problem arises when using:

• [ x] the official example scripts: (give details below)
• my own modified scripts: (give details below)

To reproduce

Hello, I am trying to run this example here: https://huggingface.co/transformers/task_summary.html#causal-language-modeling. When I run that code, exactly the same as it is on that page, I get strange/very bad results. Even when I change the input text, it still gives weird results (e.g., predicting empty spaces or strange characters). I also asked my coworker to try it on her computer, and she also got strange results.

I am planning to fine-tune GPT-2 for a different purpose later, but was a bit concerned because I couldn't even get this simple example demo to work. Thanks for your help!

Steps to reproduce the behavior:

1. Just run the exact example code that I linked above