The T5Config has the parameter n_positions set to 512 and max_position_embeddigs referring to n_positions. However, neither max_position_embeddigs nor n_positions is used in the T5Model and T5 is not limited to max_position_embeddings. E.g.:

from transformers import T5Model
model = T5Model.from_pretrained("t5-small")

model.config.max_position_embeddings # shows 512

input_ids = torch.tensor([600 * [0]])  # input of size > 512

model(input_ids, decoder_input_ids=input_ids)  # works fine

I think we should delete the parameter.

@thomwolf - do you remember why we added max_position_embeddigs and n_positions to T5? The model does not seem to use these params and also should not be limited to 512 due to its relative position embeddings.