Environment info

• transformers version: Github Main branch

Who can help

TextGeneration: @TevenLeScao

Information

Model I am using (Bert, XLNet ...): T5

I'm unable to pass preconputed encoder_outputs to the .generate() method.

I've tried defining:

model_kwargs = {"encoder_outputs": encoder_outputs}
output = model.generate(model_kwargs)

But I've noticed some validation errors for input_ids. Even if I replace input_ids with dummy values, I've noticed the model_kwargs is always replaced here:

https://github.com/huggingface/transformers/blob/master/src/transformers/generation_utils.py#L448

I think it can be fixed (without optimization) by just replacing:

model_kwargs["encoder_outputs"] = encoder_outputs

with:

model_kwargs.setdefault("encoder_outputs", encoder_outputs)

If you agree I can try to open a PR to fix this.

Best,