Loading from AutoModel gives ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds #16234
Description
Executing the following code snippet states that ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds.
from transformers import AutoModel, AutoTokenizer
model_name = "castorini/t5-base-canard"
model = AutoModel.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
context = '''
Frank Zappa ||| Disbandment ||| What group disbanded ||| Zappa and the Mothers of Invention ||| When did they disband?
'''
encoded_input = tokenizer(
context,
padding='max_length',
max_length=512,
truncation=True,
return_tensors="pt",
)
decoder_input = tokenizer(
context,
padding='max_length',
max_length=512,
truncation=True,
return_tensors="pt",
)
encoder_output = model.generate(input_ids=encoded_input["input_ids"], decoder_input_ids=decoder_input["input_ids"])
output = tokenizer.decode(
encoder_output[0],
skip_special_tokens=True
)
outputyields this error:
Some weights of the model checkpoint at castorini/t5-base-canard were not used when initializing T5Model: ['lm_head.weight']
- This IS expected if you are initializing T5Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing T5Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Input length of decoder_input_ids is 512, but ``max_length`` is set to 20. This can lead to unexpected behavior. You should consider increasing ``config.max_length`` or ``max_length``.
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
[<ipython-input-11-b9fe12b71812>](https://localhost:8080/#) in <module>()
24 )
25
---> 26 encoder_output = model.generate(input_ids=encoded_input["input_ids"], decoder_input_ids=decoder_input["input_ids"])
27 output = tokenizer.decode(
28 encoder_output[0],
6 frames
[/usr/local/lib/python3.7/dist-packages/transformers/models/t5/modeling_t5.py](https://localhost:8080/#) in forward(self, input_ids, attention_mask, encoder_hidden_states, encoder_attention_mask, inputs_embeds, head_mask, cross_attn_head_mask, past_key_values, use_cache, output_attentions, output_hidden_states, return_dict)
925 else:
926 err_msg_prefix = "decoder_" if self.is_decoder else ""
--> 927 raise ValueError(f"You have to specify either {err_msg_prefix}input_ids or {err_msg_prefix}inputs_embeds")
928
929 if inputs_embeds is None:
ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds