Skip to content

Does MBartTokenizer remove the parameter decoder_input_ids? #8416

Description

@wmathor

Environment info

  • transformers version:3.4.0
  • Platform:Google Colab
  • Python version:3.7
  • PyTorch version (GPU?):1.7.0+cu101
  • Tensorflow version (GPU?):2.x
  • Using GPU in script?: no
  • Using distributed or parallel set-up in script?:no

Who can help

Information

Model I am using (Bert, XLNet ...): mbart

The problem arises when using:

  • the official example scripts: (give details below)
  • my own modified scripts: (give details below)

The tasks I am working on is:

  • an official GLUE/SQUaD task: (give the name)
  • my own task or dataset: (give details below)

To reproduce

example_english_phrase = "UN Chief Says There Is No Military Solution in Syria"
expected_translation_romanian = "Şeful ONU declară că nu există o soluţie militară în Siria"
batch = MBartTokenizer.from_pretrained('facebook/mbart-large-cc25').prepare_seq2seq_batch(example_english_phrase, src_lang="en_XX", tgt_lang="ro_RO", tgt_texts=expected_translation_romanian)
input_ids = batch["input_ids"]
target_ids = batch["decoder_input_ids"]

Steps to reproduce the behavior:

KeyError                                  Traceback (most recent call last)
<ipython-input-11-b3eedaf10c3e> in <module>()
      3 batch = MBartTokenizer.from_pretrained('facebook/mbart-large-en-ro').prepare_seq2seq_batch(example_english_phrase, src_lang="en_XX", tgt_lang="ro_RO", tgt_texts=expected_translation_romanian)
      4 input_ids = batch["input_ids"]
----> 5 target_ids = batch["decoder_input_ids"]

/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils_base.py in __getitem__(self, item)
    232         """
    233         if isinstance(item, str):
--> 234             return self.data[item]
    235         elif self._encodings is not None:
    236             return self._encodings[item]

KeyError: 'decoder_input_ids'

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions