After quick fix #8379
Tried same flow of #8378
Gives error:
/usr/local/lib/python3.6/dist-packages/transformers/data/data_collator.py in __call__(self, examples)
321 for e in examples:
322 ref_tokens = []
--> 323 for id in e["input_ids"].tolist():
324 token = self.tokenizer._convert_id_to_token(id)
325 ref_tokens.append(token)
AttributeError: 'list' object has no attribute 'tolist'
When I tried with DataCollatorForLanguageModeling, trainer works as usual.
After quick fix #8379
Tried same flow of #8378
Gives error:
When I tried with DataCollatorForLanguageModeling, trainer works as usual.