@sgugger @stas00 (as suggested in 15196)
It is just a minor issue but it could prevent some head-scratching:
Setting group_by_length to True during training has, as far as I can see, no effect if the provided dataset is a torch.utils.data.IterableDataset, as can be seen in this line:
|
if isinstance(train_dataset, torch.utils.data.IterableDataset): |
Entering this branch will run the training without bucketing / grouping.
Imo it would be best to raise an exception or, at the very least, print a warning message for the user. Grouping only happens in _get_train_sampler() which is never reached if said condition is true.
@sgugger @stas00 (as suggested in 15196)
It is just a minor issue but it could prevent some head-scratching:
Setting
group_by_lengthtoTrueduring training has, as far as I can see, no effect if the provided dataset is atorch.utils.data.IterableDataset, as can be seen in this line:transformers/src/transformers/trainer.py
Line 646 in c4d1fd7
Entering this branch will run the training without bucketing / grouping.
Imo it would be best to raise an exception or, at the very least, print a warning message for the user. Grouping only happens in
_get_train_sampler()which is never reached if said condition is true.