Skip to content

Latest version of trl fails to train a reward model #1354

Description

@armsp

I get an unexpected error at the line where I initialize the RewardTrainer object -

trainer = RewardTrainer(
    model = model,
    args = training_args,
    tokenizer= tokenizer,
    train_dataset= train_data,
    eval_dataset= val_data,
    peft_config= peft_config
)

Error:

/python3.9/site-packages/trl/trainer/reward_trainer.py:189: UserWarning: When using RewardDataCollatorWithPadding, you should set `remove_unused_columns=False` in your RewardConfig we have set it for you, but you should do it yourself in the future.
  warnings.warn(
Traceback (most recent call last):
  File "/cluster/TRAINREWARDMODEL.py", line 189, in <module>
    trainer = RewardTrainer(
  File "/python3.9/site-packages/trl/trainer/reward_trainer.py", line 198, in __init__
    super().__init__(
  File "/python3.9/site-packages/transformers/trainer.py", line 367, in __init__
    self.create_accelerator_and_postprocess()
  File "/python3.9/site-packages/transformers/trainer.py", line 4127, in create_accelerator_and_postprocess
    self.accelerator = Accelerator(
TypeError: __init__() got an unexpected keyword argument 'use_seedable_sampler'

I have never had this error before and I was able to train reward models successfully. The code is exactly the same...I just updated the library. I am NOT using Accelerate...not even importing it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions