System Info

• transformers version: 4.39.0.dev0
• Platform: Linux-3.10.0-1160.108.1.el7.x86_64-x86_64-with-glibc2.17
• Python version: 3.9.9
• Huggingface_hub version: 0.20.3
• Safetensors version: 0.4.2
• Accelerate version: 0.25.0.dev0
• Accelerate config: not found
• PyTorch version (GPU?): 2.0.1+cu118 (False)
• Tensorflow version (GPU?): 2.14.0 (False)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?: Yes

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Just instantiate the RewardTrainer object -

trainer = RewardTrainer(
    model = model,
    args = training_args,
    tokenizer= tokenizer,
    train_dataset= train_data,
    eval_dataset= val_data,
    peft_config= peft_config
)

And I get the error -

/python3.9/site-packages/trl/trainer/reward_trainer.py:189: UserWarning: When using RewardDataCollatorWithPadding, you should set `remove_unused_columns=False` in your RewardConfig we have set it for you, but you should do it yourself in the future.
  warnings.warn(
Traceback (most recent call last):
  File "/cluster/TRAINREWARDMODEL.py", line 189, in <module>
    trainer = RewardTrainer(
  File "/python3.9/site-packages/trl/trainer/reward_trainer.py", line 198, in __init__
    super().__init__(
  File "/python3.9/site-packages/transformers/trainer.py", line 367, in __init__
    self.create_accelerator_and_postprocess()
  File "/python3.9/site-packages/transformers/trainer.py", line 4127, in create_accelerator_and_postprocess
    self.accelerator = Accelerator(
TypeError: __init__() got an unexpected keyword argument 'use_seedable_sampler'

Unfortunately I opened this issue in trl but its not a issue from there and I cannot close it huggingface/trl#1354 it is clearly an issue in the transformers library.

Note: my code was working just fine until I updated the libraries. I am NOT using Accelerate...not even importing it.

Expected behavior

The RewardTrainer should initialize and just work.