I met this bug when I used the setting below:

global_steps = 2748
len(train_dataloader) = 27484
gradient_accumulation_steps = 4

In the original code, "steps_trained_in_current_epoch" will be 2748 ! BUT this variable should be 2748*4 = 10,992

the code I suggested is below:

epochs_trained = (self.global_step * self.args.gradient_accumulation_steps) // len(train_dataloader)
steps_trained_in_current_epoch = (self.global_step * self.args.gradient_accumulation_steps) % len(train_dataloader)