Hi @lewtun! The tf.data.Dataset objects here already have a batch() operation applied - they're more like torch DataLoader objects. After a batch operation, their len() is num_samples // batch_size already, so we shouldn't need to do that division twice.

That said, the comments are confusing because it isn't clear the division has already been performed. Maybe I should clarify them?

Ah thanks for the insight! I'll close this PR and it would be great if you could open a new one with a clarifying remark because the comment in the code is indeed confusing for TF noobs (myself included!)

As a side note, how come the official examples normalize by batch size, e.g. here? In that example, we have

num_train_steps=int(training_args.num_train_epochs * train_batches_per_epoch)

where

train_batches_per_epoch = len(train_dataset) // total_train_batch_size

It was reading the official examples that convinced me there was an error in the course 😄

Agreed! In that example, train_dataset is the HuggingFace Dataset, so its len() is the number of samples. When we convert it to a TF tf.data.Dataset it becomes more like a DataLoader, so its len() is num_samples // batch_size. The confusion between the two types of Dataset object has been a problem for me, so I've tried to be clear and give the variables names like tf_train_dataset when I make the conversion, but I realize it's still quite opaque for people who aren't familiar.

Ah that makes more sense now!