In the current Python binding, the terminology for some training related terms is a bit inconsistent. Specifically, here we should determine the usage of the term for one mini-batch and one full pass through the data.

Current status is: round and iteration mean the same thing -- full path through the data. epoch means a single mini-batch. This will affect parameter and callback names, like start_round, epoch_callback, etc. We should choose a unified (and standard) terminology here before it's getting too late to change names.

My personal opinion is that epoch, if we ever want to use this name, should mean one full pass through the data. I do not find any "standard" reference for this. But looking around, almost all materials I found epoch means this. Some examples:

• One of Leon Bottou's paper on SGD
• Caffe: they do not have variable or function with epoch in it, but from here, I believe epoch means 1 full data pass
• In Theano, epoch is the outer loop, and batch is the inner loop
• In Chainer, epoch is the outer loop
• ...

Possible vote options (please add more if needed)

1. epoch for full data pass, iteration for a single minibatch
2. data_pass for full data pass, iteration for a single minibatch
3. round for full data pass, iteration for a single minibatch
4. epoch for full data pass, batch for a single minibatch