🚀 Feature

SuperLoss is a simple and generic method that can be applied to a variety of losses and tasks without any changing the learning procedure. It consists in appending a novel loss function on top of any existing task loss. Its main effect is to automatically down-weight the contribution of samples with a large loss,i.e. hard samples, effectively mimicking the curriculum learning. SuperLoss prevents the memorization of noisy samples, making it possible to train from noisy data even with non-robust loss functions. Experimental results on image classification, regression, object detection and image retrieval demonstrated consistent gain.

Motivation

https://proceedings.neurips.cc//paper/2020/file/2cfa8f9e50e0f510ede9d12338a5f564-Paper.pdf
https://proceedings.neurips.cc/paper/2020/file/2cfa8f9e50e0f510ede9d12338a5f564-Supplemental.pdf

cc @albanD @mruberry @jbschlosser