Currently, LabelKFold sorts samples by weight so as to create balanced folds in terms of size. However, this is at odds with shuffling. Moreover, the regular KFold partitions the samples without changing their order, which is useful if the order is meaningful or reflects a stratification.

My suggestion is to make the sorting for balanced fold sizes in LabelKFold optional, so that shuffling or the original order of samples is honored, by keeping the labels in the order in which they are first encountered.