Oh, so should I close mine PR? There are still some TODOs in my PR that I haven't finished though.
Great work 👍

Would be nice to add an example illustrating possible use-cases of balanced accuracy. For instance a binary classification task with highly imbalanced classes, where a classifier which assigns everything to the majority class obtains high accuracy but low balanced-accuracy.

I'm not sure if the documentation has a similar example for P/R/F, but an
annotated comparison between them (accuracy, balanced accuracy, P, R, AUC)
makes some sense.

On 14 December 2014 at 23:45, Jan Hendrik Metzen [email protected]
wrote:

Would be nice to add an example illustrating possible use-cases of
balanced accuracy. For instance a binary classification task with highly
imbalanced classes, where a classifier which assigns everything to the
majority class obtains high accuracy but low balanced-accuracy.

—
Reply to this email directly or view it on GitHub
#3929 (comment)
.

I think adding an example would be a good idea. I will work on the comment by @jnothman and then on an example in a couple of weeks during the christmas break!

The other PR says
In binary and multiclass classification, this function is equal to thejaccard_similarity_scorefunction.
So if this is added the addition is for multi-label indicator, right?

@amueller No. I don't think so. For example,

>>> jaccard_similarity_score(np.array([[0,0,1,1,1]]), np.array([[1, 1,1,0, 1]]))
0.4
>>> balanced_accuracy_score(np.array([0,0,1,1,1]), np.array([1,1,1,0,1]))
0.33333333333333331

The wikipedia definition of jaccard similarity http://en.wikipedia.org/wiki/Jaccard_index#Similarity_of_asymmetric_binary_attributes
is quite different from
http://en.wikipedia.org/wiki/Accuracy_and_precision#In_binary_classification.

closed in favor of #5588