Reference Issues/PRs

Another approach for #31338 (which superseded #25639)
closes #21391

What does this implement/fix? Explain your changes.

Uses the approach suggested by @jeremiedbb (#31338 (comment)) of using re-using confusion_matrix_at_thresholds (previously called _binary_clf_curve) to implement curve calculation - given a callable metric, calculates the metric for each threshold.

I think I like this approach better:

• Brings decision_threshold_curve in line with the other curve functions we have (det_curve, precision_recall_curve, roc_curve), which all use confusion_matrix_at_thresholds.
  • all of these curve functions would under go the same validation, errors/changes are more easily made for all curve functions
• As you can see much fewer code changes required for this vs using _CurveScorer
  • we could undo the changes in MNT Moving _CurveScorer from model_selection to metrics #29216
  • parts of _CurveScorer are probably unnecessary for this function, e.g., sign - as this is a stand alone function (vs being used inside an estimator (TunedThresholdClassifierCV), this is probably not needed the user can manipulate the output as required. Also _CurveScorer is particular in mapping thresholded labels to original classes (requiring call to unique(y)), which isn't really needed in this case
  • having to instantiate a class for this function is probably over-complicating things

Any other comments?

Working on this also raised two questions:

• I noticed in sklearn/metrics most modules have a leading _ (e.g., _regression.py). I thought this implied private module/functions?
• This function was originally named metric_per_threshold , then changed to decision_threshold_curve, since it is already in the metric module, the name metric was considered redundant (see FEA Implementation of "threshold-dependent metric per threshold value" curve #25639 (comment)). Is 'decision' meant to imply that this curve is meant to help with determining the threshold to use? Is term commonly used (genuine question, as I don't have the background in this area)?

cc @jeremiedbb @glemaitre , and just in case you have time @StefanieSenger