Description

The ensemble.StackingClassifier (slated for release in v0.22) stack_method="auto" predict-function priority does not match that of CalibratedClassifierCV.

In StackingClassifier, when stack_method="auto" the priority is:

1. predict_proba,
2. decision_function,
3. predict.

In CalibratedClassifierCV the priority is

1. decision_function,
2. predict_proba,
3. predict.

Steps/Code to Reproduce

No steps to reproduce. I raise this point for discussion.

Expected Results

I expected scikit's cross-validation routines/techniques (in which I include CalibratedClassifierCV and StackingClassifier) to use the same resolution priority when determining which predict function to invoke among predict_prob, decision_function, and predict.

Actual Results

When stack_method="auto" (the default value) in StackingClassifier the predict_proba method is preferred to decision_function and predict. The docs state:

For StackingClassifier, note that the output of the estimators is controlled by the parameter
stack_method and it is called by each estimator. This parameter is either a string, being estimator
method names, or 'auto' which will automatically identify an available method depending on the
availability, tested in the order of preference: predict_proba, decision_function and predict.

Versions

I am using version 0.22rc2.post1.