Skip to content

Coverage validity not verified in MapieRegressor when the number of calibration data is very small #452

@thibaultcordier

Description

@thibaultcordier

Describe the bug
In MapieRegressor, the quantile calculation does not allow us to check the coverage validity when the number of calibration data is very small.

To Reproduce
A rigorous approach to testing the validity of the coverage was carried out by Mahdi Torabi Rad (see his notebook: https://github.com/mtorabirad/MLBoost/blob/main/Episode15/Episode15Main.ipynb) in the special case where the number of calibration data is very small (6 in the following illustration).

Expected behavior
Use the corrected quantile ($|(n+1)(1−α)|/n$) to check the validity of the coverage for any number of calibration data.

Screenshots
image

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions