Incremental Uncertainty-aware Performance Monitoring with Active Labeling Intervention (AISTATS 2025)

This repository contains the code related to the paper Incremental Uncertainty-aware Performance Monitoring with Active Labeling Intervention, published at AISTATS 2025.

Incremental Uncertainty-aware Performance Monitoring (IUPM) is a novel label-free method that estimates performance changes by modeling gradual shifts using optimal transport. IUPM quantifies the uncertainty in the performance prediction and introduces an active labeling procedure to restore a reliable estimate under a limited labeling budget.

Usage

To install requirements use:

pip install -r ./requirements.txt

Citation

In case you find this work useful please cite:

@inproceedings{
koebler2025incremental,
title={Incremental Uncertainty-aware Performance Monitoring with Active Labeling Intervention},
author={Alexander Koebler and Thomas Decker and Ingo Thon and Volker Tresp and Florian Buettner},
booktitle={The 28th International Conference on Artificial Intelligence and Statistics},
year={2025},
url={https://openreview.net/forum?id=eX94LTe7f1}
}