Skip to content

The DeLong variance in the AUC gaps explainer is quadratic and takes 17 seconds #1169

Open

Description

@yakew7

Where: explainers/auc-gaps-by-group.md (delong_auc_var), explainers/code/auc-gaps-by-group.py.

Current gap: The placement values are computed with a Python loop over every positive and every negative ((neg < p).mean()), so a group of 10,000 rows takes most of the script's 17 seconds. The other five new explainer scripts take under 2.5 seconds.

Repro:

$ time PYTHONPATH=. python3 explainers/code/auc-gaps-by-group.py   # real 17s

Suggested approach: Compute the placement values from ranks (scipy.stats.rankdata) in O(n log n), check the numbers are unchanged, and update the markdown.

Done when: the script runs in under 3 seconds with identical printed output.

Activity

  1. dxnnystyles commented on Oct 28, 2026

    @dxnnystyles
    Contributor

    i can replace the quadratic pairwise placement loops with ranks, then verify the generated output is unchanged and the script runs under three seconds.

  2. dxnnystyles commented on Oct 29, 2026

    @dxnnystyles
    Contributor

    i wasn't able to get this to the performance target in the issue, so i'm stepping back from it for now. leaving it open for someone else to take.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions