Skip to content

improve numerical stability of pearson's correlation coefficient#3152

Merged
Borda merged 11 commits into
Lightning-AI:masterfrom
gratus907:master
Jun 27, 2025
Merged

improve numerical stability of pearson's correlation coefficient#3152
Borda merged 11 commits into
Lightning-AI:masterfrom
gratus907:master

Conversation

@gratus907

@gratus907 gratus907 commented Jun 27, 2025

Copy link
Copy Markdown
Contributor

What does this PR do?

Closes #3150 by:

  • Using Welford's algorithm for _pearson_corrcoef_update
  • Tracking maximum absolute deviation (approximately) and uses them for scaling in _pearson_corrcoef_compute
from torchmetrics.regression import PearsonCorrCoef
from torchmetrics.functional.regression.pearson import pearson_corrcoef
import torch
from scipy.stats import pearsonr

torch.manual_seed(42)
scale = 1e-4
bias = 1e2
preds = torch.randn(100)
target = preds + 0.1 * torch.randn(100)
preds *= scale
target *= scale
preds += bias
target += bias
print(pearson_corrcoef(preds, target))
print(pearsonr(preds.cpu().numpy(), target.cpu().numpy())[0])

For this test, where predictions and targets are large in their absolute values but have small variances,

Original implementation: nan
Current implementation: 0.9955
Scipy reference: 0.99274683 

For above test with scale = 1e-3 and bias = 1e1,

Original implementation: nan
Current implementation: 0.9960
Scipy reference: 0.99597996

It is not as accurate as scipy, but for a streaming algorithm I think this is reasonably good.

Especially when bias = 0, current implementation using torch.float32 works up to scale=1e-18. (original implementation fails at scale = 1e-4)

Note: The tests were modified for current implementation. Specifically, inputs in test_pearsons_warning_on_small_input are changed, and for test_final_aggregation_no_inplace_change, the test logic incorporates max_abs_dev_*.

Before submitting
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?
PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃


📚 Documentation preview 📚: https://torchmetrics--3152.org.readthedocs.build/en/3152/

@codecov

codecov Bot commented Jun 27, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.50000% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 36%. Comparing base (d1209ed) to head (f261585).
⚠️ Report is 91 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (d1209ed) and HEAD (f261585). Click for more details.

HEAD has 68 uploads less than BASE
Flag BASE (d1209ed) HEAD (f261585)
macOS 6 3
python3.10 22 11
cpu 32 16
torch2.0.1 4 2
torch2.7.1 2 1
python3.12 8 4
torch2.0.1+cpu 6 3
Windows 4 2
torch2.7.1+cpu 6 3
Linux 22 11
torch2.6.0+cpu 4 2
python3.9 2 1
gpu 2 0
unittest 2 0
torch2.1.2+cpu 2 1
torch2.4.1+cpu 2 1
torch2.5.1+cpu 2 1
torch2.2.2+cpu 2 1
torch2.3.1+cpu 2 1
Additional details and impacted files
@@           Coverage Diff            @@
##           master   #3152     +/-   ##
========================================
- Coverage      70%     36%    -34%     
========================================
  Files         364     349     -15     
  Lines       20013   19865    -148     
========================================
- Hits        14046    7123   -6923     
- Misses       5967   12742   +6775     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Borda Borda changed the title enhancement: improve numerical stability of pearson's correlation coefficient improve numerical stability of pearson's correlation coefficient Jun 27, 2025
@Borda Borda added the enhancement New feature or request label Jun 27, 2025
Comment thread src/torchmetrics/functional/regression/concordance.py Outdated
@mergify mergify Bot added the ready label Jun 27, 2025
Comment thread src/torchmetrics/functional/regression/pearson.py Outdated
@Borda
Borda merged commit 4fd06f2 into Lightning-AI:master Jun 27, 2025
41 of 43 checks passed
Borda pushed a commit that referenced this pull request Jul 3, 2025
* enhancement: improve numerical stability of pearson correlation coefficient
* Apply suggestions from code review

---------

Co-authored-by: Jirka Borovec <[email protected]>
Co-authored-by: Rittik Panda <[email protected]>
Co-authored-by: Jirka B <[email protected]>

(cherry picked from commit 4fd06f2)
Borda pushed a commit that referenced this pull request Jul 5, 2025
* enhancement: improve numerical stability of pearson correlation coefficient
* Apply suggestions from code review

---------

Co-authored-by: Jirka Borovec <[email protected]>
Co-authored-by: Rittik Panda <[email protected]>
Co-authored-by: Jirka B <[email protected]>

(cherry picked from commit 4fd06f2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PearsonCorrCoef() returns NaN when values are small

3 participants