Skip to content

Mcc metric#3537

Merged
vfdev-5 merged 3 commits intopytorch:masterfrom
aaishwarymishra:mcc_metric
Feb 21, 2026
Merged

Mcc metric#3537
vfdev-5 merged 3 commits intopytorch:masterfrom
aaishwarymishra:mcc_metric

Conversation

@aaishwarymishra
Copy link
Copy Markdown
Collaborator

@aaishwarymishra aaishwarymishra commented Feb 17, 2026

This pull request introduces a new metric, Matthews correlation coefficient (MCC), to the Ignite metrics package, enhancing the library's support for evaluating classification models. The implementation includes the metric class, integration with the metrics API, and a comprehensive test suite. Additionally, there is a minor type annotation update in the early stopping handler.

New Metric Addition and Integration:

  • Added a new MatthewsCorrCoef metric in ignite/metrics/matthews_corrcoef.py, which computes the Matthews correlation coefficient for binary and multiclass classification using scikit-learn under the hood.
  • Registered MatthewsCorrCoef in the metrics package by importing it in ignite/metrics/__init__.py and adding it to the __all__ list, making it available for users. [1] [2]

Testing Improvements:

  • Added a comprehensive test suite in tests/ignite/metrics/test_matthews_corrcoef.py to verify correct behavior, input validation, scikit-learn dependency handling, and integration with the Ignite engine.

Type Annotation Update:

  • Updated the type annotation for self.best_score in the early stopping handler for consistency with modern Python syntax, changing from Optional[float] to float | None.
  • Removed an unused import of Optional in ignite/handlers/early_stopping.py as a result of the type annotation update.Fixes #{issue number}

Fixes #3536

@github-actions github-actions bot added module: metrics Metrics module module: handlers Core Handlers module labels Feb 17, 2026
@aaishwarymishra
Copy link
Copy Markdown
Collaborator Author

aaishwarymishra commented Feb 17, 2026

Like the mcc is for classification so I can overwrite the check_shape method to allow y_pred and y_true to be of 3 dimention and just apply argmax on compute_fn but I am not sure as EpochMetric class say y_pred and y_true should be either 1d or 2d.

@vfdev-5
Copy link
Copy Markdown
Collaborator

vfdev-5 commented Feb 17, 2026

Like the mcc is for classification so I can overwrite the check_shape method to allow y_pred and y_true to be of 3 dimention and just apply argmax on compute_fn but I am not sure as EpochMetric class say y_pred and y_true should be either 1d or 2d.

I'm not totally sure to understand your point here, if you can detail a bit more, it would help

@aaishwarymishra
Copy link
Copy Markdown
Collaborator Author

aaishwarymishra commented Feb 17, 2026

I don't know why those tests were failing I used make html command and it didn't gave me no error that time.

@aaishwarymishra
Copy link
Copy Markdown
Collaborator Author

aaishwarymishra commented Feb 17, 2026

Like the mcc is for classification so I can overwrite the check_shape method to allow y_pred and y_true to be of 3 dimention and just apply argmax on compute_fn but I am not sure as EpochMetric class say y_pred and y_true should be either 1d or 2d.

I'm not totally sure to understand your point here, if you can detail a bit more, it would help

No worries I got it I will update the code.

vfdev-5
vfdev-5 reviewed Feb 18, 2026
View reviewed changes
@github-actions github-actions bot added the docs label Feb 18, 2026
vfdev-5
vfdev-5 reviewed Feb 18, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@vfdev-5 vfdev-5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this PR!
I left more comments to improve it. Please check

@aaishwarymishra
Copy link
Copy Markdown
Collaborator Author

aaishwarymishra commented Feb 20, 2026

done

vfdev-5
vfdev-5 reviewed Feb 20, 2026
View reviewed changes
@aaishwarymishra aaishwarymishra force-pushed the mcc_metric branch 3 times, most recently from 499c084 to 49c4f4a Compare February 20, 2026 21:29
@aaishwarymishra
Copy link
Copy Markdown
Collaborator Author

aaishwarymishra commented Feb 20, 2026

sorry, I don't know why why the pre-commit hook wasn't running properly on my system with docs. I have fixed it and squashed all the commits. Sorry for wasting your time everthing should pass now

@vfdev-5
Copy link
Copy Markdown
Collaborator

vfdev-5 commented Feb 20, 2026

doctest failure is real:

File "../../ignite/metrics/matthews_corrcoef.py", line ?, in default
Failed example:
    y_pred = torch.tensor([[+1, +1, +1, -1]])
    y_true = torch.tensor([[+1, -1, +1, +1]])

    matthews_corrcoef = MatthewsCorrCoef()
    matthews_corrcoef.attach(default_evaluator, 'mcc')
    state = default_evaluator.run([[y_pred, y_true]])
    print(state.metrics['mcc'])
Exception raised:
    Traceback (most recent call last):
      File "/home/runner/work/_temp/uv-python-dir/cpython-3.10-linux-x86_64-gnu/lib/python3.10/doctest.py", line 1350, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest default[0]>", line 1, in <module>
        y_pred = torch.tensor([[+1, +1, +1, -1]])
    NameError: name 'torch' is not defined

@aaishwarymishra
feat: add Matthews correlation coefficient metric and corresponding t…
f0b6945 
…ests

fix: handle dimensionality inconsistencies in matthews_corrcoef_compute_fn

feat: add Matthews correlation coefficient to the complete list of metrics

docs: update documentation for Matthews correlation coefficient metric to clarify input shapes and classification support

fix: update example input shapes in MatthewsCorrCoef documentation
@aaishwarymishra
Copy link
Copy Markdown
Collaborator Author

aaishwarymishra commented Feb 20, 2026

There was a error in tensor shape in example I fixed it.

vfdev-5
vfdev-5 approved these changes Feb 21, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@vfdev-5 vfdev-5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @aaishwarymishra !

@vfdev-5 vfdev-5 enabled auto-merge February 21, 2026 21:29
@vfdev-5 vfdev-5 added this pull request to the merge queue Feb 21, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 21, 2026
@vfdev-5 vfdev-5 enabled auto-merge February 21, 2026 22:30
@vfdev-5 vfdev-5 added this pull request to the merge queue Feb 21, 2026
Merged via the queue into pytorch:master with commit 293b257 Feb 21, 2026
26 of 27 checks passed
@aaishwarymishra aaishwarymishra deleted the mcc_metric branch March 3, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vfdev-5 vfdev-5 vfdev-5 approved these changes

Assignees

No one assigned

Labels

docs module: handlers Core Handlers module module: metrics Metrics module

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature] Add classification metric MCC

2 participants

@aaishwarymishra @vfdev-5