fix Detection IoU ignores predictions without ground truth#3025
Merged
SkafteNicki merged 9 commits intoMar 31, 2025
Merged
Conversation
Isalia20
requested review from
Borda,
SkafteNicki,
justusschock and
stancld
as code owners
March 23, 2025 12:00
Isalia20
changed the title
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #3025 +/- ##
========================================
- Coverage 70% 35% -35%
========================================
Files 355 341 -14
Lines 19654 19475 -179
========================================
- Hits 13775 6908 -6867
- Misses 5879 12567 +6688 🚀 New features to boost your workflow:
|
Contributor
Author
|
Ready for review, errors seem to be unrelated |
SkafteNicki
reviewed
Mar 24, 2025
SkafteNicki
left a comment
SkafteNicki
left a comment
Collaborator
There was a problem hiding this comment.
Overall looks good, just a couple of suggestions/fixes
Co-authored-by: Nicki Skafte Detlefsen <[email protected]>
Co-authored-by: Nicki Skafte Detlefsen <[email protected]>
Co-authored-by: Nicki Skafte Detlefsen <[email protected]>
Co-authored-by: Nicki Skafte Detlefsen <[email protected]>
Co-authored-by: Nicki Skafte Detlefsen <[email protected]>
Contributor
Author
|
Applied the fixes from review, tests fail when installing PyTDC==0.4.1 :( |
Collaborator
will try to check it... 🤔 |
Contributor
Author
|
Can this be reviewed? Tests seem to be green except for some errors about purging cache and not being able to connect to huggingface |
Borda
approved these changes
Mar 26, 2025
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR fixes issue #2905 by adjusting how the IntersectionOverUnion metric handles predictions when no corresponding ground truth exists. Key changes include:
- Updating state collection to include predicted labels in both the metric and test implementations.
- Modifying the compute logic to calculate class-wise IoU using the union of ground truth and prediction labels.
- Enhancing tests to validate the new behavior.
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| tests/unittests/detection/test_intersection.py | Updated test helper to collect predicted labels and compute per-class metrics using union labels. |
| src/torchmetrics/detection/iou.py | Updated state management and compute method to incorporate predictions without ground truth. |
| CHANGELOG.md | Documented the fix for the IoU behavior issue. |
Comments suppressed due to low confidence (1)
tests/unittests/detection/test_intersection.py:80
- [nitpick] Consider renaming 'union_cls' to 'all_labels' to be consistent with the naming convention used in the metric implementation.
union_cls = set()
Borda
added a commit
that referenced
this pull request
Apr 7, 2025
* chlog update + `1.8.0dev` * fix issue * Update CHANGELOG.md Co-authored-by: Nicki Skafte Detlefsen <[email protected]> * Update src/torchmetrics/detection/iou.py Co-authored-by: Nicki Skafte Detlefsen <[email protected]> * Update src/torchmetrics/detection/iou.py Co-authored-by: Nicki Skafte Detlefsen <[email protected]> * Update src/torchmetrics/detection/iou.py Co-authored-by: Nicki Skafte Detlefsen <[email protected]> * Update src/torchmetrics/detection/iou.py Co-authored-by: Nicki Skafte Detlefsen <[email protected]> --------- Co-authored-by: Jirka B <[email protected]> Co-authored-by: Jirka Borovec <[email protected]> Co-authored-by: Nicki Skafte Detlefsen <[email protected]> (cherry picked from commit 9e08ff2)
Borda
added a commit
that referenced
this pull request
Apr 7, 2025
* chlog update + `1.8.0dev` * fix issue * Update CHANGELOG.md Co-authored-by: Nicki Skafte Detlefsen <[email protected]> * Update src/torchmetrics/detection/iou.py Co-authored-by: Nicki Skafte Detlefsen <[email protected]> * Update src/torchmetrics/detection/iou.py Co-authored-by: Nicki Skafte Detlefsen <[email protected]> * Update src/torchmetrics/detection/iou.py Co-authored-by: Nicki Skafte Detlefsen <[email protected]> * Update src/torchmetrics/detection/iou.py Co-authored-by: Nicki Skafte Detlefsen <[email protected]> --------- Co-authored-by: Jirka B <[email protected]> Co-authored-by: Jirka Borovec <[email protected]> Co-authored-by: Nicki Skafte Detlefsen <[email protected]> (cherry picked from commit 9e08ff2)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #2905
Before submitting
📚 Documentation preview 📚: https://torchmetrics--3025.org.readthedocs.build/en/3025/