Skip to content

[metrics] Raise a clear error for self-intersecting ground truth boxes - #2219

Merged
felixdittrich92 merged 2 commits into
mindee:mainfrom
RavSinghChandan:polygon-iou-misordered-points
Nov 9, 2026
Merged

felixdittrich92 merged 2 commits into
mindee:mainfrom
RavSinghChandan:polygon-iou-misordered-points

Conversation

@RavSinghChandan

Copy link
Copy Markdown
Contributor

polygon_iou raised GEOSException: TopologyException: side location conflict when a box's points are not in order around it, e.g. a ground-truth polygon annotated as [[0, 0], [1, 1], [1, 0], [0, 1]]: the polygon crosses itself and shapely can not intersect it. This stops LocalizationConfusion, OCRMetric and DetectionMetric with use_polygons=True on the whole evaluation.

The points of such a box are now put back in order with order_points before computing the IoU; valid polygons are left as they are.

Test added (fails on main)

@codecov

codecov Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.80%. Comparing base (24ce391) to head (a2fbb00).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2219      +/-   ##
==========================================
+ Coverage   97.78%   97.80%   +0.02%     
==========================================
  Files         170      170              
  Lines       10999    11016      +17     
==========================================
+ Hits        10755    10774      +19     
+ Misses        244      242       -2     
Flag Coverage Δ
unittests 97.80% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@felixdittrich92 felixdittrich92 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes

Thanks for tracking this down. Crashing deep inside GEOS with a TopologyException is a bad experience, and the fix itself works. But I don't think we should quietly repair boxes inside polygon_iou.

A ground-truth box whose points cross each other is a broken annotation. If we reorder its points without telling anyone, the metric reports a score against a box that isn't in the dataset, and the annotation error is hidden. Rather than accept wrong ground truth, we should reject it with a clear error so the dataset gets fixed.

Requested changes:

  1. Remove the automatic reordering (_polygons / order_points) from polygon_iou.
  2. Raise a clear ValueError instead of the GEOS exception. It should name the offending boxes, e.g. "Ground truth boxes at indices [3, 17] are self-intersecting (points not in TL, TR, BR, BL order); please fix the annotations." Since polygon_iou can't tell ground truth from predictions, the cleanest place for this is a small helper called on gts in the update() methods that use use_polygons (LocalizationConfusion, OCRMetric, DetectionMetric, ObjectDetectionMetric, TableCellMetric). If you think predictions should be validated the same way, I'm open to that too.
  3. Don't reject degenerate boxes. Zero-area boxes (collinear points, repeated points) also fail is_valid, and collinear ones even report Self-intersection. Main returns IoU 0 for these today and that should stay the same. Only flag boxes that are invalid and have a non-zero convex-hull area, i.e. a real box whose points are out of order.
  4. Tests: replace test_polygon_iou_misordered_points with tests that check:
    • a crossed ground-truth box raises ValueError with its index in the message;
    • a batch mixing good and bad rows reports only the bad indices;
    • zero-area / collinear boxes still give IoU 0 with no error.

On performance: I benchmarked the is_valid check against main, and it adds about 2% end-to-end on LocalizationConfusion with polygons, so a vectorized validity check is fine to keep.

@felixdittrich92 felixdittrich92 self-assigned this Nov 4, 2026
@RavSinghChandan RavSinghChandan changed the title [metrics] Compute the IoU of a box whose points are not in order in polygon_iou [metrics] Raise a clear error for self-intersecting ground truth boxes Nov 4, 2026
@RavSinghChandan

Copy link
Copy Markdown
Contributor Author

Thanks, that makes sense: fixing them quietly hides the broken annotation. Changed as you suggested: the reordering is gone from polygon_iou, and the update() of the five polygon metrics now raise a ValueError naming the self-intersecting ground truth indices. Zero-area / collinear boxes are left alone (IoU 0, no error). Tests cover a crossed box, a mixed batch and degenerate boxes.

@felixdittrich92 felixdittrich92 added this to the 1.2.0 milestone Nov 9, 2026
@felixdittrich92 felixdittrich92 added type: enhancement Improvement module: utils Related to doctr.utils ext: tests Related to tests folder labels Nov 9, 2026

@felixdittrich92 felixdittrich92 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 👍

@felixdittrich92
felixdittrich92 merged commit 963febd into mindee:main Nov 9, 2026
57 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ext: tests Related to tests folder module: utils Related to doctr.utils type: enhancement Improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants