-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Closed
Description
Is your feature request related to a problem? Please describe.
The deeplabcut benchmark scripts currently do not check if a submission is complete, e.g. if predictions for all test images are returned, as noted by @n-poulsen .
Describe the solution you'd like
A good solution would be to add a few lines to this part of the evaluation code, sth along the lines of
predictions = self.get_predictions(name)
self._validate_predictions(name)
...
def _validate_predictions(self, predictions):
# check if predictions contains too many images not contained in the ground truth
# check if predictions contains not all images in the ground truth
# (other potential tests)
passMMathisLab