Skip to content

Conversation

@n-poulsen
Copy link
Contributor

Adds the ability to use the confidence scores for keypoints for the alpha values for disks when creating labeled videos.

@n-poulsen
Copy link
Contributor Author

n-poulsen commented Jul 20, 2023

The code was updated so that confidence_to_alpha can now be a function. As the docstring describes:

confidence_to_alpha: Union[bool, Callable[[float], float], default=False
If False, all keypoints will be plot with alpha=1. Otherwise, this can be defined as a function f: [0, 1] -> [0, 1] such that the alpha value for a keypoint will be set as a function of its score: alpha = f(score). The default function used when True is f(x) = max(0, (x - pcutoff)/(1 - pcutoff)).

As an example of the resulting video obtained with the plotting function, here are the results on a toy project of running create_video_with_all_detections with two different values:

deeplabcut.create_video_with_all_detections(..., confidence_to_alpha=False)
penDLC_resnet50_dev-maJul18shuffle1_1000_full_no_conf.mov
pcutoff = 0.1
def confidence_to_alpha(c):
    return np.clip((c - pcutoff) / (1 - pcutoff), 0, 1)

deeplabcut.create_video_with_all_detections(..., confidence_to_alpha=confidence_to_alpha)
penDLC_resnet50_dev-maJul18shuffle1_1000_full.mov

In the second video, the low scores for the red keypoint in the top right corner is very clear, while it cannot be inferred from the first video with confidence_to_alpha=False.

@n-poulsen n-poulsen assigned MMathisLab and unassigned MMathisLab Jul 20, 2023
@n-poulsen n-poulsen requested a review from MMathisLab July 20, 2023 08:52
@n-poulsen n-poulsen marked this pull request as ready for review July 20, 2023 08:57
@jeylau
Copy link
Contributor

jeylau commented Jul 24, 2023

Looks good @n-poulsen! Low confidence points though will barely be visible: should the edge color remain, and only the face color be made more/less transparent?

@MMathisLab
Copy link
Member

I think it's good if low conf. are barely/not visible; pcutoff of 0 then wouldn't be shown, and if we can pass a range that is even better

@n-poulsen
Copy link
Contributor Author

Just two things to note:

  • keypoints with score < cfg["pcutoff"] will not be displayed even if their alpha value is > 0 in the confidence_to_alpha function (so if you define your own function, you should adapt your pcutoff to see all keypoints you want to see)
  • this plotting method was only implemented for the "fastmode" (both for create_labeled_videos and create_video_with_all_detections)

@MMathisLab MMathisLab merged commit 48f59f8 into main Jul 24, 2023
@MMathisLab MMathisLab deleted the plot_confidence_as_alpha branch July 25, 2023 11:10
@MMathisLab
Copy link
Member

I think this needs revisited. It is still clumsy.

Right now a user must:

pcutoff = 0.1
import numpy as np
def confidence_to_alpha(c):
    return np.clip((c - pcutoff) / (1 - pcutoff), 0, 1)

then run:

deeplabcut.create_labeled_video(config_path,demovideo , shuffle=1,videotype="mp4", fastmode=True,pcutoff=pcutoff,confidence_to_alpha=confidence_to_alpha, overwrite=True)

Why can we not make this so people can simply pass confidence_to_alpha=[0.1,1] ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants