Skip to content

Conversation

@n-poulsen
Copy link
Contributor

This PR fixes a memory issue when running video analysis using a model with an identity head, as reported in #2771.

Improvements

  • PadOutputs: instead of needing to set "key": 0 for outputs we don't want to pad (such as identity_heatmap), check if each output is present in the max_individuals dict to see if padding is needed.
  • PredictKeypointIdentities: Models with identity heads would output the identity heatmaps, which leads to memory issues when analyzing large videos. A keep_id_maps=False parameter is added, only keep identity heatmaps in the output when explicitly requested.
  • assign_identity: use the pose confidence to weight identity scores when assigning identity, as done in the TensorFlow implementation

Bug fixes

  • make_pytorch_test_config when there are unique bodyparts for a project. The unique bodyparts need to be included in all_joint_names and all_joints, otherwise video analysis fails.

@n-poulsen n-poulsen merged commit e4ca5bc into pytorch_dlc Nov 14, 2024
@n-poulsen n-poulsen deleted the niels/analyze_videos_with_id_memory_issue branch November 14, 2024 10:43
xiu-cs pushed a commit to xiu-cs/DeepLabCut that referenced this pull request Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DLC3.0🔥 enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants