Skip to content

Unsupervised ID tracking - This function is not implemented for Engine.PYTORCH #2809

@Andrianarivelo

Description

@Andrianarivelo

Is there an existing issue for this?

  • I have searched the existing issues

Bug description

Hello,

We are a lab working on social behavior and we use a lot DLC ma. However identity swap was always an issue and fixing ID swaps takes a lot of our time and even after many iterations we still have issues... We were very happy to see the new unsupervised ID tracking function !

I'm using DLC3.0rc6 with pytorch on videos with 2 mice interacting. The mice are very easy to distinguish by eyes since one has an implant on the head, training and analysis went well with an initial dataset of 1000 frames labeled from 15 videos and two iterations. Looking at example videos, we still have a lot of ID swaps.

I tried to use the Unsupervised ID tracking function pointing at the folder containing the videos and the analysis files and I got this error :
Traceback (most recent call last):

  File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\gui\utils.py", line 25, in run
    self.func()
  File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\pose_tracking_pytorch\apis.py", line 97, in transformer_reID
    deeplabcut.compat.create_tracking_dataset(
  File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\compat.py", line 941, in create_tracking_dataset
    raise NotImplementedError(f"This function is not implemented for {engine}")
NotImplementedError: This function is not implemented for Engine.PYTORCH

I tried to train a Tensorflow network, analysed the videos with it but I got this error with the GUI frozen for hours :
Traceback (most recent call last):

File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\gui\utils.py", line 25, in run
   self.func()
 File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\pose_tracking_pytorch\apis.py", line 97, in transformer_reID
   deeplabcut.compat.create_tracking_dataset(
 File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\compat.py", line 922, in create_tracking_dataset
   return create_tracking_dataset(
 File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\pose_estimation_tensorflow\predict_videos.py", line 208, in create_tracking_dataset
   extract_bpt_feature_from_video(
 File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\pose_estimation_tensorflow\predict_multianimal.py", line 104, in extract_bpt_feature_from_video
   raise NotImplementedError(
NotImplementedError: Not implemented yet, please raise an GitHub issue if you need this.
INFO:console:900.7
INFO:console:
INFO:console:, recorded with
INFO:console:
INFO:console:60.0
INFO:console:
INFO:console:fps!
INFO:console:Overall # of frames:
INFO:console:
INFO:console:54042
INFO:console:
INFO:console: found with (before cropping) frame dimensions:
INFO:console:
INFO:console:800
INFO:console:
INFO:console:600
INFO:console:Starting to extract posture

Could you please help us ! Would be great for us working on social behavior to have a way to easily fix identity swaps, its very time consumming to do it frame by frame.

Thanks

Operating System

Windows 11

DeepLabCut version

dlc version 3.0rc6

DeepLabCut mode

multi animal

Device type

gpu RTX 3060 12Gb

Steps To Reproduce

1-Activate DLC environment
2-run gui
3-load project
-in unsupervised ID tracking points to videos (analysis results files are in the folder), choose shuffle, number of animals 3, number of triplets 1000

Relevant log output

(dcl3) PS C:\Users\bellone> python -m deeplabcut
Loading DLC 3.0.0rc6...
Starting GUI...
QPixmap::scaled: Pixmap is a null pixmap
Traceback (most recent call last):
  File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\gui\utils.py", line 25, in run
    self.func()
  File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\pose_tracking_pytorch\apis.py", line 97, in transformer_reID
    deeplabcut.compat.create_tracking_dataset(
  File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\compat.py", line 941, in create_tracking_dataset
    raise NotImplementedError(f"This function is not implemented for {engine}")
NotImplementedError: This function is not implemented for Engine.PYTORCH



(dcl3) PS C:\Users\bellone> python -m deeplabcut
Loading DLC 3.0.0rc6...
Starting GUI...
QPixmap::scaled: Pixmap is a null pixmap
QPixmap::scaled: Pixmap is a null pixmap
INFO:console:Using snapshot-200000
INFO:console:
INFO:console:for model
INFO:console:
INFO:console:D:\Models\DLC-model\25-11-2024-free-interaction-NPX-CB-andry-2024-11-25\dlc-models\iteration-0\25-11-2024-free-interaction-NPX-CBNov25-trainset95shuffle2
C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\tensorflow\python\keras\engine\base_layer_v1.py:1694: UserWarning: `layer.apply` is deprecated and will be removed in a future version. Please use `layer.__call__` method instead.
  warnings.warn('`layer.apply` is deprecated and '
INFO:console:Activating extracting of PAFs
INFO:console:Starting to analyze %
INFO:console:
INFO:console:D:\Models\DLC-model\25-11-2024-free-interaction-NPX-CB-andry-2024-11-25\videos\converted_Trial    10.mp4
INFO:console:Loading
INFO:console:
INFO:console:D:\Models\DLC-model\25-11-2024-free-interaction-NPX-CB-andry-2024-11-25\videos\converted_Trial    10.mp4
INFO:console:Duration of video [s]:
INFO:console:
Traceback (most recent call last):
  File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\gui\utils.py", line 25, in run
    self.func()
  File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\pose_tracking_pytorch\apis.py", line 97, in transformer_reID
    deeplabcut.compat.create_tracking_dataset(
  File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\compat.py", line 922, in create_tracking_dataset
    return create_tracking_dataset(
  File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\pose_estimation_tensorflow\predict_videos.py", line 208, in create_tracking_dataset
    extract_bpt_feature_from_video(
  File "C:\Users\bellone\.conda\envs\dcl3\lib\site-packages\deeplabcut\pose_estimation_tensorflow\predict_multianimal.py", line 104, in extract_bpt_feature_from_video
    raise NotImplementedError(
NotImplementedError: Not implemented yet, please raise an GitHub issue if you need this.
INFO:console:900.7
INFO:console:
INFO:console:, recorded with
INFO:console:
INFO:console:60.0
INFO:console:
INFO:console:fps!
INFO:console:Overall # of frames:
INFO:console:
INFO:console:54042
INFO:console:
INFO:console: found with (before cropping) frame dimensions:
INFO:console:
INFO:console:800
INFO:console:
INFO:console:600
INFO:console:Starting to extract posture

Anything else?

No response

Code of Conduct

Metadata

Metadata

Assignees

Labels

DLC3.0🔥WORK IN PROGRESS!developers are currently working on this feature... stay tuned.reID

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions