Skip to content

PyTorch maDLC on 1 indiv creates saDLC tracks which fail in create_labeled_video #2741

@backyardbiomech

Description

@backyardbiomech

Is there an existing issue for this?

  • I have searched the existing issues

Bug description

I have a long-running maDLC project which I recently decided to try with PyTorch. While it is maDLC, it always only tracks a single animal.
The new model trained to 120 epochs with batch_size = 8, and evaluated to pixel errors about 2.1, which is as good (or better) than the last tensorflow model.
Upon analyzing a new video with auto_track=True, I now get three output files:

  • ...shuffle1_snapshot_115.h5
  • ...shuffle1_snapshot_115_full.pickle
  • ...shuffle1_snapshot_115_meta.pickle

So, unlike the previous version, I don't get assemblies or ..._el.h5 for the tracker.

The full.pickle file appears rather empty when I load it directly, and create_video_with_all_detections creates a video with no markers and prints errors for each frame indicating no data exists.
The h5 file appears to be assembled tracks, which is great. However, it doesn't have the _el that downstream methods (like create_labeled_video are looking for. I can add the _el to the name, but the file is also missing the "indiv" level of the multi_index.

So short of changing my config to not be maDLC, the downstream methods don't work. This includes create_labeled_video and filterpredictions at least. Both of which return an error like

Filtering with median model C:\Users\...\videos-raw\cam1_28-Mar-24_18-00-34.mov
No unfiltered data file found in C:\Users\...\videos-raw for video cam1_28-Mar-24_18-00-34 and scorer DLC_DlcrnetStride16Ms5_...shuffle1_snapshot_115 and ellipse tracker.

By the way, if I edit my config file to pretend to be a single-animal project, filtering and creating videos works without a problem.

Operating System

Windows 10

DeepLabCut version

DLC 3.0.0rc4

DeepLabCut mode

multi animal

Device type

gpu

Steps To Reproduce

No response

Relevant log output

No response

Anything else?

No response

Code of Conduct

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions