Skip to content

ValueError while tracking with "calibrate animal assembly" and missing data #2127

@Yizhar-Lab

Description

@Yizhar-Lab

Is there an existing issue for this?

  • I have searched the existing issues

Bug description

The model trained for 2e5 iters using DLC v2.2. Two .pickle files were created:

  1. vid_nameDLC_dlcrnetms5_NLG_RBY_Arrows_B1_MAJan13shuffle1_200000_full.pickle
  2. vid_nameDLC_dlcrnetms5_NLG_RBY_Arrows_B1_MAJan13shuffle1_200000_meta.pickle

Full error message/tracback:

Error message:

C:\Users\Mouse\anaconda3\envs\dlc22\lib\site-packages\tensorflow\python\keras\engine\base_layer_v1.py:1694: UserWarning: layer.apply is deprecated and will be removed in a future version. Please use layer.__call__ method instead.
warnings.warn('layer.apply is deprecated and ’
2023-01-18 12:34:00.345212: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-01-18 12:34:01.128537: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 8060 MB memory: → device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:21:00.0, compute capability: 7.5
2023-01-18 12:34:01.192387: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
0%| | 4/53918 [00:00<24:36, 36.52it/s]2023-01-18 12:34:07.758768: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8700
100%|█████████████████████████████████████████████████████████████████████████| 53918/53918 [58:48<00:00, 15.28it/s]
Traceback (most recent call last):
File “C:\Users\Mouse\anaconda3\envs\dlc22\lib\site-packages\deeplabcut\gui\utils.py”, line 22, in run
self.func()
File “C:\Users\Mouse\anaconda3\envs\dlc22\lib\site-packages\deeplabcut\pose_estimation_tensorflow\predict_videos.py”, line 640, in analyze_videos
convert_detections2tracklets(
File “C:\Users\Mouse\anaconda3\envs\dlc22\lib\site-packages\deeplabcut\pose_estimation_tensorflow\predict_videos.py”, line 1818, in convert_detections2tracklets
ass.calibrate(train_data_file)
File “C:\Users\Mouse\anaconda3\envs\dlc22\lib\site-packages\deeplabcut\pose_estimation_tensorflow\lib\inferenceutils.py”, line 300, in calibrate
dists = np.vstack([pdist(data, “sqeuclidean”) for data in xy])
File “<array_function internals>”, line 180, in vstack
File “C:\Users\Mouse\anaconda3\envs\dlc22\lib\site-packages\numpy\core\shape_base.py”, line 282, in vstack
return _nx.concatenate(arrs, 0)
File “<array_function internals>”, line 180, in concatenate
ValueError: need at least one array to concatenate

Finally running with "calibrate animal assembly=False" solved the problem. Also discussed in the forum.

Operating System

Windows 10

DeepLabCut version

Trained with v2.2
Tracked with v2.3 (similar error with v2.2 as well)

DeepLabCut mode

multi animal

Device type

Nvidia Geforce 2080ti gpu

Steps To Reproduce

No response

Relevant log output

No response

Anything else?

No response

Code of Conduct

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions