Failing tests on github and local run `./test.sh`

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Bug description

Would like an "all hands on deck" to this 👍 :

When running `./test/sh` locally I find the following sets of errors: 

- [ ] 3d isn't tested properly, and is passed over: 
```Loading DLC 2.3rc1...
Imported DLC!
Traceback (most recent call last):
  File "/Users/mwmathis/Documents/DeepLabCut/examples/testscript_3d.py", line 95, in <module>
    config = glob.glob(os.path.join(basepath, "TEST*", "config.yaml"))[-1]
IndexError: list index out of range
```
- [ ] in the multi-animal test script, there are several issues; (1) ffmpeg issue with plots (which might be the source of the github failing builds, btw), and (2) data missing: 
<details>

```
Analyzing data...
3it [00:00,  3.07it/s]
Saving plots...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:05<00:00,  1.69s/it]
/bin/sh: ffmpeg: command not found
Analyzing video...
Config:
{'all_joints': [[0], [1], [2], [3], [4]],
 'all_joints_names': ['bodypart1',
                      'bodypart2',
                      'bodypart3',
                      'corner1',
                      'corner2'],
 'batch_size': 1,
 'crop_pad': 0,
 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_multi_mouseOct9/multi_mouse_dlc_team80shuffle1.pickle',
 'dataset_type': 'multi-animal-imgaug',
 'deterministic': False,
 'fg_fraction': 0.25,
 'global_scale': 0.8,
 'init_weights': '/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/efficientnet-b0/model.ckpt',
 'intermediate_supervision': False,
 'intermediate_supervision_layer': 12,
 'location_refinement': True,
 'locref_huber_loss': True,
 'locref_loss_weight': 1.0,
 'locref_smooth': False,
 'locref_stdev': 7.2801,
 'log_dir': 'log',
 'mean_pixel': [123.68, 116.779, 103.939],
 'minconfidence': 0.01,
 'mirror': False,
 'multi_stage': False,
 'net_type': 'efficientnet-b0',
 'nmsradius': 5.0,
 'num_idchannel': 3,
 'num_joints': 5,
 'num_limbs': 3,
 'optimizer': 'sgd',
 'paf_best': [1, 0],
 'pairwise_huber_loss': True,
 'pairwise_predict': False,
 'partaffinityfield_graph': [[0, 1], [0, 2], [1, 2]],
 'partaffinityfield_predict': True,
 'regularize': False,
 'scoremap_dir': 'test',
 'shuffle': True,
 'sigma': 1,
 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/dlc-models/iteration-0/multi_mouseOct9-trainset80shuffle1/test/snapshot',
 'stride': 8.0,
 'weigh_negatives': False,
 'weigh_only_present_joints': False,
 'weigh_part_predictions': False,
 'weight_decay': 0.0001}
Using snapshot-5 for model /Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/dlc-models/iteration-0/multi_mouseOct9-trainset80shuffle1
Activating extracting of PAFs
No video(s) were found. Please check your paths and/or 'video_type'.
Video analyzed.
Create video with all detections...
Video created.
Convert detections to tracklets...
Config:
{'all_joints': [[0], [1], [2], [3], [4]],
 'all_joints_names': ['bodypart1',
                      'bodypart2',
                      'bodypart3',
                      'corner1',
                      'corner2'],
 'batch_size': 1,
 'crop_pad': 0,
 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_multi_mouseOct9/multi_mouse_dlc_team80shuffle1.pickle',
 'dataset_type': 'multi-animal-imgaug',
 'deterministic': False,
 'fg_fraction': 0.25,
 'global_scale': 0.8,
 'init_weights': '/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/efficientnet-b0/model.ckpt',
 'intermediate_supervision': False,
 'intermediate_supervision_layer': 12,
 'location_refinement': True,
 'locref_huber_loss': True,
 'locref_loss_weight': 1.0,
 'locref_smooth': False,
 'locref_stdev': 7.2801,
 'log_dir': 'log',
 'mean_pixel': [123.68, 116.779, 103.939],
 'minconfidence': 0.01,
 'mirror': False,
 'multi_stage': False,
 'net_type': 'efficientnet-b0',
 'nmsradius': 5.0,
 'num_idchannel': 3,
 'num_joints': 5,
 'num_limbs': 3,
 'optimizer': 'sgd',
 'paf_best': [1, 0],
 'pairwise_huber_loss': True,
 'pairwise_predict': False,
 'partaffinityfield_graph': [[0, 1], [0, 2], [1, 2]],
 'partaffinityfield_predict': True,
 'regularize': False,
 'scoremap_dir': 'test',
 'shuffle': True,
 'sigma': 1,
 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/dlc-models/iteration-0/multi_mouseOct9-trainset80shuffle1/test/snapshot',
 'stride': 8.0,
 'weigh_negatives': False,
 'weigh_only_present_joints': False,
 'weigh_part_predictions': False,
 'weight_decay': 0.0001}
Using snapshot-5 for model /Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/dlc-models/iteration-0/multi_mouseOct9-trainset80shuffle1
No video(s) found. Please check your path!
Tracklets created...
No video(s) found. Please check your path!
Plotting trajectories...
No videos found. Make sure you passed a list of videos and that *videotype* is right.
Trajectory plotted.
Creating labeled video...
Labeled video created.
Filtering predictions...
No video(s) were found. Please check your paths and/or 'videotype'.
Predictions filtered.
Extracting outlier frames...
No suitable videos found in ['/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/videos/m3v1mp4short.mp4']
Outlier frames extracted.
RELABELING
Traceback (most recent call last):
  File "/Users/mwmathis/Documents/DeepLabCut/examples/testscript_multianimal.py", line 232, in <module>
    DF = pd.read_hdf(file, "df_with_missing")
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/pandas/io/pytables.py", line 414, in read_hdf
    raise FileNotFoundError(f"File {path_or_buf} does not exist")
FileNotFoundError: File /Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/labeled-data/m3v1mp4short/machinelabels-iter0.h5 does not exist
```
</details>

### Operating System

macOS 12.5.1, new conda env based on `DEEPLABCUT_newGUI` (see #1984 for build instructions).

### DeepLabCut version

2.3rc1

from test.sh:

```
Loading DLC 2.3rc1...
Imported DLC!
```

### DeepLabCut mode

single animal, multi-animal, 3d

### Device type

CPU 

### Steps To Reproduce

1. in conda env with latest 2.3rc1 installed, run `./test.sh`

### Relevant log output

<details>

```
Loading DLC 2.3rc1...
Imported DLC!
On Windows/OSX tensorpack is not tested by default.
CREATING PROJECT
Created "/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/videos"
Created "/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/labeled-data"
Created "/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/training-datasets"
Created "/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/dlc-models"
Copying the videos
/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/videos/reachingvideo1.avi
Generated "/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/config.yaml"

A new project with name TEST-Alex-2022-10-09 is created at /Users/mwmathis/Documents/DeepLabCut/examples and a configurable file (config.yaml) is stored there. Change the parameters in this file to adapt to your project's needs.
 Once you have changed the configuration file, use the function 'extract_frames' to select frames for labeling.
. [OPTIONAL] Use the function 'add_new_videos' to add new videos to your project (at any stage).
EXTRACTING FRAMES
Config file read successfully.
Extracting frames based on kmeans ...
Kmeans-quantization based extracting of frames from 0.0  seconds to 8.53  seconds.
Extracting and downsampling... 256  frames from the video.
256it [00:01, 149.29it/s]
Kmeans clustering ... (this might take a while)
Frames were successfully extracted, for the videos listed in the config.yaml file.

You can now label the frames using the function 'label_frames' (Note, you should label frames extracted from diverse videos (and many videos; we do not recommend training on single videos!)).
CREATING-SOME LABELS FOR THE FRAMES
Plot labels...
Creating images with labels by Alex.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:01<00:00,  2.97it/s]
If all the labels are ok, then use the function 'create_training_dataset' to create the training dataset!
CREATING TRAININGSET
Downloading a ImageNet-pretrained model from http://download.tensorflow.org/models/resnet_v1_50_2016_08_28.tar.gz....
The training dataset is successfully created. Use the function 'train_network' to start training. Happy training!
CHANGING training parameters to end quickly!
TRAIN
Selecting single-animal trainer
Config:
{'all_joints': [[0], [1], [2], [3]],
 'all_joints_names': ['bodypart1', 'bodypart2', 'bodypart3', 'objectA'],
 'alpha_r': 0.02,
 'apply_prob': 0.5,
 'batch_size': 1,
 'contrast': {'clahe': True,
              'claheratio': 0.1,
              'histeq': True,
              'histeqratio': 0.1},
 'convolution': {'edge': False,
                 'emboss': {'alpha': [0.0, 1.0], 'strength': [0.5, 1.5]},
                 'embossratio': 0.1,
                 'sharpen': False,
                 'sharpenratio': 0.3},
 'crop_pad': 0,
 'cropratio': 0.4,
 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_TESTOct9/TEST_Alex80shuffle1.mat',
 'dataset_type': 'default',
 'decay_steps': 30000,
 'deterministic': False,
 'display_iters': 2,
 'fg_fraction': 0.25,
 'global_scale': 0.8,
 'init_weights': '/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/resnet_v1_50.ckpt',
 'intermediate_supervision': False,
 'intermediate_supervision_layer': 12,
 'location_refinement': True,
 'locref_huber_loss': True,
 'locref_loss_weight': 0.05,
 'locref_stdev': 7.2801,
 'log_dir': 'log',
 'lr_init': 0.0005,
 'max_input_size': 1500,
 'mean_pixel': [123.68, 116.779, 103.939],
 'metadataset': 'training-datasets/iteration-0/UnaugmentedDataSet_TESTOct9/Documentation_data-TEST_80shuffle1.pickle',
 'min_input_size': 64,
 'mirror': False,
 'multi_stage': False,
 'multi_step': [[0.001, 5]],
 'net_type': 'resnet_50',
 'num_joints': 4,
 'optimizer': 'sgd',
 'pairwise_huber_loss': False,
 'pairwise_predict': False,
 'partaffinityfield_predict': False,
 'pos_dist_thresh': 17,
 'project_path': '/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09',
 'regularize': False,
 'rotation': 25,
 'rotratio': 0.4,
 'save_iters': 5,
 'scale_jitter_lo': 0.5,
 'scale_jitter_up': 1.25,
 'scoremap_dir': 'test',
 'shuffle': True,
 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/dlc-models/iteration-0/TESTOct9-trainset80shuffle1/train/snapshot',
 'stride': 8.0,
 'weigh_negatives': False,
 'weigh_only_present_joints': False,
 'weigh_part_predictions': False,
 'weight_decay': 0.0001}
Batch Size is 1
/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer_v1.py:1694: UserWarning: `layer.apply` is deprecated and will be removed in a future version. Please use `layer.__call__` method instead.
  warnings.warn('`layer.apply` is deprecated and '
2022-10-09 16:15:10.800158: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Loading ImageNet-pretrained resnet_50
2022-10-09 16:15:12.309192: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
Training parameter:
{'stride': 8.0, 'weigh_part_predictions': False, 'weigh_negatives': False, 'fg_fraction': 0.25, 'mean_pixel': [123.68, 116.779, 103.939], 'shuffle': True, 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/dlc-models/iteration-0/TESTOct9-trainset80shuffle1/train/snapshot', 'log_dir': 'log', 'global_scale': 0.8, 'location_refinement': True, 'locref_stdev': 7.2801, 'locref_loss_weight': 0.05, 'locref_huber_loss': True, 'optimizer': 'sgd', 'intermediate_supervision': False, 'intermediate_supervision_layer': 12, 'regularize': False, 'weight_decay': 0.0001, 'crop_pad': 0, 'scoremap_dir': 'test', 'batch_size': 1, 'dataset_type': 'default', 'deterministic': False, 'mirror': False, 'pairwise_huber_loss': False, 'weigh_only_present_joints': False, 'partaffinityfield_predict': False, 'pairwise_predict': False, 'all_joints': [[0], [1], [2], [3]], 'all_joints_names': ['bodypart1', 'bodypart2', 'bodypart3', 'objectA'], 'alpha_r': 0.02, 'apply_prob': 0.5, 'contrast': {'clahe': True, 'claheratio': 0.1, 'histeq': True, 'histeqratio': 0.1, 'gamma': False, 'sigmoid': False, 'log': False, 'linear': False}, 'convolution': {'edge': False, 'emboss': {'alpha': [0.0, 1.0], 'strength': [0.5, 1.5]}, 'embossratio': 0.1, 'sharpen': False, 'sharpenratio': 0.3}, 'cropratio': 0.4, 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_TESTOct9/TEST_Alex80shuffle1.mat', 'decay_steps': 30000, 'display_iters': 2, 'init_weights': '/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/resnet_v1_50.ckpt', 'lr_init': 0.0005, 'max_input_size': 1500, 'metadataset': 'training-datasets/iteration-0/UnaugmentedDataSet_TESTOct9/Documentation_data-TEST_80shuffle1.pickle', 'min_input_size': 64, 'multi_stage': False, 'multi_step': [[0.001, 5]], 'net_type': 'resnet_50', 'num_joints': 4, 'pos_dist_thresh': 17, 'project_path': '/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09', 'rotation': 25, 'rotratio': 0.4, 'save_iters': 5, 'scale_jitter_lo': 0.5, 'scale_jitter_up': 1.25, 'covering': True, 'elastic_transform': True, 'motion_blur': True, 'motion_blur_params': {'k': 7, 'angle': (-90, 90)}}
Starting training....
iteration: 2 loss: 1.2213 lr: 0.001
iteration: 4 loss: 0.7012 lr: 0.001
2022-10-09 16:15:30.177725: W tensorflow/core/kernels/queue_base.cc:277] _0_fifo_queue: Skipping cancelled enqueue attempt with queue not closed
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 1378, in _do_call
    return fn(*args)
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 1361, in _run_fn
    return self._call_tf_sessionrun(options, feed_dict, fetch_list,
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 1454, in _call_tf_sessionrun
    return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,
tensorflow.python.framework.errors_impl.CancelledError: Enqueue operation was cancelled
	 [[{{node fifo_queue_enqueue}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/core/train.py", line 83, in load_and_enqueue
    sess.run(enqueue_op, feed_dict=food)
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 968, in run
    result = self._run(None, fetches, feed_dict, options_ptr,
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 1191, in _run
    results = self._do_run(handle, final_targets, final_fetches,
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 1371, in _do_run
    return self._do_call(_run_fn, feeds, fetches, targets, options,
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 1397, in _do_call
    raise type(e)(node_def, op, message)  # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors_impl.CancelledError: Graph execution error:

Detected at node 'fifo_queue_enqueue' defined at (most recent call last):
    File "/Users/mwmathis/Documents/DeepLabCut/examples/testscript.py", line 164, in <module>
      deeplabcut.train_network(path_config_file)
    File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/training.py", line 210, in train_network
      train(
    File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/core/train.py", line 168, in train
      batch, enqueue_op, placeholders = setup_preloading(batch_spec)
    File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/core/train.py", line 69, in setup_preloading
      enqueue_op = q.enqueue(placeholders_list)
Node: 'fifo_queue_enqueue'
Enqueue operation was cancelled
	 [[{{node fifo_queue_enqueue}}]]

Original stack trace for 'fifo_queue_enqueue':
  File "/Users/mwmathis/Documents/DeepLabCut/examples/testscript.py", line 164, in <module>
    deeplabcut.train_network(path_config_file)
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/training.py", line 210, in train_network
    train(
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/core/train.py", line 168, in train
    batch, enqueue_op, placeholders = setup_preloading(batch_spec)
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/core/train.py", line 69, in setup_preloading
    enqueue_op = q.enqueue(placeholders_list)
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/ops/data_flow_ops.py", line 346, in enqueue
    return gen_data_flow_ops.queue_enqueue_v2(
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 4063, in queue_enqueue_v2
    _, _, _op, _outputs = _op_def_library._apply_op_helper(
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/framework/op_def_library.py", line 797, in _apply_op_helper
    op = g._create_op_internal(op_type_name, inputs, dtypes=None,
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/framework/ops.py", line 3800, in _create_op_internal
    ret = Operation(

The network is now trained and ready to evaluate. Use the function 'evaluate_network' to evaluate the network.
EVALUATE
Config:
{'all_joints': [[0], [1], [2], [3]],
 'all_joints_names': ['bodypart1', 'bodypart2', 'bodypart3', 'objectA'],
 'batch_size': 1,
 'crop_pad': 0,
 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_TESTOct9/TEST_Alex80shuffle1.mat',
 'dataset_type': 'imgaug',
 'deterministic': False,
 'fg_fraction': 0.25,
 'global_scale': 0.8,
 'init_weights': '/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/resnet_v1_50.ckpt',
 'intermediate_supervision': False,
 'intermediate_supervision_layer': 12,
 'location_refinement': True,
 'locref_huber_loss': True,
 'locref_loss_weight': 1.0,
 'locref_stdev': 7.2801,
 'log_dir': 'log',
 'mean_pixel': [123.68, 116.779, 103.939],
 'mirror': False,
 'net_type': 'resnet_50',
 'num_joints': 4,
 'optimizer': 'sgd',
 'pairwise_huber_loss': True,
 'pairwise_predict': False,
 'partaffinityfield_predict': False,
 'regularize': False,
 'scoremap_dir': 'test',
 'shuffle': True,
 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/dlc-models/iteration-0/TESTOct9-trainset80shuffle1/test/snapshot',
 'stride': 8.0,
 'weigh_negatives': False,
 'weigh_only_present_joints': False,
 'weigh_part_predictions': False,
 'weight_decay': 0.0001}
Running  DLC_resnet50_TESTOct9shuffle1_5  with # of training iterations: 5
/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer_v1.py:1694: UserWarning: `layer.apply` is deprecated and will be removed in a future version. Please use `layer.__call__` method instead.
  warnings.warn('`layer.apply` is deprecated and '
Running evaluation ...
5it [00:04,  1.13it/s]
Analysis is done and the results are stored (see evaluation-results) for snapshot:  snapshot-5
Results for 5  training iterations: 80 1 train error: 413.96 pixels. Test error: 448.53  pixels.
With pcutoff of 0.01  train error: 413.96 pixels. Test error: 448.53 pixels
Thereby, the errors are given by the average distances between the labels by DLC and the scorer.
Plotting...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:01<00:00,  2.55it/s]
The network is evaluated and the results are stored in the subdirectory 'evaluation_results'.
Please check the results, then choose the best model (snapshot) for prediction. You can update the config.yaml file with the appropriate index for the 'snapshotindex'.
Use the function 'analyze_video' to make predictions on new videos.
Otherwise, consider adding more labeled-data and retraining the network (see DeepLabCut workflow Fig 2, Nath 2019)
CUT SHORT VIDEO AND ANALYZE (with dynamic cropping!)
/bin/sh: ffmpeg: command not found
Config:
{'all_joints': [[0], [1], [2], [3]],
 'all_joints_names': ['bodypart1', 'bodypart2', 'bodypart3', 'objectA'],
 'batch_size': 1,
 'crop_pad': 0,
 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_TESTOct9/TEST_Alex80shuffle1.mat',
 'dataset_type': 'imgaug',
 'deterministic': False,
 'fg_fraction': 0.25,
 'global_scale': 0.8,
 'init_weights': '/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/resnet_v1_50.ckpt',
 'intermediate_supervision': False,
 'intermediate_supervision_layer': 12,
 'location_refinement': True,
 'locref_huber_loss': True,
 'locref_loss_weight': 1.0,
 'locref_stdev': 7.2801,
 'log_dir': 'log',
 'mean_pixel': [123.68, 116.779, 103.939],
 'mirror': False,
 'net_type': 'resnet_50',
 'num_joints': 4,
 'optimizer': 'sgd',
 'pairwise_huber_loss': True,
 'pairwise_predict': False,
 'partaffinityfield_predict': False,
 'regularize': False,
 'scoremap_dir': 'test',
 'shuffle': True,
 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/dlc-models/iteration-0/TESTOct9-trainset80shuffle1/test/snapshot',
 'stride': 8.0,
 'weigh_negatives': False,
 'weigh_only_present_joints': False,
 'weigh_part_predictions': False,
 'weight_decay': 0.0001}
Using snapshot-5 for model /Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/dlc-models/iteration-0/TESTOct9-trainset80shuffle1
Starting analysis in dynamic cropping mode with parameters: (True, 0.1, 5)
Switching batchsize to 1, num_outputs (per animal) to 1 and TFGPUinference to False (all these features are not supported in this mode).
/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer_v1.py:1694: UserWarning: `layer.apply` is deprecated and will be removed in a future version. Please use `layer.__call__` method instead.
  warnings.warn('`layer.apply` is deprecated and '
No video(s) were found. Please check your paths and/or 'video_type'.
analyze again...
Config:
{'all_joints': [[0], [1], [2], [3]],
 'all_joints_names': ['bodypart1', 'bodypart2', 'bodypart3', 'objectA'],
 'batch_size': 1,
 'crop_pad': 0,
 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_TESTOct9/TEST_Alex80shuffle1.mat',
 'dataset_type': 'imgaug',
 'deterministic': False,
 'fg_fraction': 0.25,
 'global_scale': 0.8,
 'init_weights': '/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/resnet_v1_50.ckpt',
 'intermediate_supervision': False,
 'intermediate_supervision_layer': 12,
 'location_refinement': True,
 'locref_huber_loss': True,
 'locref_loss_weight': 1.0,
 'locref_stdev': 7.2801,
 'log_dir': 'log',
 'mean_pixel': [123.68, 116.779, 103.939],
 'mirror': False,
 'net_type': 'resnet_50',
 'num_joints': 4,
 'optimizer': 'sgd',
 'pairwise_huber_loss': True,
 'pairwise_predict': False,
 'partaffinityfield_predict': False,
 'regularize': False,
 'scoremap_dir': 'test',
 'shuffle': True,
 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/dlc-models/iteration-0/TESTOct9-trainset80shuffle1/test/snapshot',
 'stride': 8.0,
 'weigh_negatives': False,
 'weigh_only_present_joints': False,
 'weigh_part_predictions': False,
 'weight_decay': 0.0001}
Using snapshot-5 for model /Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/dlc-models/iteration-0/TESTOct9-trainset80shuffle1
/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer_v1.py:1694: UserWarning: `layer.apply` is deprecated and will be removed in a future version. Please use `layer.__call__` method instead.
  warnings.warn('`layer.apply` is deprecated and '
No video(s) were found. Please check your paths and/or 'video_type'.
CREATE VIDEO
Making plots
No videos found. Make sure you passed a list of videos and that *videotype* is right.
EXTRACT OUTLIERS
No suitable videos found in ['/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/videos/reachingvideo1short.avi']
No suitable videos found in ['/Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/videos/reachingvideo1short.avi']
RELABELING
Traceback (most recent call last):
  File "/Users/mwmathis/Documents/DeepLabCut/examples/testscript.py", line 245, in <module>
    DF = pd.read_hdf(file, "df_with_missing")
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/pandas/io/pytables.py", line 414, in read_hdf
    raise FileNotFoundError(f"File {path_or_buf} does not exist")
FileNotFoundError: File /Users/mwmathis/Documents/DeepLabCut/examples/TEST-Alex-2022-10-09/labeled-data/reachingvideo1short/machinelabels-iter0.h5 does not exist
2022-10-09 16:15:50.808057: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Loading DLC 2.3rc1...
Imported DLC!
Traceback (most recent call last):
  File "/Users/mwmathis/Documents/DeepLabCut/examples/testscript_3d.py", line 95, in <module>
    config = glob.glob(os.path.join(basepath, "TEST*", "config.yaml"))[-1]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/mwmathis/Documents/DeepLabCut/examples/testscript_3d.py", line 97, in <module>
    raise RuntimeError("Please run the testscript.py first before testing for 3d")
RuntimeError: Please run the testscript.py first before testing for 3d
2022-10-09 16:15:57.497719: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Loading DLC 2.3rc1...
Creating project...
Created "/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/videos"
Created "/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/labeled-data"
Created "/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/training-datasets"
Created "/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/dlc-models"
Copying the videos
/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/videos/m3v1mp4.mp4
Generated "/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/config.yaml"

A new project with name multi_mouse-dlc_team-2022-10-09 is created at /Users/mwmathis/Documents/DeepLabCut/examples and a configurable file (config.yaml) is stored there. Change the parameters in this file to adapt to your project's needs.
 Once you have changed the configuration file, use the function 'extract_frames' to select frames for labeling.
. [OPTIONAL] Use the function 'add_new_videos' to add new videos to your project (at any stage).
Project created.
Editing config...
Config edited.
Extracting frames...
Config file read successfully.
Extracting frames based on kmeans ...
Kmeans-quantization based extracting of frames from 0.0  seconds to 77.67  seconds.
Extracting and downsampling... 2330  frames from the video.
2330it [00:07, 323.28it/s]
Kmeans clustering ... (this might take a while)
Frames were successfully extracted, for the videos listed in the config.yaml file.

You can now label the frames using the function 'label_frames' (Note, you should label frames extracted from diverse videos (and many videos; we do not recommend training on single videos!)).
Frames extracted.
Creating artificial data...
Artificial data created.
Checking labels...
Creating images with labels by dlc_team.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00,  6.66it/s]
If all the labels are ok, then use the function 'create_training_dataset' to create the training dataset!
Labels checked.
Creating train dataset...
Utilizing the following graph: [[0, 1], [0, 2], [1, 2]]
Downloading a ImageNet-pretrained model from https://storage.googleapis.com/cloud-tpu-checkpoints/efficientnet/ckptsaug/efficientnet-b0.tar.gz....
Creating training data for: Shuffle: 1 TrainFraction:  0.8
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 1824.01it/s]
The training dataset is successfully created. Use the function 'train_network' to start training. Happy training!
Train dataset created.
Editing pose config...
Pose config edited.
Training network...
Selecting multi-animal trainer
Config:
{'all_joints': [[0], [1], [2], [3], [4]],
 'all_joints_names': ['bodypart1',
                      'bodypart2',
                      'bodypart3',
                      'corner1',
                      'corner2'],
 'alpha_r': 0.02,
 'apply_prob': 0.5,
 'batch_size': 1,
 'contrast': {'clahe': True,
              'claheratio': 0.1,
              'histeq': True,
              'histeqratio': 0.1},
 'convolution': {'edge': False,
                 'emboss': {'alpha': [0.0, 1.0], 'strength': [0.5, 1.5]},
                 'embossratio': 0.1,
                 'sharpen': False,
                 'sharpenratio': 0.3},
 'crop_pad': 0,
 'crop_sampling': 'hybrid',
 'crop_size': [200, 200],
 'cropratio': 0.4,
 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_multi_mouseOct9/multi_mouse_dlc_team80shuffle1.pickle',
 'dataset_type': 'multi-animal-imgaug',
 'decay_steps': 30000,
 'deterministic': False,
 'display_iters': 2,
 'fg_fraction': 0.25,
 'global_scale': 0.5,
 'init_weights': '/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/efficientnet-b0/model.ckpt',
 'intermediate_supervision': False,
 'intermediate_supervision_layer': 12,
 'location_refinement': True,
 'locref_huber_loss': True,
 'locref_loss_weight': 0.05,
 'locref_stdev': 7.2801,
 'log_dir': 'log',
 'lr_init': 0.0005,
 'max_input_size': 1500,
 'max_shift': 0.4,
 'mean_pixel': [123.68, 116.779, 103.939],
 'metadataset': 'training-datasets/iteration-0/UnaugmentedDataSet_multi_mouseOct9/Documentation_data-multi_mouse_80shuffle1.pickle',
 'min_input_size': 64,
 'mirror': False,
 'multi_stage': False,
 'multi_step': [[0.0001, 7500], [5e-05, 12000], [1e-05, 200000]],
 'net_type': 'efficientnet-b0',
 'num_idchannel': 3,
 'num_joints': 5,
 'num_limbs': 3,
 'optimizer': 'adam',
 'pafwidth': 20,
 'pairwise_huber_loss': False,
 'pairwise_loss_weight': 0.1,
 'pairwise_predict': False,
 'partaffinityfield_graph': [[0, 1], [0, 2], [1, 2]],
 'partaffinityfield_predict': True,
 'pos_dist_thresh': 17,
 'pre_resize': [],
 'project_path': '/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09',
 'regularize': False,
 'rotation': 25,
 'rotratio': 0.4,
 'save_iters': 5,
 'scale_jitter_lo': 0.5,
 'scale_jitter_up': 1.25,
 'scoremap_dir': 'test',
 'shuffle': True,
 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/dlc-models/iteration-0/multi_mouseOct9-trainset80shuffle1/train/snapshot',
 'stride': 8.0,
 'weigh_negatives': False,
 'weigh_only_present_joints': False,
 'weigh_part_predictions': False,
 'weight_decay': 0.0001}
Activating limb prediction...
Batch Size is 1
Getting specs multi-animal-imgaug 3 5
/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer_v1.py:1694: UserWarning: `layer.apply` is deprecated and will be removed in a future version. Please use `layer.__call__` method instead.
  warnings.warn('`layer.apply` is deprecated and '
2022-10-09 16:16:21.945126: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Loading ImageNet-pretrained efficientnet-b0
2022-10-09 16:16:22.723005: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
Switching to cosine decay schedule with adam!
Max_iters overwritten as 5
Training parameters:
{'stride': 8.0, 'weigh_part_predictions': False, 'weigh_negatives': False, 'fg_fraction': 0.25, 'mean_pixel': [123.68, 116.779, 103.939], 'shuffle': True, 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/dlc-models/iteration-0/multi_mouseOct9-trainset80shuffle1/train/snapshot', 'log_dir': 'log', 'global_scale': 0.5, 'location_refinement': True, 'locref_stdev': 7.2801, 'locref_loss_weight': 0.05, 'locref_huber_loss': True, 'optimizer': 'adam', 'intermediate_supervision': False, 'intermediate_supervision_layer': 12, 'regularize': False, 'weight_decay': 0.0001, 'crop_pad': 0, 'scoremap_dir': 'test', 'batch_size': 1, 'dataset_type': 'multi-animal-imgaug', 'deterministic': False, 'mirror': False, 'pairwise_huber_loss': False, 'weigh_only_present_joints': False, 'partaffinityfield_predict': True, 'pairwise_predict': True, 'all_joints': [[0], [1], [2], [3], [4]], 'all_joints_names': ['bodypart1', 'bodypart2', 'bodypart3', 'corner1', 'corner2'], 'alpha_r': 0.02, 'apply_prob': 0.5, 'contrast': {'clahe': True, 'claheratio': 0.1, 'histeq': True, 'histeqratio': 0.1, 'gamma': False, 'sigmoid': False, 'log': False, 'linear': False}, 'convolution': {'edge': False, 'emboss': {'alpha': [0.0, 1.0], 'strength': [0.5, 1.5]}, 'embossratio': 0.1, 'sharpen': False, 'sharpenratio': 0.3}, 'crop_sampling': 'hybrid', 'crop_size': [200, 200], 'cropratio': 0.4, 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_multi_mouseOct9/multi_mouse_dlc_team80shuffle1.pickle', 'decay_steps': 30000, 'display_iters': 2, 'init_weights': '/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/efficientnet-b0/model.ckpt', 'lr_init': 0.0005, 'max_input_size': 1500, 'max_shift': 0.4, 'metadataset': 'training-datasets/iteration-0/UnaugmentedDataSet_multi_mouseOct9/Documentation_data-multi_mouse_80shuffle1.pickle', 'min_input_size': 64, 'multi_stage': False, 'multi_step': [[0.0001, 7500], [5e-05, 12000], [1e-05, 200000]], 'net_type': 'efficientnet-b0', 'num_idchannel': 3, 'num_joints': 5, 'num_limbs': 3, 'pafwidth': 20, 'pairwise_loss_weight': 0.1, 'partaffinityfield_graph': [[0, 1], [0, 2], [1, 2]], 'pos_dist_thresh': 17, 'pre_resize': [], 'project_path': '/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09', 'rotation': 25, 'rotratio': 0.4, 'save_iters': 5, 'scale_jitter_lo': 0.5, 'scale_jitter_up': 1.25, 'use_batch_norm': False, 'use_drop_out': False}
Starting multi-animal training....
iteration: 2 loss: 1.2181 scmap loss: 0.9926 locref loss: 0.0333 limb loss: 0.1922 lr: 0.0005000000237487257
iteration: 4 loss: 0.3450 scmap loss: 0.2988 locref loss: 0.0110 limb loss: 0.0352 lr: 0.0005000000237487257
2022-10-09 16:16:32.490305: W tensorflow/core/kernels/queue_base.cc:277] _0_fifo_queue: Skipping cancelled enqueue attempt with queue not closed
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 1378, in _do_call
    return fn(*args)
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 1361, in _run_fn
    return self._call_tf_sessionrun(options, feed_dict, fetch_list,
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 1454, in _call_tf_sessionrun
    return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,
tensorflow.python.framework.errors_impl.CancelledError: Enqueue operation was cancelled
	 [[{{node fifo_queue_enqueue}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/core/train.py", line 83, in load_and_enqueue
    sess.run(enqueue_op, feed_dict=food)
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 968, in run
    result = self._run(None, fetches, feed_dict, options_ptr,
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 1191, in _run
    results = self._do_run(handle, final_targets, final_fetches,
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 1371, in _do_run
    return self._do_call(_run_fn, feeds, fetches, targets, options,
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/client/session.py", line 1397, in _do_call
    raise type(e)(node_def, op, message)  # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors_impl.CancelledError: Graph execution error:

Detected at node 'fifo_queue_enqueue' defined at (most recent call last):
    File "/Users/mwmathis/Documents/DeepLabCut/examples/testscript_multianimal.py", line 139, in <module>
      deeplabcut.train_network(config_path, maxiters=N_ITER)
    File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/training.py", line 197, in train_network
      train(
    File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/core/train_multianimal.py", line 66, in train
      batch, enqueue_op, placeholders = setup_preloading(batch_spec)
    File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/core/train.py", line 69, in setup_preloading
      enqueue_op = q.enqueue(placeholders_list)
Node: 'fifo_queue_enqueue'
Enqueue operation was cancelled
	 [[{{node fifo_queue_enqueue}}]]

Original stack trace for 'fifo_queue_enqueue':
  File "/Users/mwmathis/Documents/DeepLabCut/examples/testscript_multianimal.py", line 139, in <module>
    deeplabcut.train_network(config_path, maxiters=N_ITER)
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/training.py", line 197, in train_network
    train(
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/core/train_multianimal.py", line 66, in train
    batch, enqueue_op, placeholders = setup_preloading(batch_spec)
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/core/train.py", line 69, in setup_preloading
    enqueue_op = q.enqueue(placeholders_list)
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/ops/data_flow_ops.py", line 346, in enqueue
    return gen_data_flow_ops.queue_enqueue_v2(
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 4063, in queue_enqueue_v2
    _, _, _op, _outputs = _op_def_library._apply_op_helper(
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/framework/op_def_library.py", line 797, in _apply_op_helper
    op = g._create_op_internal(op_type_name, inputs, dtypes=None,
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/framework/ops.py", line 3800, in _create_op_internal
    ret = Operation(

The network is now trained and ready to evaluate. Use the function 'evaluate_network' to evaluate the network.
Network trained.
Evaluating network...
Config:
{'all_joints': [[0], [1], [2], [3], [4]],
 'all_joints_names': ['bodypart1',
                      'bodypart2',
                      'bodypart3',
                      'corner1',
                      'corner2'],
 'batch_size': 1,
 'crop_pad': 0,
 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_multi_mouseOct9/multi_mouse_dlc_team80shuffle1.pickle',
 'dataset_type': 'multi-animal-imgaug',
 'deterministic': False,
 'fg_fraction': 0.25,
 'global_scale': 0.8,
 'init_weights': '/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/efficientnet-b0/model.ckpt',
 'intermediate_supervision': False,
 'intermediate_supervision_layer': 12,
 'location_refinement': True,
 'locref_huber_loss': True,
 'locref_loss_weight': 1.0,
 'locref_smooth': False,
 'locref_stdev': 7.2801,
 'log_dir': 'log',
 'mean_pixel': [123.68, 116.779, 103.939],
 'minconfidence': 0.01,
 'mirror': False,
 'multi_stage': False,
 'net_type': 'efficientnet-b0',
 'nmsradius': 5.0,
 'num_idchannel': 3,
 'num_joints': 5,
 'num_limbs': 3,
 'optimizer': 'sgd',
 'pairwise_huber_loss': True,
 'pairwise_predict': False,
 'partaffinityfield_graph': [[0, 1], [0, 2], [1, 2]],
 'partaffinityfield_predict': True,
 'regularize': False,
 'scoremap_dir': 'test',
 'shuffle': True,
 'sigma': 1,
 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/dlc-models/iteration-0/multi_mouseOct9-trainset80shuffle1/test/snapshot',
 'stride': 8.0,
 'weigh_negatives': False,
 'weigh_only_present_joints': False,
 'weigh_part_predictions': False,
 'weight_decay': 0.0001}
Running  DLC_effnet_b0_multi_mouseOct9shuffle1_5  with # of trainingiterations: 5
/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer_v1.py:1694: UserWarning: `layer.apply` is deprecated and will be removed in a future version. Please use `layer.__call__` method instead.
  warnings.warn('`layer.apply` is deprecated and '
Activating extracting of PAFs
Network Evaluation underway...
5it [00:02,  2.23it/s]
Results for 5 training iterations, training fraction of 80, and shuffle 1:
Train error: 60.07 pixels. Test error: 51.39 pixels.
With pcutoff of 0.6:
Train error: 57.65 pixels. Test error: 60.39 pixels.
##########################################
Average Euclidean distance to GT per individual (in pixels; test-only)
individuals
individual1    60.393749
individual2          NaN
individual3          NaN
single               NaN
Average Euclidean distance to GT per bodypart (in pixels; test-only)
bodyparts
bodypart1          NaN
bodypart2          NaN
bodypart3    60.393749
corner1            NaN
corner2            NaN
Done and results stored for snapshot:  snapshot-5
Selecting best skeleton...
Graph 1|2
5it [00:00, 676.13it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 1013.73it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1546.00it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 1228.78it/s]
Graph 2|2
5it [00:00, 823.83it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 1293.24it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1053.85it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 2807.43it/s]
Network evaluated....
Extracting maps...
Config:
{'all_joints': [[0], [1], [2], [3], [4]],
 'all_joints_names': ['bodypart1',
                      'bodypart2',
                      'bodypart3',
                      'corner1',
                      'corner2'],
 'batch_size': 1,
 'crop_pad': 0,
 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_multi_mouseOct9/multi_mouse_dlc_team80shuffle1.pickle',
 'dataset_type': 'multi-animal-imgaug',
 'deterministic': False,
 'fg_fraction': 0.25,
 'global_scale': 0.8,
 'init_weights': '/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/efficientnet-b0/model.ckpt',
 'intermediate_supervision': False,
 'intermediate_supervision_layer': 12,
 'location_refinement': True,
 'locref_huber_loss': True,
 'locref_loss_weight': 1.0,
 'locref_smooth': False,
 'locref_stdev': 7.2801,
 'log_dir': 'log',
 'mean_pixel': [123.68, 116.779, 103.939],
 'minconfidence': 0.01,
 'mirror': False,
 'multi_stage': False,
 'net_type': 'efficientnet-b0',
 'nmsradius': 5.0,
 'num_idchannel': 3,
 'num_joints': 5,
 'num_limbs': 3,
 'optimizer': 'sgd',
 'paf_best': [1, 0],
 'pairwise_huber_loss': True,
 'pairwise_predict': False,
 'partaffinityfield_graph': [[0, 1], [0, 2], [1, 2]],
 'partaffinityfield_predict': True,
 'regularize': False,
 'scoremap_dir': 'test',
 'shuffle': True,
 'sigma': 1,
 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/dlc-models/iteration-0/multi_mouseOct9-trainset80shuffle1/test/snapshot',
 'stride': 8.0,
 'weigh_negatives': False,
 'weigh_only_present_joints': False,
 'weigh_part_predictions': False,
 'weight_decay': 0.0001}
/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer_v1.py:1694: UserWarning: `layer.apply` is deprecated and will be removed in a future version. Please use `layer.__call__` method instead.
  warnings.warn('`layer.apply` is deprecated and '
Activating extracting of PAFs
Analyzing data...
3it [00:00,  3.07it/s]
Saving plots...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:05<00:00,  1.69s/it]
/bin/sh: ffmpeg: command not found
Analyzing video...
Config:
{'all_joints': [[0], [1], [2], [3], [4]],
 'all_joints_names': ['bodypart1',
                      'bodypart2',
                      'bodypart3',
                      'corner1',
                      'corner2'],
 'batch_size': 1,
 'crop_pad': 0,
 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_multi_mouseOct9/multi_mouse_dlc_team80shuffle1.pickle',
 'dataset_type': 'multi-animal-imgaug',
 'deterministic': False,
 'fg_fraction': 0.25,
 'global_scale': 0.8,
 'init_weights': '/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/efficientnet-b0/model.ckpt',
 'intermediate_supervision': False,
 'intermediate_supervision_layer': 12,
 'location_refinement': True,
 'locref_huber_loss': True,
 'locref_loss_weight': 1.0,
 'locref_smooth': False,
 'locref_stdev': 7.2801,
 'log_dir': 'log',
 'mean_pixel': [123.68, 116.779, 103.939],
 'minconfidence': 0.01,
 'mirror': False,
 'multi_stage': False,
 'net_type': 'efficientnet-b0',
 'nmsradius': 5.0,
 'num_idchannel': 3,
 'num_joints': 5,
 'num_limbs': 3,
 'optimizer': 'sgd',
 'paf_best': [1, 0],
 'pairwise_huber_loss': True,
 'pairwise_predict': False,
 'partaffinityfield_graph': [[0, 1], [0, 2], [1, 2]],
 'partaffinityfield_predict': True,
 'regularize': False,
 'scoremap_dir': 'test',
 'shuffle': True,
 'sigma': 1,
 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/dlc-models/iteration-0/multi_mouseOct9-trainset80shuffle1/test/snapshot',
 'stride': 8.0,
 'weigh_negatives': False,
 'weigh_only_present_joints': False,
 'weigh_part_predictions': False,
 'weight_decay': 0.0001}
Using snapshot-5 for model /Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/dlc-models/iteration-0/multi_mouseOct9-trainset80shuffle1
Activating extracting of PAFs
No video(s) were found. Please check your paths and/or 'video_type'.
Video analyzed.
Create video with all detections...
Video created.
Convert detections to tracklets...
Config:
{'all_joints': [[0], [1], [2], [3], [4]],
 'all_joints_names': ['bodypart1',
                      'bodypart2',
                      'bodypart3',
                      'corner1',
                      'corner2'],
 'batch_size': 1,
 'crop_pad': 0,
 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_multi_mouseOct9/multi_mouse_dlc_team80shuffle1.pickle',
 'dataset_type': 'multi-animal-imgaug',
 'deterministic': False,
 'fg_fraction': 0.25,
 'global_scale': 0.8,
 'init_weights': '/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/efficientnet-b0/model.ckpt',
 'intermediate_supervision': False,
 'intermediate_supervision_layer': 12,
 'location_refinement': True,
 'locref_huber_loss': True,
 'locref_loss_weight': 1.0,
 'locref_smooth': False,
 'locref_stdev': 7.2801,
 'log_dir': 'log',
 'mean_pixel': [123.68, 116.779, 103.939],
 'minconfidence': 0.01,
 'mirror': False,
 'multi_stage': False,
 'net_type': 'efficientnet-b0',
 'nmsradius': 5.0,
 'num_idchannel': 3,
 'num_joints': 5,
 'num_limbs': 3,
 'optimizer': 'sgd',
 'paf_best': [1, 0],
 'pairwise_huber_loss': True,
 'pairwise_predict': False,
 'partaffinityfield_graph': [[0, 1], [0, 2], [1, 2]],
 'partaffinityfield_predict': True,
 'regularize': False,
 'scoremap_dir': 'test',
 'shuffle': True,
 'sigma': 1,
 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/dlc-models/iteration-0/multi_mouseOct9-trainset80shuffle1/test/snapshot',
 'stride': 8.0,
 'weigh_negatives': False,
 'weigh_only_present_joints': False,
 'weigh_part_predictions': False,
 'weight_decay': 0.0001}
Using snapshot-5 for model /Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/dlc-models/iteration-0/multi_mouseOct9-trainset80shuffle1
No video(s) found. Please check your path!
Tracklets created...
No video(s) found. Please check your path!
Plotting trajectories...
No videos found. Make sure you passed a list of videos and that *videotype* is right.
Trajectory plotted.
Creating labeled video...
Labeled video created.
Filtering predictions...
No video(s) were found. Please check your paths and/or 'videotype'.
Predictions filtered.
Extracting outlier frames...
No suitable videos found in ['/Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/videos/m3v1mp4short.mp4']
Outlier frames extracted.
RELABELING
Traceback (most recent call last):
  File "/Users/mwmathis/Documents/DeepLabCut/examples/testscript_multianimal.py", line 232, in <module>
    DF = pd.read_hdf(file, "df_with_missing")
  File "/Users/mwmathis/opt/anaconda3/envs/DEEPLABCUT_newGUI/lib/python3.9/site-packages/pandas/io/pytables.py", line 414, in read_hdf
    raise FileNotFoundError(f"File {path_or_buf} does not exist")
FileNotFoundError: File /Users/mwmathis/Documents/DeepLabCut/examples/multi_mouse-dlc_team-2022-10-09/labeled-data/m3v1mp4short/machinelabels-iter0.h5 does not exist
```
</details>

### Anything else?

_No response_

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/DeepLabCut/DeepLabCut/blob/master/CODE_OF_CONDUCT.md)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Failing tests on github and local run `./test.sh` #2006

Is there an existing issue for this?

Bug description

Operating System

DeepLabCut version

DeepLabCut mode

Device type

Steps To Reproduce

Relevant log output

Anything else?

Code of Conduct

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Uh oh!

Failing tests on github and local run ./test.sh #2006

Description

Is there an existing issue for this?

Bug description

Operating System

DeepLabCut version

DeepLabCut mode

Device type

Steps To Reproduce

Relevant log output

Anything else?

Code of Conduct

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Failing tests on github and local run `./test.sh` #2006