Skip to content

Conversation

@maximpavliv
Copy link
Contributor

@maximpavliv maximpavliv commented Feb 21, 2025

This Pull Request adds Pytorch support for the create_pretrained_project() method (and closes #2732)

Specifically:

  • create_pretrained_project() becomes a compat.py-like method, and calls the specific method implementation based on the specified engine parameter.
    The old create_pretrained_project() is renamed to create_pretrained_project_tensorflow() and a new create_pretrained_project_pytorch() method with the Pytorch-shuffle logic is added.
    The existing usages of create_pretrained_project() are also adapted.
  • Both create_pretrained_project_* implementations call _create_training_datasets_metadata().
    This method has been cherry-picked from [WIP] DLC 3.0 - Review documentation #2881(where a first version, _create_and_save_fake_metadata(), has been implemented as a temporary fix), and adapted to work with both Tensorflow and Pytorch. It creates the metadata.yaml file in the training-datasets directory, and fills it with a first shuffle corresponding to the downloaded modelzoo model.
  • The video processing logic (that is optionally done after the pretrained project has been created) is extracted to
    _process_videos(), and called by both create_pretrained_project_* implementations.
  • A new individuals parameter is added to create_new_project() and to create_pretrained_project(), so that the list of individuals can be automatically filled in the different configuration files of the new project.
  • An integration test script for create_pretrained_project(engine=Engine.PYTORCH) is added.

This Pull Request also does some fixes to issues that have been spotted during its development:

  • Fixes list_videos_in_folder() : in original (Tensorflow-implementation) methods (f.ex. analyze_videos()), videotype argument's value by default was "" (instead of None), and this default argument value was not treated properly.
  • get_super_animal_project_cfg() was using read_config() to read the superanimal project configuration template embedded into the DeepLabCut repository, which then modified the file in the DeepLabCut codebase. Now, the method reads the template with read_config_as_dict(), which doesn't modify the file.
  • The previous fix created a circular-import issue, that has been resolved by moving the read_config_as_dict(), write_config() and pretty_print() methods to deeplabcut.core.config_utils.py module.
  • The other superanimal integration test scripts (testscript_superanimal_adaptation.py, testscript_superanimal_inference.py and testscript_superanimal_transfer_learning.py) have been fixed.

@maximpavliv maximpavliv changed the title Implement create_pretrained_project for Pytorch engine Implement create_pretrained_project() for Pytorch engine Feb 21, 2025
@maximpavliv maximpavliv changed the title Implement create_pretrained_project() for Pytorch engine Implement create_pretrained_project() for Pytorch engine Feb 21, 2025
@maximpavliv
Copy link
Contributor Author

The Tensorflow implementation of create_pretrained_project() fills the "default_net_type" field in the project configuration file with the chosen modelzoo's model architecture.
In the Pytorch implementation of create_pretrained_project() I wasn't sure how to do this, because we would also then need a "default detector type" field (particularly since Superanimal models are all Top-Down currently), but such a field doesn't exist. Therefore I left the "default_net_type" field emtpy for now.
Any suggestions welcome on this point.

@maximpavliv maximpavliv force-pushed the maxim/create_pretrained_project_torch branch from ae721ac to ea78c81 Compare February 21, 2025 13:09
@maximpavliv maximpavliv added enhancement New feature or request ModelZoo related to models/code for modezoo.deeplabcut.org DLC3.0🔥 labels Feb 21, 2025
@maximpavliv maximpavliv force-pushed the maxim/create_pretrained_project_torch branch 2 times, most recently from 9324d57 to 48f0fd4 Compare February 25, 2025 14:05
Copy link
Contributor

@n-poulsen n-poulsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM @maximpavliv, thanks! just one minor comment

@maximpavliv maximpavliv force-pushed the maxim/create_pretrained_project_torch branch from 48f0fd4 to 0c4db27 Compare February 25, 2025 14:57
@maximpavliv maximpavliv force-pushed the maxim/create_pretrained_project_torch branch from 0c4db27 to a67974b Compare February 25, 2025 15:11
@maximpavliv maximpavliv merged commit 70fd919 into pytorch_dlc Feb 25, 2025
1 check passed
@n-poulsen n-poulsen deleted the maxim/create_pretrained_project_torch branch February 26, 2025 14:05
@maximpavliv maximpavliv mentioned this pull request Mar 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DLC3.0🔥 enhancement New feature or request ModelZoo related to models/code for modezoo.deeplabcut.org

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants