DAViD: Modeling Dynamic Affordance of 3D Objects Using Pre-trained Video Diffusion Models (ICCV 2025)

Project Page  |  Paper

This is the official code for the paper "DAViD: Modeling Dynamic Affordance of 3D Objects Using Pre-trained Video Diffusion Models".

Installation

To setup the environment for running DAViD, please refer to the instructions provided here.

Quick Start

2D HOI Image Generation

To generate 2D HOI Images of given 3D object (in this case, barbell), use following command.

bash scripts/generate_2d_hoi_images.sh --dataset "ComAsset" --category "barbell" --device 0 --skip_done

Rendered Image

Canny Edges

2D HOI Image

Image-to-Video

We leverage commercial image-to-video diffusion model Kling AI to make 2D HOI videos from 2D HOI images. Specifically, we use imgur and PiAPI for uploading image and calling API for Kling AI. Check out scripts/videos/get.sh, scripts/videos/post_i2v.sh and setup your X-API-key of your PiAPI account. Also check out constants/videos.py and setup your client id of your imgur account. Note that you need paid version of Kling AI for directly follow our setting.

CUDA_VISIBLE_DEVICES=0 python src/generation/generate_videos.py --dataset "ComAsset" --category "barbell" --skip_done

Otherwise, you can use open-source image-to-video models such as Wan2.1, but we haven't tested it yet.

4D HOI Sample Generation

To generate 4D HOI Samples from the generated 2D HOI Images (of the given 3D object, barbell), use following command.

bash scripts/generate_4d_hoi_samples.sh --dataset "ComAsset" --category "barbell" --device 0 --skip_done

2D HOI Image

2D HOI Video

4D HOI Sample (Camera View)

Visualization

To visualize generated 4D HOI Samples, use following command.

blenderproc debug src/visualization/visualize_4d_hoi_sample.py --dataset "ComAsset" --category "barbell" --idx 0

Train LoRA for MDM

To train LoRA for MDM (of the given 3D object, barbell), use following command.

bash scripts/train_lora.sh --dataset "ComAsset" --category "barbell" --device 0

Train Object Motion Diffusion Model

To train Object Motion Diffusion Model (of the given 3D object, barbell), use following command.

bash scripts/train_omdm.sh --dataset "ComAsset" --category "barbell" --device 0

Sample Human Motion (Inference)

bash scripts/generate_human_motion.sh --max_seed 5 --dataset "ComAsset" --category "barbell" --device 0

Sample Object Motion (Inference)

bash scripts/generate_object_motion.sh --dataset "ComAsset" --category "barbell" --device 0

Regarding Code Release

• We are keep updating the code (including dataset and environment setup)!
• [2025/07/10] Initial skeleton code release!

Citation

@misc{david,
      title={DAViD: Modeling Dynamic Affordance of 3D Objects Using Pre-trained Video Diffusion Models}, 
      author={Hyeonwoo Kim and Sangwon Baik and Hanbyul Joo},
      year={2025},
      eprint={2501.08333},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2501.08333}, 
}