Swift4D: Adaptive divide-and-conquer Gaussian Splatting for compact and efficient reconstruction of dynamic scene

Jiahao Wu , Rui Peng , Zhiyan Wang, Lu Xiao,
Luyang Tang, Kaiqiang Xiong, Ronggang Wang ^✉

ICLR 2025 Paper | VRU Dataset

Environmental Setups

Please follow the 3D-GS to install the relative packages.

git clone --recursive https://github.com/WuJH2001/Swift4d.git
cd Swift4d
conda create -n swift4d python=3.8
conda activate swift4d

pip install -r requirements.txt
pip install  submodules/diff-gaussian-rasterization
pip install  submodules/simple-knn

After that, you need to install tiny-cuda-nn.

Data Preparation

1. For N3DV dynamic scenes:

Plenoptic Dataset could be downloaded from their official websites. To save the memory, you should extract the frames of each video and then organize your dataset as follows.

├── data
│   | dynerf
│     ├── cook_spinach
│       ├── cam00
│           ├── images
│               ├── 0000.png
│               ├── 0001.png
│               ├── 0002.png
│               ├── ...
│       ├── cam01
│           ├── images
│               ├── 0000.png
│               ├── 0001.png
│               ├── ...
│     ├── cut_roasted_beef
|     ├── ...

2. For other multi-view dynamic scenes (e.g., MeetRoom, VRU dataset):

Please follow the step-by-step instructions in this script to process your data. The processed camera poses for MeetRoom are available here. If you use the camera parameters we provided, you can skip steps 1 and 2 and go directly to step 3.

If you want to train the model on multi-view dataset, you may need to modify this code. For the small-motion dataset MeetRoom, you can train 300 frames at once, so simply set the first frame and final frame to 0 and 300, respectively. In contrast, for the large-motion dataset VRU, you can only train 20 frames at a time, so you need to set them as 0–20, 20–40, and so on.

Training

For training dynerf scenes such as cut_roasted_beef, run

# First, extract the frames of each video.
python scripts/preprocess_dynerf.py --datadir data/dynerf/cut_roasted_beef
# Second, generate point clouds from input data.
bash colmap.sh data/dynerf/cut_roasted_beef llff
# Third, downsample the point clouds generated in the second step.
python scripts/downsample_point.py data/dynerf/cut_roasted_beef/colmap/dense/workspace/fused.ply data/dynerf/cut_roasted_beef/points3D_downsample2.ply
# Finally, train.
python train.py -s data/dynerf/cut_roasted_beef --port 6017 --expname "dynerf/cut_roasted_beef" --configs arguments/dynerf/cut_roasted_beef.py 

Rendering

Run the following script to render the images.

python render.py --model_path output/dynerf/cut_roasted_beef --skip_train --skip_video --iteration 13000 --configs  arguments/dynerf/cut_roasted_beef.py

Evaluation

You can just run the following script to evaluate the model.

python metrics.py --model_path output/dynerf/coffee_martini/

Trained Models

You can find our dynerf models here. The Virtual Reality Unit (VRU) Basketball dataset, which we used in the paper, is available here. It was created by the AVS-VRU work unit. You can also download our VRU Basketball dataset from 🤗 Hugging Face. Feel free to use it for training your model or validating your method!

If you find our VRU Basketball dataset or code helpful, we’d greatly appreciate it if you could give us a star and consider citing our work.

Contributions

This project is still under development. Please feel free to raise issues or submit pull requests to contribute to our codebase. Thanks to 4DGS

---

Citation

If you find our code useful, we would greatly appreciate it if you could give us a star and cite our work.

@article{wu2025swift4d,
  title={Swift4D: Adaptive divide-and-conquer Gaussian Splatting for compact and efficient reconstruction of dynamic scene},
  author={Wu, Jiahao and Peng, Rui and Wang, Zhiyan and Xiao, Lu and Tang, Luyang and Yan, Jinbo and Xiong, Kaiqiang and Wang, Ronggang},
  journal={arXiv preprint arXiv:2503.12307},
  year={2025}
}