Learning Global Motion with Compact Gaussians
for Feed-Forward 4D Reconstruction

Mungyeom Kim^1* · Minkyeong Jeon^1* · Honggyu An^1* · Jaewoo Jung¹ · Hyunah Ko¹ · Jisang Han¹ · Hyeonseo Yu¹ · Donghwan Shin¹ · Sunghwan Hong² · Takuya Narihira³ · Kazumi Fukuda³ · Yuki Mitsufuji^3,4† · Seungryong Kim^1†

¹KAIST AI, ²ETH Zurich, ETH AI Center, ³SONY AI, ⁴Sony Group Corporation

^*Co-first author, †Co-corresponding author

Project Page

We propose a feed-forward dynamic reconstruction network that effectively captures the global motion of dynamic scenes. Our method employs a timestamp-conditioned, query-based transformer Gaussian decoder that aggregates geometrically consistent features from multi-frame videos, enabling each Gaussian to model globally coherent motion.

What to Expect

• Training code for C4G Gaussian reconstruction.
  
• VDM-based rendering enhancement module code.
  
• Pretrained C4G weights.
  

Installation

Our code is developed based on PyTorch 2.2.0, CUDA 12.1, and Python 3.10.

We recommend using conda for installation:

conda create -n c4g python=3.10
conda activate c4g
bash scripts/install.sh

Data Preparation

For training, we use the preprocessed RealEstate10K dataset following pixelSplat and MVSplat. Set the dataset path in config/dataset/re10k.yaml before training.

For Spring, please refer to the official website.

The default paths are placeholders written as /path/to/....

Pretrained Weights

C4G initializes from the C3G Gaussian decoder checkpoint. Download gaussian_decoder.ckpt from the C3G Hugging Face repository and place it under pretrained_weights/gaussian_decoder.ckpt.

C4G checkpoints are available in the C4G Hugging Face repository.

Training

To train C4G, you can run the following commands:

bash scripts/train.sh

If you want to change configs of our training code, you can just modify the main training config in config/training/c4g.yaml.

If you do not want to log to wandb, keep wandb.mode=disabled.

VDM-based Rendering Enhancement Module

The optional VDM-based rendering enhancement module code is included under submodules/DiffSynth-Studio.

To train and inference the VDM-based rendering enhancement module, please follow submodules/DiffSynth-Studio/README.md.

Citation

@article{kim2026learning,
  title={Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction},
  author={Kim, Mungyeom and Jeon, Minkyeong and An, Honggyu and Jung, Jaewoo and Ko, Hyuna and Han, Jisang and Yu, Hyeonseo and Shin, Donghwan and Hong, Sunghwan and Narihira, Takuya and others},
  journal={arXiv preprint arXiv:2605.31595},
  year={2026}
}

Acknowledgement

We thank the authors of VGGT, MoGe, and CoWTracker for their excellent work and code.