MetaGS: A Meta-Learned Gaussian-Phong Model for Out-of-Distribution 3D Scene Relighting

Yumeng He, Yunbo Wang^†, Xiaokang Yang

arXiv | PDF | Dataset

This repository contains the official code for our paper: MetaGS: A Meta-Learned Gaussian-Phong Model for Out-of-Distribution 3D Scene Relighting.

Installation

1. Create an environment

   conda create -n metags python=3.10
   conda activate metags

2. Install dependencies

   git clone https://github.com/raynehe/MetaGS.git
   cd MetaGS
   git submodule update --init --recursive
   pip install -r requirements.txt
   pip install -e submodules/depth-diff-gaussian-rasterization
   pip install -e submodules/simple-knn
   pip install ./bvh

Experiments

Training

1. Stage 1&2: Gaussian initialization & Normal finetuning

   python train.py -s <path_to_your_dataset> -m <path_to_ouput_folder> --eval

2. Stage 3: Meta-learning

   python train_meta.py -s <path_to_your_dataset> -m <path_to_ouput_folder> --eval

Evaluation

1. Generate NVS renderings

   python render.py -m <path_to_ouput_folder>

2. Calculate error metrics

    python metrics.py -m <path_to_ouput_folder>

Full script

You can run the full experiment using: (remember ro edit the $DATADIR and $OUTPUT location)

sh run.sh

Acknowledgements

We appreciate the following github repos where we borrow code from:

• gaussian splatting
• Relightable3DGaussian

Thanks for their amazing works!

Citation

If you find our work helps, please cite our paper:

@article{he2024metags,
  title={MetaGS: A Meta-Learned Gaussian-Phong Model for Out-of-Distribution 3D Scene Relighting},
  author={He, Yumeng and Wang, Yunbo and Yang, Xiaokang},
  journal={arXiv preprint arXiv:2405.20791},
  year={2024}
}