METASCENES: Towards Automated Replica Creation for Real-world 3D Scans

CVPR 2025

Huangyue Yu, Baoxiong Jia, Yixin Chen, Yandan Yang, Puhao Li, Rongpeng Su,
Jiaxin Li, Qing Li, Wei Liang, Song-Chun Zhu, Tengyu Liu, Siyuan Huang

We propose METASCENES, a large-scale simulatable 3D scene dataset constructed by replacing objects in real-world 3D scans with realistic and high-quality object assets retrieved or reconstructed from diverse sources.

News

• [2025-03] Training & Inference code as well as preprocessing code is released!
• [2025-03] We release the MetaScenes dataset. Fill out the form for the download link!
• [2025-02] 🎉MetaScenes is accepted by CVPR 2025! Code and dataset will come shortly, stay tuned!

⚙️Installation

conda create -n metascenes python=3.9
conda activate metascenes
pip install torch==2.0.1 torchvision==0.15.2 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt

🏘️Data

MetaScenes Dataset

See DATA.md for detailed instructions on data download, processing, visualization.

Fill out the form for the download!

MetaScenes Preprocessing

MetaScenes provides a comprehensive pipeline to construct replica scenes of real-world environments. We provide the code of the following three key preprocessing components:

1. Heuristic-based Room Layout Estimation
2. Object Pose Alignment
3. Physics-based Scene Optimization

See PREPROCESS.md for detailed instructions for setting up the environment, running each component, and understanding the output generated by each process.

🔥Scan2Sim

Scan2Sim is a multi-modal alignment model designed to retrieve the most optimal asset candidate from a set of candidates, leveraging ground truth optimal asset selection annotations from METASCENES.

See MODEL.md for the inventory of available checkpoints and detailed instructions on training and testing

😃Acknowledgements

Some codes are borrowed from ULIP2. We thank all the authors for their great work.

Citation

@inproceedings{yu2025metascenes,
  title={METASCENES: Towards Automated Replica Creation for Real-world 3D Scans},
  author={Yu, Huangyue and Jia, Baoxiong and Chen, Yixin and Yang, yandan and Li, Puhao and Su, Rongpeng and Li, Jiaxin and Li, Qing and Liang, Wei and Zhu, Song-Chun and Liu, Tengyu and Huang, Siyuan},
  booktitle=Conference on Computer Vision and Pattern Recognition(CVPR),
  year={2025}
}