AuraFusion360: Augmented Unseen Region Alignment for Reference-based 360° Unbounded Scene Inpainting

Chung-Ho Wu* · Yang-Jung Chen* · Ying-Huan Chen · Jie-Ying Lee · Bo-Hsu Ke · Chun-Wei Tuan Mu · Yi-Chuan Huang · Chin-Yang Lin · Min-Hung Chen · Yen-Yu Lin
· Yu-Lun Liu


NYCU | NVIDIA

News

• [2025.02.10] Release project page, arXiv paper, dataset, and evaluation results!
• [2025.02.27] Accepted by CVPR 2025!
• [2025.06.29] Release Full Code.
• [2025.06.30] All AuraFusion360 results are now available on HuggingFace. Note that these results may differ slightly from the paper. For the complete evaluation results and other baseline methods presented in the paper, please refer to here.

Get Started

Environment Setup

git clone https://github.com/kkennethwu/AuraFusion360_official.git --recursive
export HF_TOKEN=<your hf token>
export HF_HOME=<your hf home>
source install.sh

Download Dataset

In addition to Google Drive, the 360-USID (our dataset) and Other-360 (collected dataset) are now available for download via HuggingFace.

huggingface-cli login
huggingface-cli download kkennethwu/360-USID --repo-type dataset --local-dir ./data --resume-download --quiet --max-workers 32

Running

1. Training Object-Masked Gaussians

python train.py --config configs/{dataset_name}/{scene_name}/train.config
python render.py -s data/{dataset_name}/{scene_name} -m output/{dataset_name}/{scene_name} --skip_mesh --render_path --iteration 30000

2. Removing Objects & Generating Unseen Masks

python remove.py --config configs/{dataset_name}/{scene_name}/remove.config
python utils/sam2_utils.py --dataset {dataset_name} --scene {scene_name}
# python scripts/visualize_mask.py --dataset {dataset_name} --scene {scene_name} --type mask # (optional) 
# python scripts/visualize_mask.py --dataset {dataset_name} --scene {scene_name} --type contour # (optional)

3. Unproject & Inpaint

python inpaint.py --config configs/$dataset_name/$scene_name/inpaint.config
python utils/LeftRefill/sdedit_utils.py --config configs/$dataset_name/$scene_name/sdedit.config 
python inpaint.py --config configs/$dataset_name/$scene_name/inpaint.config --images inpaint --finetune_iteration 10000

Citation

If you find our dataset, evaluation results, or code useful, please cite this paper and give us a ⭐️.

@InProceedings{wu2025aurafusion,
    author    = {Wu, Chung-Ho and Chen, Yang-Jung and Chen, Ying-Huan and Lee, Jie-Ying and Ke, Bo-Hsu and Mu, Chun-Wei Tuan and Huang, Yi-Chuan and Lin, Chin-Yang and Chen, Min-Hung and Lin, Yen-Yu and Liu, Yu-Lun},
    title     = {AuraFusion360: Augmented Unseen Region Alignment for Reference-based 360deg Unbounded Scene Inpainting},
    booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
    month     = {June},
    year      = {2025},
    pages     = {16366-16376}
}

Acknowledgements

This work was supported by NVIDIA Taiwan AI Research & Development Center (TRDC). This research was funded by the National Science and Technology Council, Taiwan, under Grants NSTC 112-2222-E-A49-004-MY2 and 113-2628-E-A49-023-. Yu-Lun Liu acknowledges the Yushan Young Fellow Program by the MOE in Taiwan.