[ICLR 2025] IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminations

     

Zhibing Li¹, Tong Wu^{1 †}, Jing Tan¹, Mengchen Zhang^2,3, Jiaqi Wang³, Dahua Lin^{1,3 †
1}The Chinese University of Hong Kong ²Zhejiang University ³Shanghai AI Laboratory
†: Corresponding Authors

v2_1.mp4

• Release inference code and pretrained checkpoints.
• Release training dataset.
• Release training code.

News

• [04.25] See you in Singapore!
• [01.25] We have released the training code!
• [12.24] We have released the dataset and rendering script.

Install

Our environment has been tested on CUDA 11.8 with A100.

git clone [email protected]:Lizb6626/IDArb.git && cd IDArb
conda create -n idarb python==3.8 -y
conda activate idarb
conda install pytorch==2.2.1 torchvision==0.17.1 torchaudio==2.2.1 pytorch-cuda=11.8 -c pytorch -c nvidia
pip install -r requirements.txt

Inference

Single Image Intrinsic Decomposition

python main.py --data_dir example/single --output_dir output/single --input_type single

Multi-view Intrinsic Decomposition

For multi-view intrinsic decomposition, camera pose can be incorporated by enabling the --cam option.

## --num_views: number of input views

# Without camera pose information
python main.py --data_dir example/multi --output_dir output/multi --input_type multi --num_views 4

# With camera pose information
python main.py --data_dir example/multi --output_dir output/multi --input_type multi --num_views 4 --cam

Training

Dataset

The training data consists of a combination of our Arb-Objaverse, ABO, and G-Objaverse datasets. The dataset list is available in datalist/train.json.gz.

For the Arb-Objaverse dataset, we first rendered all 347K 3D models from Objaverse that use BSDF shaders. From this, we curated a high-quality subset of 68K models for training. You can access uncurated dataset and curated dataset.

Training Script

To train the model, update the dataset_root in the configuration file configs/train.yaml. Then, run the following command:

accelerate launch --config_file configs/acc/8gpu.yaml train.py --config configs/train.yaml

Acknowledgement

This project relies on many amazing repositories. Thanks to the authors for sharing their code and data.

• Wonder3D
• GeoWizard
• EscherNet
• Stanford-ORB
• NeRD

Citation

@inproceedings{
li2025idarb,
title={{IDA}rb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminations},
author={Zhibing Li and Tong Wu and Jing Tan and Mengchen Zhang and Jiaqi Wang and Dahua Lin},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025},
url={https://openreview.net/forum?id=uuef1HP6X7}
}