[SIGGRAPH ASIA 2025] Hierarchical Neural Semantic Representation for 3D Semantic Correspondence

Keyu Du¹, Jingyu Hu^2†, Haipeng Li¹, Hao Xu², Haibing Huang³, Chi-Wing Fu², Shuaicheng Liu¹

¹University of Electronic Science and Technology of China (UESTC) ²The Chinese University of Hong Kong (CUHK) ³TeleAI

† Corresponding author

This is the official LAS-Backbone implementation of our SIGGRAPH ASIA 2025 paper, Hierarchical Neural Semantic Representation for 3D Semantic Correspondence. Trellis-Backbone implementation can be found mapledky/NSR_PyTorch.

Installation

Following is the suggested way to install the dependencies of our code:

conda create -n nsr_las
conda activate nsr_las

conda install pytorch=1.9.0 torchvision=0.10.0 cudatoolkit=10.2 -c pytorch -c nvidia

pip install tqdm fire einops pyrender pyrr trimesh ocnn timm scikit-image==0.18.2 scikit-learn==0.24.2 pytorch-lightning==1.6.1

Data Preparation

SDF data creation

Please ref to SDF-StyleGAN for generating the SDF field from ShapeNet data or your customized data.

Pre-trained Models

We provide the pretrained models for the category-conditioned generation. Please download the pretrained models from Google Drive and put them in checkpoints/.

Usage

For several main categories in ShapeNet (such as airplane, chair, car, etc.), we provide pretrained models available on Google Drive. Download the desired models and place them in the checkpoints/ directory.

For unseen shapes or your own custom datasets (e.g., human or animal datasets), it is recommended to train the model from scratch using the scripts provided below. Make sure to follow the SDF data creation process mentioned earlier to prepare your data properly.

The training and generation-testing scripts are located in the scripts/ directory.

Train from Scratch

bash scripts/train_category.sh

Category-conditioned generation

bash scripts/generate_category.sh

We provide the following demo script for generating dense matching between source and target shapes in the ShapeNet dataset. Make sure to adjust the hyperparameters in test_file.json, as well as update the dataset path and checkpoint path in the scripts.

Generating Dense Matching

python dense_matching/colormap.py

Citation

If you find our work useful in your research, please consider citing:

@article{du2025hnsr,
    title   = {Hierarchical Neural Semantic Representation for 3D Semantic Correspondence},
    author  = {Keyu Du and Jingyu Hu and Haipeng Li and Hao Xu and Haibing Huang and Chi-Wing Fu and Shuaicheng Liu},
    journal = {SIGGRAPH ASIA},
    year    = {2025}
}

Acknowledgements

• LAS-Diffusion