Sketchy-3DIS

This is the official PyTorch implementation of Sketchy-3DIS.

Sketchy Bounding-box Supervision for 3D Instance Segmentation (CVPR2025) [Paper]

Qian Deng, Le Hui, Jin Xie, Jian Yang

Get Started

Environment

Requirements

• Python 3.x
• Pytorch 1.10
• CUDA 10.x or higher

The following installation suppose python=3.8 pytorch=1.10 and cuda=11.3.

• Create a conda virtual environment

  conda create -n Sketchy3DIS python=3.8
  conda activate Sketchy3DIS
  

• Install the dependencies

  Install Pytorch 1.10

  pip install spconv-cu113
  conda install pytorch-scatter -c pyg
  pip install -r requirements.txt
  

  Install segmentator from this repo.

• Setup, Install spformer and pointgroup_ops.

  sudo apt-get install libsparsehash-dev
  python setup.py develop
  cd spformer/lib/
  python setup.py develop
  

Data Preparation

ScanNet v2 dataset

Download the ScanNet v2 dataset.

Put the downloaded scans and scans_test folder as follows.

MMImp
├── data
│   ├── scannetv2
│   │   ├── scans
│   │   ├── scans_test

Split and preprocess data

cd Sketchy3DIS/SPFormer/data/scannetv2
bash prepare_data.sh

The script data into train/val/test folder and preprocess the data. After running the script the scannet dataset structure should look like below.

SPFormer
├── data
├── dataset
│   ├── scannetv2
│   │   ├── scans
│   │   ├── scans_test
│   │   ├── train
│   │   ├── val
│   │   ├── test
│   │   ├── superpoints
│   │   ├── normals

S3DIS dataset

Download the S3DIS dataset (v1.2_Aligned_Version).

Download the preprocessed superpoints and normals from Box2Mask, and organize as below.

ISBNet
├── dataset
│   ├── s3dis
│   │   ├── Stanford3dDataset_v1.2_Aligned_Version
│   │   │   ├── Area_1
│   │   │   │   ├── hallway_1 
│   │   │   │   │   ├── Annotations # Contains instances information 
│   │   │   │   │   │   ├── door_2.txt 
│   │   │   │   │   │   ├── floor_1.txt
│   │   │   │   │   │   ├── wall_2.txt
│   │   │   │   │   │   ├── ...
│   │   │   │   │   ├── hallway_1.txt # Contains positions and colors of scene points
│   │   │   │   ├── office_1
│   │   │   │   ├── ...
│   │   │   ├── Area_2
│   │   │   ├── Area_3
│   │   │   ├── Area_4
│   │   │   ├── Area_5
│   │   │   ├── Area_6
│   │   ├── learned_superpoin_graph_segmentations
│   │   ├── normals

Preprocess data

cd Sketchy3DIS/ISBNet/dataset/s3dis
bash prepare_data.sh

After running the script the s3dis dataset structure should look like below.

ISBNet
├── dataset
│   ├── s3dis
│   │   ├── Stanford3dDataset_v1.2_Aligned_Version
│   │   ├── learned_superpoin_graph_segmentations
│   │   ├── preprocess
│   │   ├── superpoints
│   │   ├── normals

Training

The training steps are the same as the corresponding origin repositories. More details can be referred to SPFormer, ISBNet.

Acknowledgements

This repo is built upon SPFormer,ISBNet.

Citation

If you find this project useful, please consider citing:

@InProceedings{Deng_2025_CVPR,
    author    = {Deng, Qian and Hui, Le and Xie, Jin and Yang, Jian},
    title     = {Sketchy Bounding-box Supervision for 3D Instance Segmentation},
    booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
    month     = {June},
    year      = {2025},
    pages     = {8879-8888}
}