SCFlow2: Plug-and-Play Object Pose Refiner with Shape-Constraint Scene Flow (CVPR 2025)

Qingyuan Wang, Rui Song, Jiaojiao Li, Kerui Cheng, David Ferstl, Yinlin Hu

🔍Introduction

Overview of SCFlow2. Given an RGBD image and an initial pose of the target, we first render the target to obtain a synthetic RGBD image as the reference, and use an RGB encoder and depth encoder to extract features from the image pair, which will be used to create a 4D correlation volume. Based on the correlation volume and GRU, We use an intermediate flow regressor to predict 3D scene flow that is represented as a dense 3D transformation field T^′_k. We then use a pose regressor to predict a global pose update ∆P_k based on implicit voting from the pixel-wise 3D transformation field. Finally, the updated pose P_k is used to compute the pose-induced flow F_k based on the target mesh to index the correlation volume for the next iteration. Note how the depth and 3D shape of the target are embedded into the framework to guide the optimization iteratively.

📝 Checklist

• ✅ Upload inference code
• ✅ Upload data processing code
• ✅ Upload Upload pre-trained weights
• ⬛ Upload training code

🛠️ Installation

Basic Dependencies:

• CUDA Version >= 11.8
• Python >= 3.8
• Pytorch >= 2.0.0

Create a new environment:

conda create -n scflow2 python==3.10.18
conda activate scflow2

Install necessary packages:

# Configure basic environment
git clone https://github.com/W-QY/SCFlow2
cd SCFlow2
pip install -r requirements.txt

# Install mmcv
python -m pip install openmim
mim install mmcv_full==1.7.2

# Install pointnet2_ops
git clone https://github.com/erikwijmans/Pointnet2_PyTorch
cd Pointnet2_PyTorch/pointnet2_ops_lib
python setup.py install

# Install bop_toolkit
git clone https://github.com/thodan/bop_toolkit
cd bop_toolkit
python setup.py develop

📁 Dataset Preparation

Data Structure：

We follow the data construction methods of SCFlow and SAM6D :

data
├── Training-Data
    ├── MegaPose-GSO
        ├──google_scanned_objects
        ├──templates
        └──train_pbr_web
    ├── MegaPose-ShapeNetCore
        ├──shapenetcorev2
        ├──templates
        └──train_pbr_web
    ├── FoundataionPose-Objaverse
        ├──shapenetcorev2
        ├──templates
        └──train_pbr_web
└── Testing-BOP   # https://bop.felk.cvut.cz/datasets/
    ├──tudl
    ├──lmo
    ├──ycbv
    ├──icbin
    ├──hb
    ├──itodd
    └──tless

Data Processing:

For MegaPose-GSO and MegaPose-ShapeNetCore, you can refer to this link to download.

For FoundataionPose-Objaverse, you can download the raw data from this link, and then use the tools/objaverse_process.py script to process the data into BOP format.

For Testing-BOP datasets, you need to use the tools/collect_imagelist.py script to get the paths to all images in each test_bop19 folder and add it to image_lists. You can download ycbv image_lists and place them under the data/Testing-BOP/ycbv/image_lists directory.

📦 Pretrained weights:

We provide weights pre-trained using the MegaPose-GSO, MegaPose-ShapeNetCore and FoundataionPose-Objaverse datasets at here.

🤖 Testing

Prepare initial pose:

You can use the tools/bop19_to_refannots.py script to process any BOP format .csv file results and use it as the initial pose, or download PoseCNN initial pose and place it under data/initial_poses/ycbv_posecnn directory.

Evaluate the performance and save the results:

python test.py --config configs/refine_models/scflow2.py  --checkpoint *** --format-only --save-dir ***

📑 Citation

If you find this project is helpful, please cite:

@inproceedings{wang2025scflow2,
  title={SCFlow2: Plug-and-Play Object Pose Refiner with Shape-Constraint Scene Flow},
  author={Qingyuan Wang and Rui Song and Jiaojiao Li and Kerui Cheng and David Ferstl and Yinlin Hu},
  booktitle={CVPR},
  year={2025}
}

@inproceedings{yang2023scflow,
  title={Shape-Constraint Recurrent Flow for 6D Object Pose Estimation},
  author={Yang Hai and Rui Song and Jiaojiao Li and Yinlin Hu},
  booktitle={CVPR},
  year={2023}
}

👍Acknowledgement

We build this project based on SCFlow, RAFT-3D, SAM6D and MMFlow, We thank the authors for their great code repositories.