VT-Refine

VT-Refine is the official code release for the CoRL 2025 paper:

VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning
Binghao Huang, Jie Xu, Iretiayo Akinola, Wei Yang, Balakumar Sundaralingam, Rowland O'Flaherty, Dieter Fox, Xiaolong Wang, Arsalan Mousavian, Yu-Wei Chao†, Yunzhu Li†
Conference on Robot Learning (CoRL), 2025
[ OpenReview ] [ arXiv ] [ video ] [ poster ] [ project site ]

Citing VT-Refine

@INPROCEEDINGS{huang:corl2025,
  author    = {Binghao Huang and Jie Xu and Iretiayo Akinola and Wei Yang and Balakumar Sundaralingam and Rowland O'Flaherty and Dieter Fox and Xiaolong Wang and Arsalan Mousavian and Yu-Wei Chao and Yunzhu Li},
  booktitle = {Conference on Robot Learning (CoRL)},
  title     = {{VT-Refine}: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning},
  year      = {2025},
}

License

VT-Refine is released under the NVIDIA License.

The pre-training data are licensed under CC BY-NC 4.0.

Contents

1. Installation
2. Pre-training
3. Evaluation - Pre-training
4. Fine-tuning
5. Evaluation - Fine-tuning

Installation

git clone --recursive https://github.com/NVlabs/vt-refine.git
cd vt-refine

./docker/build.sh

DPPO_WANDB_ENTITY=$DPPO_WANDB_ENTITY ./docker/run.sh

Pre-training

./data/fetch_vt-refine_data.sh

# Aloha - 00186
python3 dppo/script/run.py \
    --config-name=pre_tactile \
    --config-path=../cfg/aloha/pretrain/00186

# SRL_Rummy - 00186
python3 dppo/script/run.py \
    --config-name=pre_tactile \
    --config-path=../cfg/srl_rummy/pretrain/00186

Evaluation - Pre-training

# Aloha - 00186
python3 dppo/script/run.py \
    --config-name=eval_pre_tactile \
    --config-path=../cfg/aloha/eval/00186 \
    base_policy_path=$PRETRAIN_POLICY_PATH

# SRL_Rummy - 00186
python3 dppo/script/run.py \
    --config-name=eval_pre_tactile \
    --config-path=../cfg/srl_rummy/eval/00186 \
    base_policy_path=$PRETRAIN_POLICY_PATH

Fine-tuning

# Aloha - 00186
python3 dppo/script/run.py \
    --config-name=ft_tactile \
    --config-path=../cfg/aloha/finetune/00186 \
    base_policy_path=$PRETRAIN_POLICY_PATH

# SRL_Rummy - 00186
python3 dppo/script/run.py \
    --config-name=ft_tactile \
    --config-path=../cfg/srl_rummy/finetune/00186 \
    base_policy_path=$PRETRAIN_POLICY_PATH

Evaluation - Fine-tuning

# Aloha - 00186
python3 dppo/script/run.py \
    --config-name=eval_ft_tactile \
    --config-path=../cfg/aloha/eval/00186 \
    base_policy_path=$FINETUNE_POLICY_PATH

# SRL_Rummy - 00186
python3 dppo/script/run.py \
    --config-name=eval_ft_tactile \
    --config-path=../cfg/srl_rummy/eval/00186 \
    base_policy_path=$FINETUNE_POLICY_PATH