ReferEverything [ICCV 2025]

Anurag Bagchi · Zhipeng Bao · Yu-Xiong Wang · Pavel Tokmakov · Martial Hebert

Official PyTorch implementation of the ICCV 2025 paper "ReferEverything".

---

TL;DR:

We present Refer Everything Model (REM) by re-purposing Text-to-Video generation models to zero-shot segment any concept in a Video using Text.

📰 News

• [Coming Soon] Interactive demos, datasets, Mevis ckpts
• [Oct, 2025] Released the code and pretrained checkpoints for ModelScopeT2V-1.4B and Wan2.1-14B.

📦 Installation

1. Clone this repository

git clone https://github.com/yourusername/ReferEverything.git
cd ReferEverything

2. Install the Modelscope REM environment

conda env create -f MS_env.yml
conda activate MS_env

3. Install the Wan REM environment

conda env create -f Wan_env.yml
conda activate Wan_env

4. Download Checkpoints

Finetuned checkpoints for both models can be downloaded from Huggingface

Run REM on your samples

Modelscope

bash run_REM_MS_sample.sh #Change the arguments in the script accordingly.

Wan2.1

The Wan2.1-T2V-14B model is quite large. Please download the base Wan2.1-T2V-14B model from Huggingface to an approriate disk with enough space.

bash run_REM_Wan14b_sample.sh #Change the arguments in the script accordingly.

Data Preparation

We use RefCOCO/+/g and Refer-Youtube to train REM. Please follow ReferFormer to prepare the training data.

Train REM

ModelScope

Train the spatial weights on Refcoco/+/g

bash train_REM_MS_imgs.sh #Change the arguments in the script accordingly.

Train on Refer-Youtube

bash train_REM_MS_vid.sh #Change the arguments in the script accordingly.

Wan2.1

To save memory during training we pre-compute the T5 text embeddings using utils/encode_wantxt_T5.py

Train jointly on Refer-Youtube and Refcoco/+/g

bash train_REM_Wan.sh #Change the arguments in the script accordingly.

Infer on datasets

Please follow the instructions in Ref-Davis, Ref-Youtube, Burst, VSPW-stuff