VLM-R1

1. Setup

Dependencies Installation

# Install transformers from source
pip install git+ssh://[email protected]/huggingface/transformers.git@89d27fa6fff206c0153e9670ae09e2766eb75cdf


# Run setup script
bash setup.sh

# Install Qwen related packages
pip install git+ssh://[email protected]/cjakfskvnad/Qwen-Agent.git
pip install qwen-vl-utils
pip install torch==2.6.0 torchvision==0.21.0

2. RL Training

Run the training script:

bash src/open-r1-multimodal/run_grpo_rec.sh

Training outputs will be saved to: src/open-r1-multimodal/output

3. Supervised Fine-Tuning (SFT)

Running the SFT Script

bash /data/muze/VLM-R1/src/open-r1-multimodal/sft.sh

Data Configuration

Before running the script, please ensure your data follows the required format as shown in processed_data.json.

4. Evaluation

Note: Before running evaluation, make sure to configure the path to your model.

Multiple Checkpoints Evaluation

bash src/open-r1-multimodal/src/open_r1/eval.sh

Single Checkpoint Evaluation

bash src/open-r1-multimodal/src/open_r1/eval_1.sh