Bayesian Principles Improve Prompt Learning In Vision-Language Models

[ArXiv] [Project]

Official PyTorch implementation of BP CLIP, as presented in our paper:

Bayesian Principles Improve Prompt Learning In Vision-Language Models (AISTATS2025)
Mingyu Kim^1*, Jongwoo Ko^2*, and Mijung Park^3
1UBC CS, ²KAIST AI, ³UBC CS
(^* indicates co-first authors)

Update

• Training code.
• Inference code.
• Datasets.

🚀 How to Run

Official scripts are provided under the following directories:

• scripts/coop_ove_pg/
• scripts/cocoop_ove_pg/
• scripts/maple_ove_pg/
• scripts/apex_ove_pg/

Note:
Replace {model} with one of: coop, cocoop, maple, apex.
Make sure to set the correct DATA path before running any script.
Run the commands inside the corresponding script directories.

---

🟦 1. Generalization: Base to New Classes

You will need two scripts:

• base2new_train.sh   →  Train on base classes
• base2new_test.sh   →  Evaluate on novel classes

Each script accepts the following arguments:

• DATASET   (e.g., UCF101, caltech101, corresponding to files in configs/datasets/)
• SEED   (random seed)
• GPUIDS   (comma-separated GPU id list)
• EPOCHS   (number of training epochs)
• LAMBDA1   (weight for KLD)

Example:

# Generalization from base to new classes
python scripts/{model}_ove_pg/script_base2new_all_datasets.py --gpuids 0 --epochs 10 

---

🟩 2. Cross-Dataset Transfer

For cross-dataset transfer experiments, set lambda_1 to the default value of 0.2.

You will need two scripts:

• xd_train.sh   →  Train on ImageNet
• xd_test.sh   →  Evaluate on downstream datasets (e.g., UCF101, caltech101, corresponding to files in configs/datasets/)

Example usage:

# Training ImageNet dataset and Evaluate downstreamed datset
python scripts/{model}_ove_pg/script_cross_all_datasets.py --gpuids 0 --epochs 10