Code release of paper:
Chao Yin, Hao Li, Kequan Yang, Jide Li, Pinpin Zhu, Xiaoqiang Li
Shanghai University, University of the Chinese Academy of Sciences
- [2025.8.15] Demo of RDVP-MSD is released.
- [2025.8.14] the code of RDVP-MSD is released.
- [2025.8.14] the results of the LLaVA1.5 version RDVP-MSD on multiple COS datasets are released.
While promptable segmentation (e.g., SAM) has shown promise for various segmentation tasks, it still requires manual visual prompts for each object to be segmented. In contrast, task-generic promptable segmentation aims to reduce the need for such detailed prompts by employing only a task-generic prompt to guide segmentation across all test samples. However, when applied to Camouflaged Object Segmentation (COS), current methods still face two critical issues: 1) semantic ambiguity in getting instance-specific text prompts, which arises from insufficient discriminative cues in holistic captions, leading to foreground-background confusion; 2) semantic discrepancy combined with spatial separation in getting instance-specific visual prompts, which results from global background sampling far from object boundaries with low feature correlation, causing SAM to segment irrelevant regions. To mitigate the issues above, we propose RDVP-MSD, a novel training-free test-time adaptation framework that synergizes Region-constrained Dual-stream Visual Prompting (RDVP) via Multimodal Stepwise Decomposition Chain of Thought (MSD-CoT).
MSD-CoT progressively disentangles image captions to eliminate semantic ambiguity, while RDVP injects spatial constraints into visual prompting and independently samples visual prompts for foreground and background points, effectively mitigating semantic discrepancy and spatial separation.
- Download the datasets from the follow links:
Camouflaged Object Segmentation Datasets
- Put it in
../Dataset/TestDataset/.
# create and activate conda environment
conda create -n RDVP-MSD python=3.8
conda activate RDVP-MSD
# pip install torch
pip install torch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 --index-url https://download.pytorch.org/whl/cu118
# LLaVA
cd LLaVA-1.1.0
pip install --upgrade pip # enable PEP 660 support
pip install -e .
cd ..
# SAM-HQ
pip install segment-anything-hq
pip install opencv-python imageio ftfyOur RDVP-MSD is a training-free test-time adaptation approach, so you can play with it by running:
CUDA_VISIBLE_DEVICES=0,1 python main.py --config config/COD10K_LLaVA1.5.yaml --save_path ./res/prediction_RDVP_MSD/The more qualitative results of RDVP-MSD on four benchmarks (NC4K, COD10K, CAMO, CHAMELEON) have already been stored in Google Drive, please unzip it into the fold './res/'.
evaluate the results on COD10K dataset:
python eval/test_metrics.pyThe quantitative results will be save in ./res/prediction_RDVP_MSD/log.txt
The more results of evaluation are also in Google Drive.
We further prepare a jupyter notebook demo for visualization.
- Complete the following steps in the shell before opening the jupyter notebook.
The virtualenv environment named RDVP-MSD needs to be created first following Environment.
pip install notebook
pip install ipykernel ipywidgets
python -m ipykernel install --user --name RDVP-MSD
- Open demo.ipynb and select the '-' kernel in the running notebook.
- Update datasets and implementation scripts
- Demo and Codes
If you find our work useful in your research, please consider citing:
@inproceedings{yin2025stepwise,
title={Stepwise Decomposition and Dual-stream Focus: A Novel Approach for Training-free Camouflaged Object Segmentation},
author={Yin, Chao and Li, Hao and Yang, Kequan and Li, Jide and Zhu, Pinpin and Li, Xiaoqiang},
booktitle={ACM MM},
year={2025}
}
The source code is free for research and education use only. Any commercial use should get formal permission first.