The official implementation of our ACM Multimedia 2024 paper "Model X-ray: Detecting Backdoored Models via Decision Boundary".[Paper]
Backdoor attacks pose a significant security vulnerability for deep neural networks (DNNs), enabling them to operate normally on clean inputs but manipulate predictions when specific trigger patterns occur. Currently, post-training backdoor detection approaches often operate under the assumption that the defender has knowledge of the attack information, logit output from the model, and knowledge of the model parameters. In contrast, our approach functions as a lightweight diagnostic scanning tool offering interpretability and visualization. By accessing the model to obtain hard labels, we construct decision boundaries within the convex combination of three samples. We present an intriguing observation of two phenomena in backdoored models: a noticeable shrinking of areas dominated by clean samples and a significant increase in the surrounding areas dominated by target labels. Leveraging this observation, we propose Model X-ray, a novel backdoor detection approach based on the analysis of illustrated two-dimensional (2D) decision boundaries. Our approach includes two strategies focused on the decision areas dominated by clean samples and the concentration of label distribution, and it can not only identify whether the target model is infected but also determine the target attacked label under the all-to-one attack strategy. Importantly, it accomplishes this solely by the predicted hard labels of clean inputs, regardless of any assumptions about attacks and prior knowledge of the training details of the model. Extensive experiments demonstrated that Model X-ray has outstanding effectiveness and efficiency across diverse backdoor attacks, datasets, and architectures. Besides, ablation studies on hyperparameters and more attack strategies and discussions are also provided.
- Get Model X-ray
git clone https://github.com/SuYanghao/Model_X-ray.git
cd Model_X-ray- Get BackdoorBench-v2
Merge Model X-ray into the BackdoorBench-v2 codebase
git clone https://github.com/SCLBD/BackdoorBench.git
rsync -av BackdoorBench-v2-merge/BackdoorBench/
cd BackdoorBench
conda create -n backdoorbench python=3.8
conda activate backdoorbench
sh ./sh/install.sh
sh ./sh/init_folders.sh
mkdir record
mkdir data
mkdir data/cifar10
mkdir data/cifar100
mkdir data/gtsrb
mkdir data/tiny- Train a Backdoor Model
python ./attack/badnet.py --yaml_path ../config/attack/badnet/cifar10.yaml --dataset cifar10 --dataset_path ../data --model preactresnet18 --save_folder_name cifar10_preactresnet18_badnet
- Model X-ray Evaluation
python analysis/modelxray.py --result_file cifar10_preactresnet18_badnet --yaml_path ./config/visualization/modelxray.yaml --dataset cifar10 --model preactresnet18
For guidance on conducting more evaluations, such as using different attacks, datasets, and model architectures, please refer to BackdoorBench.
If you find Model X-ray both interesting and helpful, please consider citing us in your research or publications:
@inproceedings{su2024model,
title={Model X-ray: Detecting Backdoored Models via Decision Boundary},
author={Su, Yanghao and Zhang, Jie and Xu, Ting and Zhang, Tianwei and Zhang, Weiming and Yu, Nenghai},
booktitle={Proceedings of the 32nd ACM International Conference on Multimedia},
pages={10296--10305},
year={2024}
}@inproceedings{backdoorbench,
title={BackdoorBench: A Comprehensive Benchmark of Backdoor Learning},
author={Wu, Baoyuan and Chen, Hongrui and Zhang, Mingda and Zhu, Zihao and Wei, Shaokui and Yuan, Danni and Shen, Chao},
booktitle={Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year={2022}
}