Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Model X-ray:Detecting Backdoored Models via Decision Boundary

Yanghao Su, Jie Zhang, Ting Xu, Tianwei Zhang, Weiming Zhang, Nenghai Yu

TL;DR

Model X-ray is proposed, a novel backdoor detection approach based on the analysis of illustrated two-dimensional decision boundaries that can not only identify whether the target model is infected but also determine the target attacked label under the all-to-one attack strategy.

Abstract

Backdoor attacks pose a significant security vulnerability for deep neural networks (DNNs), enabling them to operate normally on clean inputs but manipulate predictions when specific trigger patterns occur. Currently, post-training backdoor detection approaches often operate under the assumption that the defender has knowledge of the attack information, logit output from the model, and knowledge of the model parameters. In contrast, our approach functions as a lightweight diagnostic scanning tool offering interpretability and visualization. By accessing the model to obtain hard labels, we construct decision boundaries within the convex combination of three samples. We present an intriguing observation of two phenomena in backdoored models: a noticeable shrinking of areas dominated by clean samples and a significant increase in the surrounding areas dominated by target labels. Leveraging this observation, we propose Model X-ray, a novel backdoor detection approach based on the analysis of illustrated two-dimensional (2D) decision boundaries. Our approach includes two strategies focused on the decision areas dominated by clean samples and the concentration of label distribution, and it can not only identify whether the target model is infected but also determine the target attacked label under the all-to-one attack strategy. Importantly, it accomplishes this solely by the predicted hard labels of clean inputs, regardless of any assumptions about attacks and prior knowledge of the training details of the model. Extensive experiments demonstrated that Model X-ray has outstanding effectiveness and efficiency across diverse backdoor attacks, datasets, and architectures. Besides, ablation studies on hyperparameters and more attack strategies and discussions are also provided.

Model X-ray:Detecting Backdoored Models via Decision Boundary

TL;DR

Model X-ray is proposed, a novel backdoor detection approach based on the analysis of illustrated two-dimensional decision boundaries that can not only identify whether the target model is infected but also determine the target attacked label under the all-to-one attack strategy.

Abstract

Backdoor attacks pose a significant security vulnerability for deep neural networks (DNNs), enabling them to operate normally on clean inputs but manipulate predictions when specific trigger patterns occur. Currently, post-training backdoor detection approaches often operate under the assumption that the defender has knowledge of the attack information, logit output from the model, and knowledge of the model parameters. In contrast, our approach functions as a lightweight diagnostic scanning tool offering interpretability and visualization. By accessing the model to obtain hard labels, we construct decision boundaries within the convex combination of three samples. We present an intriguing observation of two phenomena in backdoored models: a noticeable shrinking of areas dominated by clean samples and a significant increase in the surrounding areas dominated by target labels. Leveraging this observation, we propose Model X-ray, a novel backdoor detection approach based on the analysis of illustrated two-dimensional (2D) decision boundaries. Our approach includes two strategies focused on the decision areas dominated by clean samples and the concentration of label distribution, and it can not only identify whether the target model is infected but also determine the target attacked label under the all-to-one attack strategy. Importantly, it accomplishes this solely by the predicted hard labels of clean inputs, regardless of any assumptions about attacks and prior knowledge of the training details of the model. Extensive experiments demonstrated that Model X-ray has outstanding effectiveness and efficiency across diverse backdoor attacks, datasets, and architectures. Besides, ablation studies on hyperparameters and more attack strategies and discussions are also provided.
Paper Structure (30 sections, 6 equations, 26 figures, 9 tables)

This paper contains 30 sections, 6 equations, 26 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Backdoor Attacks
  4. Backdoor Defenses
  5. Decision Boundary of Deep Neural Networks
  6. Preliminaries
  7. Recap of the Decision Boundary in somepalli2022can
  8. Threat Model
  9. Method
  10. An Intriguing Observation
  11. Two Strategies for Backdoor Detection via the Decision Boundary
  12. Backdoor Detection based on Rényi Entropy
  13. Backdoor Detection based on Areas dominated by Triple Samples
  14. Determine the Target Label
  15. Experiment
  16. ...and 15 more sections

Figures (26)

  • Figure 1: Comparison of the decision boundaries between the clean model and the backdoored model (taking BadNets gu2017badnets as an example, and the target label is "airplane") on the CIFAR-10 dataset.
  • Figure 2: The pipeline of the backdoor defense.
  • Figure 3: Visual examples of the decision boundary used in somepalli2022can (left) and in this paper (right).
  • Figure 4: Visual examples of decision boundaries of the clean model and different backdoored models on CIFAR-10 and ImageNet-10.
  • Figure 5: Illustration on calculation of RE and ATS.
  • ...and 21 more figures