Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TERD: A Unified Framework for Safeguarding Diffusion Models Against Backdoors

Yichuan Mo, Hui Huang, Mingjie Li, Ang Li, Yisen Wang

TL;DR

TERD presents a unified framework for defending diffusion models against backdoors by unifying attack dynamics into a single reversed-loss and employing a two-stage trigger-reversion process. It then enables backdoor detection from the noise space via input-probability comparisons and from the model space via KL-divergence-based metrics between reversed and benign triggers. Empirically, TERD achieves 100% TPR and TNR across multiple backdoor attacks on CIFAR-10 and high-resolution datasets, with strong transferability to SDE-based models and minimal runtime overhead. The approach offers a practical blueprint for securing diffusion systems and potentially other stochastic differential equation-based generative models.

Abstract

Diffusion models have achieved notable success in image generation, but they remain highly vulnerable to backdoor attacks, which compromise their integrity by producing specific undesirable outputs when presented with a pre-defined trigger. In this paper, we investigate how to protect diffusion models from this dangerous threat. Specifically, we propose TERD, a backdoor defense framework that builds unified modeling for current attacks, which enables us to derive an accessible reversed loss. A trigger reversion strategy is further employed: an initial approximation of the trigger through noise sampled from a prior distribution, followed by refinement through differential multi-step samplers. Additionally, with the reversed trigger, we propose backdoor detection from the noise space, introducing the first backdoor input detection approach for diffusion models and a novel model detection algorithm that calculates the KL divergence between reversed and benign distributions. Extensive evaluations demonstrate that TERD secures a 100% True Positive Rate (TPR) and True Negative Rate (TNR) across datasets of varying resolutions. TERD also demonstrates nice adaptability to other Stochastic Differential Equation (SDE)-based models. Our code is available at https://github.com/PKU-ML/TERD.

TERD: A Unified Framework for Safeguarding Diffusion Models Against Backdoors

TL;DR

TERD presents a unified framework for defending diffusion models against backdoors by unifying attack dynamics into a single reversed-loss and employing a two-stage trigger-reversion process. It then enables backdoor detection from the noise space via input-probability comparisons and from the model space via KL-divergence-based metrics between reversed and benign triggers. Empirically, TERD achieves 100% TPR and TNR across multiple backdoor attacks on CIFAR-10 and high-resolution datasets, with strong transferability to SDE-based models and minimal runtime overhead. The approach offers a practical blueprint for securing diffusion systems and potentially other stochastic differential equation-based generative models.

Abstract

Diffusion models have achieved notable success in image generation, but they remain highly vulnerable to backdoor attacks, which compromise their integrity by producing specific undesirable outputs when presented with a pre-defined trigger. In this paper, we investigate how to protect diffusion models from this dangerous threat. Specifically, we propose TERD, a backdoor defense framework that builds unified modeling for current attacks, which enables us to derive an accessible reversed loss. A trigger reversion strategy is further employed: an initial approximation of the trigger through noise sampled from a prior distribution, followed by refinement through differential multi-step samplers. Additionally, with the reversed trigger, we propose backdoor detection from the noise space, introducing the first backdoor input detection approach for diffusion models and a novel model detection algorithm that calculates the KL divergence between reversed and benign distributions. Extensive evaluations demonstrate that TERD secures a 100% True Positive Rate (TPR) and True Negative Rate (TNR) across datasets of varying resolutions. TERD also demonstrates nice adaptability to other Stochastic Differential Equation (SDE)-based models. Our code is available at https://github.com/PKU-ML/TERD.
Paper Structure (35 sections, 1 theorem, 46 equations, 6 figures, 10 tables, 5 algorithms)

This paper contains 35 sections, 1 theorem, 46 equations, 6 figures, 10 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Backdoor Attacks in Diffusion Models
  4. Existing Backdoor Defense
  5. Preliminary
  6. Discrete Diffusion Model
  7. Continuous Diffusion Model
  8. Backdoor Diffusion Model
  9. Reverse Engineering
  10. A Unified Loss for Trigger Reversion
  11. Trigger Estimation
  12. Trigger Refinement
  13. Backdoor Detection
  14. Input Detection
  15. Model Detection
  16. ...and 20 more sections

Key Result

Theorem 4.1

Given the target image $\mathbf{x}_0\sim p_{target}$ and a surrogate image $\hat{\mathbf{x}}_0\sim\hat{p}_{prior}$, let $\mathbf{p}_t$ and $\mathbf{q}_t$ denotes the distribution of $\mathbf{x}_0$ and $\hat{\mathbf{x}}_0$ at timestep $t$. we can prove that:

Figures (6)

  • Figure 1: An illustration for our proposed reverse engineering method.
  • Figure 2: $M_\textbf{r}$ and $V_\textbf{r}$ for clean and backdoor models.
  • Figure 3: Reversed results. From left to right: one of the triggers inserted by badiffusion, TrojDiff and VillanDiffusion. As can be seen, TERD more accurately reverses the triggers.
  • Figure 4: The performance of our proposed model detection against the adaptive attack.
  • Figure 5: The effect of each component on the final performances of our proposed defense. The best results are in bold.
  • ...and 1 more figures

Theorems & Definitions (1)

  • Theorem 4.1