Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Consistency Purification: Effective and Efficient Diffusion Purification towards Certified Robustness

Yiquan Li, Zhongzhu Chen, Kun Jin, Jiongxiao Wang, Bo Li, Chaowei Xiao

TL;DR

This work tackles the efficiency–effectiveness tension in diffusion-based purification for randomized smoothing. It introduces Consistency Purification, which uses a consistency model distilled from the probability-flow ODE to perform one-step, on-manifold purification, and further refines it with Consistency Fine-tuning using LPIPS to preserve semantic alignment. Theoretical analysis links purification quality to transport between data and purified distributions, and empirical results on CIFAR-10 and ImageNet-64 show state-of-the-art certified robustness and efficiency, with clear gains from the fine-tuning stage. The approach enables highly robust predictions with a single network evaluation for purification, suggesting practical impact for scalable certified robustness in vision systems.

Abstract

Diffusion Purification, purifying noised images with diffusion models, has been widely used for enhancing certified robustness via randomized smoothing. However, existing frameworks often grapple with the balance between efficiency and effectiveness. While the Denoising Diffusion Probabilistic Model (DDPM) offers an efficient single-step purification, it falls short in ensuring purified images reside on the data manifold. Conversely, the Stochastic Diffusion Model effectively places purified images on the data manifold but demands solving cumbersome stochastic differential equations, while its derivative, the Probability Flow Ordinary Differential Equation (PF-ODE), though solving simpler ordinary differential equations, still requires multiple computational steps. In this work, we demonstrated that an ideal purification pipeline should generate the purified images on the data manifold that are as much semantically aligned to the original images for effectiveness in one step for efficiency. Therefore, we introduced Consistency Purification, an efficiency-effectiveness Pareto superior purifier compared to the previous work. Consistency Purification employs the consistency model, a one-step generative model distilled from PF-ODE, thus can generate on-manifold purified images with a single network evaluation. However, the consistency model is designed not for purification thus it does not inherently ensure semantic alignment between purified and original images. To resolve this issue, we further refine it through Consistency Fine-tuning with LPIPS loss, which enables more aligned semantic meaning while keeping the purified images on data manifold. Our comprehensive experiments demonstrate that our Consistency Purification framework achieves state-of the-art certified robustness and efficiency compared to baseline methods.

Consistency Purification: Effective and Efficient Diffusion Purification towards Certified Robustness

TL;DR

This work tackles the efficiency–effectiveness tension in diffusion-based purification for randomized smoothing. It introduces Consistency Purification, which uses a consistency model distilled from the probability-flow ODE to perform one-step, on-manifold purification, and further refines it with Consistency Fine-tuning using LPIPS to preserve semantic alignment. Theoretical analysis links purification quality to transport between data and purified distributions, and empirical results on CIFAR-10 and ImageNet-64 show state-of-the-art certified robustness and efficiency, with clear gains from the fine-tuning stage. The approach enables highly robust predictions with a single network evaluation for purification, suggesting practical impact for scalable certified robustness in vision systems.

Abstract

Diffusion Purification, purifying noised images with diffusion models, has been widely used for enhancing certified robustness via randomized smoothing. However, existing frameworks often grapple with the balance between efficiency and effectiveness. While the Denoising Diffusion Probabilistic Model (DDPM) offers an efficient single-step purification, it falls short in ensuring purified images reside on the data manifold. Conversely, the Stochastic Diffusion Model effectively places purified images on the data manifold but demands solving cumbersome stochastic differential equations, while its derivative, the Probability Flow Ordinary Differential Equation (PF-ODE), though solving simpler ordinary differential equations, still requires multiple computational steps. In this work, we demonstrated that an ideal purification pipeline should generate the purified images on the data manifold that are as much semantically aligned to the original images for effectiveness in one step for efficiency. Therefore, we introduced Consistency Purification, an efficiency-effectiveness Pareto superior purifier compared to the previous work. Consistency Purification employs the consistency model, a one-step generative model distilled from PF-ODE, thus can generate on-manifold purified images with a single network evaluation. However, the consistency model is designed not for purification thus it does not inherently ensure semantic alignment between purified and original images. To resolve this issue, we further refine it through Consistency Fine-tuning with LPIPS loss, which enables more aligned semantic meaning while keeping the purified images on data manifold. Our comprehensive experiments demonstrate that our Consistency Purification framework achieves state-of the-art certified robustness and efficiency compared to baseline methods.
Paper Structure (15 sections, 1 theorem, 11 equations, 5 figures, 4 tables, 2 algorithms)

This paper contains 15 sections, 1 theorem, 11 equations, 5 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Backgrounds
  3. Theoretical Analysis
  4. Method
  5. Consistency Purification
  6. Consistency Fine-tuning
  7. Experiments
  8. Experimental Settings.
  9. Main Results.
  10. Ablation Studies.
  11. Conclusion
  12. Consistency Purification Algorithm
  13. Proof of Theorem \ref{['thm:tranport']}
  14. Training Unconditional Consistency Model for ImageNet-64
  15. Certified Accuracy with Fine-grained $\epsilon$

Key Result

Theorem 3.3

Given the transport $T_{\pi_t}(p)$ between the data distribution $p$ and the corresponding purified distribution under $g_t$, then for any $r > 0$, the probability that the distance between the original sample ${\bm{x}}$ and purified sample $\hat{{\bm{x}}} = \pi_t({\bm{x}})$ is larger than $r$ is up

Figures (5)

  • Figure 1: An illustration of Consistency Purification framework.
  • Figure 2: Transport between purified images and clean images with $\sigma \in \{0.25,0.5,0.75,1.0\}$.
  • Figure 3: Certified Accuracy of Consistency Purification with different loss functions during fine-tuning for CIFAR-10. "- -" represents the setting without fine-tuning.
  • Figure 4: Certified Accuracy of Consistency Purification with continuous and discrete sampling schedules during fine-tuning for CIFAR-10. "- -" represents the setting without fine-tuning.
  • Figure 5: Left figure shows experiments on CIFAR-10, right figure shows experiments on ImageNet-64. The lines demonstrate the certified accuracy with different $\ell_2$ perturbation bound with different Gaussian noise levels.

Theorems & Definitions (5)

  • Example 3.1
  • Definition 3.2
  • Theorem 3.3
  • Remark 3.4
  • proof