Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Purify++: Improving Diffusion-Purification with Advanced Diffusion Models and Control of Randomness

Boya Zhang, Weijian Luo, Zhihua Zhang

TL;DR

Purify++ addresses adversarial robustness by enhancing diffusion purification through three targeted improvements: a stronger diffusion model (EDM/VE-based forward diffusion), efficient yet effective purification simulation (favoring stable, low-NFE Heun-style updates), and an optimal randomness control that mixes ODE and Langevin dynamics. It demonstrates that these components yield state-of-the-art robustness on CIFAR-10 and MNIST across black-box, gray-box, and adaptive attacks, while maintaining solid standard accuracy. The work provides both theoretical and empirical insights into diffusion-purification design, including how forward diffusion choice and stochasticity influence purification effectiveness. Practically, Purify++ offers a more reliable, compute-budget-friendly purification paradigm that can be deployed without retraining classifiers.

Abstract

Adversarial attacks can mislead neural network classifiers. The defense against adversarial attacks is important for AI safety. Adversarial purification is a family of approaches that defend adversarial attacks with suitable pre-processing. Diffusion models have been shown to be effective for adversarial purification. Despite their success, many aspects of diffusion purification still remain unexplored. In this paper, we investigate and improve upon three limiting designs of diffusion purification: the use of an improved diffusion model, advanced numerical simulation techniques, and optimal control of randomness. Based on our findings, we propose Purify++, a new diffusion purification algorithm that is now the state-of-the-art purification method against several adversarial attacks. Our work presents a systematic exploration of the limits of diffusion purification methods.

Purify++: Improving Diffusion-Purification with Advanced Diffusion Models and Control of Randomness

TL;DR

Purify++ addresses adversarial robustness by enhancing diffusion purification through three targeted improvements: a stronger diffusion model (EDM/VE-based forward diffusion), efficient yet effective purification simulation (favoring stable, low-NFE Heun-style updates), and an optimal randomness control that mixes ODE and Langevin dynamics. It demonstrates that these components yield state-of-the-art robustness on CIFAR-10 and MNIST across black-box, gray-box, and adaptive attacks, while maintaining solid standard accuracy. The work provides both theoretical and empirical insights into diffusion-purification design, including how forward diffusion choice and stochasticity influence purification effectiveness. Practically, Purify++ offers a more reliable, compute-budget-friendly purification paradigm that can be deployed without retraining classifiers.

Abstract

Adversarial attacks can mislead neural network classifiers. The defense against adversarial attacks is important for AI safety. Adversarial purification is a family of approaches that defend adversarial attacks with suitable pre-processing. Diffusion models have been shown to be effective for adversarial purification. Despite their success, many aspects of diffusion purification still remain unexplored. In this paper, we investigate and improve upon three limiting designs of diffusion purification: the use of an improved diffusion model, advanced numerical simulation techniques, and optimal control of randomness. Based on our findings, we propose Purify++, a new diffusion purification algorithm that is now the state-of-the-art purification method against several adversarial attacks. Our work presents a systematic exploration of the limits of diffusion purification methods.
Paper Structure (41 sections, 4 theorems, 29 equations, 4 figures, 13 tables)

This paper contains 41 sections, 4 theorems, 29 equations, 4 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Backgrounds
  3. Adversarial Attacks and Defense
  4. Diffusion Purification
  5. Purify++
  6. Improved Forward Diffusion
  7. Variance Preserving Forward Diffusion
  8. Better Diffusion for Purification
  9. Efficient Purification Simulation
  10. Optimal Control of Randomness
  11. Experiments
  12. Models
  13. Adversarial Attacks and Evaluation Metrics
  14. Defending against Black-Box Attacks
  15. Defending against Gray-Box Attacks
  16. ...and 26 more sections

Key Result

Theorem 3.3

For VP diffusion with linear $\beta$ schedule equation equ:vp_forward and VE diffusion with log-linear $\sigma$ schedule equation equ:ve_forward respectively. The interaction time of VP diffusion is second-order $t^{(VP)}_* \sim o(h^2)$. The interaction time of $t^{(VE)}_* \sim o(h)$.

Figures (4)

  • Figure 1: Illustration of Purify++. Purify++ improves previous diffusion purification methods in three aspects: (a) improved diffusion model; (b) efficient simulation of purification SDE; (c) optimal control of randomness.
  • Figure 2: Comparison of VE and VP diffusion-based purification test accuracy.(Left) Standard Accuracy; (Right) Robust Accuracy. The green line represents diffusion purification with VE, the blue line the VP purification. The VE diffusion's optimal time is larger than VP purification which is the same as our proposed Theorem \ref{['thm:inter_time']} pointed out.
  • Figure 3: Illustration of Purify++ on MNIST for defending against Classifier-PGD attack under $\ell_\infty(\epsilon=0.25)$ threat model.
  • Figure 4: Purified Adversarial Examples on the CIFAR10 Dataset.

Theorems & Definitions (9)

  • Definition 3.1: Univariate Gaussian Interaction
  • Remark 3.2
  • Theorem 3.3: Order of Interaction Time
  • Remark 3.4
  • Theorem 3.5
  • Theorem A.1: Order of Interaction Time
  • proof
  • Theorem A.2
  • proof