Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PurSAMERE: Reliable Adversarial Purification via Sharpness-Aware Minimization of Expected Reconstruction Error

Vinh Hoang, Sebastian Krumscheid, Holger Rauhut, Raúl Tempone

TL;DR

PurSAMERE addresses adversarial vulnerability by deterministic purification that moves adversarial inputs toward high-density regions of the data distribution. It combines score-based generative modeling with Tweedie’s MMSE framework and a sharpness-aware minimization of the expected reconstruction error to produce purified samples that reside near local density maxima, even in the presence of noise. The method yields strong adversarial robustness under strong deterministic white-box attacks while preserving high clean accuracy, outperforming existing stochastic purification strategies. This approach offers reliable robustness guarantees and practical efficiency by avoiding reliance on stochastic transformation during evaluation.

Abstract

We propose a novel deterministic purification method to improve adversarial robustness by mapping a potentially adversarial sample toward a nearby sample that lies close to a mode of the data distribution, where classifiers are more reliable. We design the method to be deterministic to ensure reliable test accuracy and to prevent the degradation of effective robustness observed in stochastic purification approaches when the adversary has full knowledge of the system and its randomness. We employ a score model trained by minimizing the expected reconstruction error of noise-corrupted data, thereby learning the structural characteristics of the input data distribution. Given a potentially adversarial input, the method searches within its local neighborhood for a purified sample that minimizes the expected reconstruction error under noise corruption and then feeds this purified sample to the classifier. During purification, sharpness-aware minimization is used to guide the purified samples toward flat regions of the expected reconstruction error landscape, thereby enhancing robustness. We further show that, as the noise level decreases, minimizing the expected reconstruction error biases the purified sample toward local maximizers of the Gaussian-smoothed density; under additional local assumptions on the score model, we prove recovery of a local maximizer in the small-noise limit. Experimental results demonstrate significant gains in adversarial robustness over state-of-the-art methods under strong deterministic white-box attacks.

PurSAMERE: Reliable Adversarial Purification via Sharpness-Aware Minimization of Expected Reconstruction Error

TL;DR

PurSAMERE addresses adversarial vulnerability by deterministic purification that moves adversarial inputs toward high-density regions of the data distribution. It combines score-based generative modeling with Tweedie’s MMSE framework and a sharpness-aware minimization of the expected reconstruction error to produce purified samples that reside near local density maxima, even in the presence of noise. The method yields strong adversarial robustness under strong deterministic white-box attacks while preserving high clean accuracy, outperforming existing stochastic purification strategies. This approach offers reliable robustness guarantees and practical efficiency by avoiding reliance on stochastic transformation during evaluation.

Abstract

We propose a novel deterministic purification method to improve adversarial robustness by mapping a potentially adversarial sample toward a nearby sample that lies close to a mode of the data distribution, where classifiers are more reliable. We design the method to be deterministic to ensure reliable test accuracy and to prevent the degradation of effective robustness observed in stochastic purification approaches when the adversary has full knowledge of the system and its randomness. We employ a score model trained by minimizing the expected reconstruction error of noise-corrupted data, thereby learning the structural characteristics of the input data distribution. Given a potentially adversarial input, the method searches within its local neighborhood for a purified sample that minimizes the expected reconstruction error under noise corruption and then feeds this purified sample to the classifier. During purification, sharpness-aware minimization is used to guide the purified samples toward flat regions of the expected reconstruction error landscape, thereby enhancing robustness. We further show that, as the noise level decreases, minimizing the expected reconstruction error biases the purified sample toward local maximizers of the Gaussian-smoothed density; under additional local assumptions on the score model, we prove recovery of a local maximizer in the small-noise limit. Experimental results demonstrate significant gains in adversarial robustness over state-of-the-art methods under strong deterministic white-box attacks.
Paper Structure (20 sections, 6 theorems, 60 equations, 1 figure, 3 tables, 1 algorithm)

This paper contains 20 sections, 6 theorems, 60 equations, 1 figure, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Generative models via conditional expectation
  4. Adversarial purification
  5. Adversarial attack
  6. Method
  7. Learning the data distribution via score-based generative models
  8. Purification via sharpness-aware minimization of the expected reconstruction error (PurSAMERE)
  9. Relation between purified samples and the local maxima of the data density
  10. Algorithm
  11. Data augmentation for training the classifier
  12. Experiments
  13. Numerical results on CIFAR-10 dataset
  14. Conclusion
  15. Acknowledgments
  16. ...and 5 more sections

Key Result

Proposition 1

Let $s:\mathbb{R}^d\times[0,\sigma_{\max}]\to\mathbb{R}^d$ and let $\varXi\sim\mathcal{N}(0,I_d)$. Assume that for every $x\in\mathbb{R}^d$ and $\xi\in \mathbb{R}^d$, the map $\sigma \mapsto s(x+\sigma\xi,\sigma)$ is twice continuously differentiable on $[0,\sigma_{\max}]$. Define Fix a compact set $K \subset \mathbb{R}^d$. Assume there exist constants $C>0$, and $m\ge 0$, such that for all $x\in

Figures (1)

  • Figure 1: The expected reconstruction error $R(x; \sigma)$ of a truncated Gaussian mixture model in 1D, $p_X(x) \propto 0.5 \; \mathcal{N}(x; 0.25, 0.1) + 0.5 \; \mathcal{N}(x; 0.75, 0.1)$. The expected reconstruction error $R(x; \sigma)$ with $\sigma=0.1$ (right) attains local minima approximately at local maxima of the density $\log p_X$ (left).

Theorems & Definitions (11)

  • Proposition 1: Expected Reconstruction Error Expansion
  • Theorem 1: Local Recovery of Density Maximizer (Informal)
  • Lemma 1
  • proof
  • proof : Proof of Proposition \ref{['thm:ere_expansion']}
  • Theorem 2: Local Recovery of Density Maximizer (Formal)
  • Lemma 2
  • Lemma 3
  • proof : Proof of Theorem \ref{['thm:local_max_density']}
  • proof : Proof of Lemma \ref{['lem:integral_bound_outside_domain']}
  • ...and 1 more