Towards the first adversarially robust neural network model on MNIST

TL;DR

The paper tackles the problem of adversarial robustness on MNIST, showing that the prevailing Madry et al. defense overfits to $L_\infty$ perturbations and yields unrecognizable adversaries. It introduces Analysis by Synthesis (ABS), a Bayesian classifier that learns class-conditional distributions $p(\mathbf{x}|y)$ via per-class VAEs and uses gradient-descent in latent space to compute $\ell^{*}_y(\mathbf{x})$, then combines these into $p(y|\mathbf{x})$ with a scaled softmax and a nonzero offset to avoid overconfident rejections on out-of-distribution inputs. The authors derive instance-specific robustness bounds and conduct a comprehensive attack suite—including Latent Descent, decision-based, transfer-based, gradient-based, and score-based attacks—across $L_2$, $L_\infty$, and $L_0$ norms. Their results indicate state-of-the-art robustness on MNIST for multiple norms and reveal that many adversarial examples remain semantically meaningful to humans, while distal adversarials outside the data distribution are not confidently classified by ABS. Overall, the work argues that MNIST is not solved for adversarial robustness and demonstrates the potential of analysis-by-synthesis with class-conditional generative modeling to improve alignment between machine and human perception, while acknowledging evaluation limitations and outlining paths for scalability and independent verification.

Abstract

Despite much effort, deep neural networks remain highly susceptible to tiny input perturbations and even for MNIST, one of the most common toy datasets in computer vision, no neural network model exists for which adversarial perturbations are large and make semantic sense to humans. We show that even the widely recognized and by far most successful defense by Madry et al. (1) overfits on the L-infinity metric (it's highly susceptible to L2 and L0 perturbations), (2) classifies unrecognizable images with high certainty, (3) performs not much better than simple input binarization and (4) features adversarial perturbations that make little sense to humans. These results suggest that MNIST is far from being solved in terms of adversarial robustness. We present a novel robust classification model that performs analysis by synthesis using learned class-conditional data distributions. We derive bounds on the robustness and go to great length to empirically evaluate our model using maximally effective adversarial attacks by (a) applying decision-based, score-based, gradient-based and transfer-based attacks for several different Lp norms, (b) by designing a new attack that exploits the structure of our defended model and (c) by devising a novel decision-based attack that seeks to minimize the number of perturbed pixels (L0). The results suggest that our approach yields state-of-the-art robustness on MNIST against L0, L2 and L-infinity perturbations and we demonstrate that most adversarial examples are strongly perturbed towards the perceptual boundary between the original and the adversarial class.

Figures (8)

• Figure 1: Overview over model architecture. In a nutshell: I) for each sample ${\bf x}$ we compute a lower bound on the log-likelihood (ELBO) under each class using gradient descent in the latent space. II) A class-dependent scalar weighting of the class-conditional ELBOs forms the final class prediction.
• Figure 2: Accuracy-distortion plots for each distance metric and all models. In (b) we see that a threshold at $0.3$ favors Madry et al. while a threshold of $0.35$ would have favored the Binary ABS.
• Figure 3: Adversarial examples for the ABS models are perceptually meaningful: For each sample (randomly chosen from each class) we show the minimally perturbed $L_2$ adversarial found by any attack. Our ABS models have clearly visible and often semantically meaningful adversarials. Madry et al. requires perturbations that are clearly visible, but their semantics are less clear.
• Figure 4: Images of ones classified with a probability above 90%.
• Figure 5: $L_0$ error quantiles: We always choose the minimally perturbed $L_0$ adversarial found by any attack for each model. For an unbiased selection, we then randomly sample images within four error quantiles ($0-25\%, \: 25-50\%, \: 50-75\%, \: \text{and} \: 75-100\%)$. Where $100\%$ corresponds to the maximal (over samples) minimum (over attacks) perturbation found for each model.