Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients

TL;DR

This work tackles the dual problems of adversarial vulnerability and interpretability in deep neural networks. It introduces input gradient regularization (double backpropagation) as a differentiable training objective that penalizes sensitivity to input perturbations, and demonstrates that this approach improves robustness to transferred adversarial examples while yielding more interpretable gradient explanations. The study compares gradient regularization with distillation and adversarial training, showing that regularization can outperform or complement these defenses, and includes a human-subject study indicating that adversarial perturbations crafted against gradient-regularized models are more interpretable to people. Overall, the results suggest a strong link between robustness and interpretability, with gradient regularization offering a principled route to jointly improve both aspects and providing actionable insights for safer, more transparent DNN deployments.

Abstract

Deep neural networks have proven remarkably effective at solving many classification problems, but have been criticized recently for two major weaknesses: the reasons behind their predictions are uninterpretable, and the predictions themselves can often be fooled by small adversarial perturbations. These problems pose major obstacles for the adoption of neural networks in domains that require security or transparency. In this work, we evaluate the effectiveness of defenses that differentiably penalize the degree to which small changes in inputs can alter model predictions. Across multiple attacks, architectures, defenses, and datasets, we find that neural networks trained with this input gradient regularization exhibit robustness to transferred adversarial examples generated to fool all of the other models. We also find that adversarial examples generated to fool gradient-regularized models fool all other models equally well, and actually lead to more "legitimate," interpretable misclassifications as rated by people (which we confirm in a human subject experiment). Finally, we demonstrate that regularizing input gradients makes them more naturally interpretable as rationales for model predictions. We conclude by discussing this relationship between interpretability and robustness in deep neural networks.

Figures (11)

• Figure 1: Accuracy of all CNNs on FGSM examples generated to fool undefended models, defensively distilled, adversarially trained, and gradient regularized models (from left to right) on MNIST, SVHN, and notMNIST (from top to bottom). Gradient-regularized models are the most resistant to other models' adversarial examples at high $\epsilon$, while all models are fooled by gradient-regularized model examples. On MNIST and notMNIST, distilled model examples are usually identical to non-adversarial examples (due to gradient underflow), so they fail to fool any of the other models.
• Figure 2: Applying both gradient regularization and adversarial training ("both defenses") allows us to obtain maximal robustness to white-box and normal black-box attacks on SVHN (with a very slight label-leaking effect on the FGSM, perhaps due to the inclusion of the $\nabla_x H(y,\hat{y})$ term). However, no models are able to maintain robustness to black-box attacks using gradient regularization.
• Figure 3: CNN accuracy on $y_{+1}$ TGSM examples generated to fool the four models on three datasets (see Figure \ref{['fig:fgsm-comp']} for more explanation). Gradient-regularized models again exhibit robustness to other models' adversarial examples. Distilled model adversarial perturbations fool other models again since their input gradients no longer underflow.
• Figure 4: Distributions of (L2 norm) magnitudes of FGSM input gradients (top), TGSM input gradients (middle), and predicted log probabilities across all classes (bottom) for each defense. Note the logarithmic scales. Gradient-regularized models tend to assign non-predicted classes higher probabilities, and the L2 norms of the input gradients of their FGSM and TGSM loss function terms have similar orders of magnitude. Distilled models (evaluated at $T=0$) assign extremely small probabilities to all but the predicted class, and their TGSM gradients explode while their FGSM gradients vanish (we set a minimum value of $10^{-20}$ to prevent underflow). Normal and adversarially trained models lie somewhere in the middle.
• Figure 5: Results of applying the JSMA to MNIST 0 and 1 images with maximum distortion parameter $\gamma=0.25$ for a distilled model (top) and a gradient-regularized model (bottom). Examples in each row start out as the highlighted digit but are modified until the model predicts the digit corresponding to their column or the maximum distortion is reached.