Structured Gradient-based Interpretations via Norm-Regularized Adversarial Training

TL;DR

The paper tackles unstructured gradient-based explanations by introducing norm-regularized adversarial training as an in-processing mechanism to sculpt sparse, group-sparse, and human-aligned saliency maps without sacrificing fidelity to the simple gradient. A Fenchel conjugate–based duality links perturbation norms to gradient-map penalties, enabling a unified framework that includes elastic net and group-norm regularizations and a gaze-map harmonization strategy. Empirical results across synthetic data, ImageNette, and gaze-aligned bird datasets demonstrate improved sparsity, interpretability (DiffROAR), robustness to interpretation attacks, and stability of explanations. This approach offers a principled path to more trustworthy and human-aligned explanations while preserving the integrity of the original gradient interpretation.

Abstract

Gradient-based saliency maps have been widely used to explain the decisions of deep neural network classifiers. However, standard gradient-based interpretation maps, including the simple gradient and integrated gradient algorithms, often lack desired structures such as sparsity and connectedness in their application to real-world computer vision models. A frequently used approach to inducing sparsity structures into gradient-based saliency maps is to alter the simple gradient scheme using sparsification or norm-based regularization. A drawback with such post-processing methods is their frequently-observed significant loss in fidelity to the original simple gradient map. In this work, we propose to apply adversarial training as an in-processing scheme to train neural networks with structured simple gradient maps. We show a duality relation between the regularized norms of the adversarial perturbations and gradient-based maps, based on which we design adversarial training loss functions promoting sparsity and group-sparsity properties in simple gradient maps. We present several numerical results to show the influence of our proposed norm-based adversarial training methods on the standard gradient-based maps of standard neural network architectures on benchmark image datasets.

Figures (16)

• Figure 1: The original simple gradient map could be dense and noisy. Post-processing methods such as sparsification enhance the sparsity at the expense of lower fidelity to the original map. Our proposed in-processing strategy with adversarial training results in higher sparsity without losing fidelity to the simple-grad map. (We use the Gini index as a sparsity measure, and relative $\text{AOPC}_\text{MoRF}$ with respect to simple gradient as a fidelity score.)
• Figure 2: Qualitative results based on the synthesized dataset.
• Figure 3: Images optimized for maximizing class-logit activations.
• Figure 4: Qualitative comparison of saliency maps generated by networks with different adversarial training protocols (fast, iterative) and standard training with additional regularization on input gradients (penalty).
• Figure 5: Quantitative robustness and stability comparison. Left/Middle: Comparison of SSIM/top-k intersection of saliency maps before and after the attack. Right: Comparison of SSIM/top-k overlap of saliency maps generated by networks with different stochastic training.

Theorems & Definitions (5)

• Definition 1
• Proposition 1
• Proposition 2
• Proposition 3
• Proposition 4