Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Path Choice Matters for Clear Attribution in Path Methods

Borui Zhang, Wenzhao Zheng, Jie Zhou, Jiwen Lu

TL;DR

Path-based attributions in DNNs suffer from ambiguity due to varying path choices. The authors introduce the Concentration Principle to focus attributions on indispensable features and propose SAMP, a model-agnostic interpreter that searches near-optimal manipulation paths under an IC–MS framework. The approach yields sparser, more localized saliency maps and consistently improves Deletion/Insertion metrics across MNIST, CIFAR-10, and ImageNet, with ablations validating the contributions. This work enhances rigor and clarity in post-hoc explanations and points to scalable, domain-agnostic applications in trustworthy AI.

Abstract

Rigorousness and clarity are both essential for interpretations of DNNs to engender human trust. Path methods are commonly employed to generate rigorous attributions that satisfy three axioms. However, the meaning of attributions remains ambiguous due to distinct path choices. To address the ambiguity, we introduce \textbf{Concentration Principle}, which centrally allocates high attributions to indispensable features, thereby endowing aesthetic and sparsity. We then present \textbf{SAMP}, a model-agnostic interpreter, which efficiently searches the near-optimal path from a pre-defined set of manipulation paths. Moreover, we propose the infinitesimal constraint (IC) and momentum strategy (MS) to improve the rigorousness and optimality. Visualizations show that SAMP can precisely reveal DNNs by pinpointing salient image pixels. We also perform quantitative experiments and observe that our method significantly outperforms the counterparts. Code: https://github.com/zbr17/SAMP.

Path Choice Matters for Clear Attribution in Path Methods

TL;DR

Path-based attributions in DNNs suffer from ambiguity due to varying path choices. The authors introduce the Concentration Principle to focus attributions on indispensable features and propose SAMP, a model-agnostic interpreter that searches near-optimal manipulation paths under an IC–MS framework. The approach yields sparser, more localized saliency maps and consistently improves Deletion/Insertion metrics across MNIST, CIFAR-10, and ImageNet, with ablations validating the contributions. This work enhances rigor and clarity in post-hoc explanations and points to scalable, domain-agnostic applications in trustworthy AI.

Abstract

Rigorousness and clarity are both essential for interpretations of DNNs to engender human trust. Path methods are commonly employed to generate rigorous attributions that satisfy three axioms. However, the meaning of attributions remains ambiguous due to distinct path choices. To address the ambiguity, we introduce \textbf{Concentration Principle}, which centrally allocates high attributions to indispensable features, thereby endowing aesthetic and sparsity. We then present \textbf{SAMP}, a model-agnostic interpreter, which efficiently searches the near-optimal path from a pre-defined set of manipulation paths. Moreover, we propose the infinitesimal constraint (IC) and momentum strategy (MS) to improve the rigorousness and optimality. Visualizations show that SAMP can precisely reveal DNNs by pinpointing salient image pixels. We also perform quantitative experiments and observe that our method significantly outperforms the counterparts. Code: https://github.com/zbr17/SAMP.
Paper Structure (53 sections, 2 theorems, 20 equations, 17 figures, 8 tables, 1 algorithm)

This paper contains 53 sections, 2 theorems, 20 equations, 17 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Perturbation Methods.
  4. Back-propagation Methods.
  5. Method
  6. Preliminary: Path Method
  7. Criterion and Candidate Set
  8. Salient Manipulation Path
  9. Towards Rigorousness and Optimality
  10. Infinitesimal Constraint (IC).
  11. Momentum Strategy (MS).
  12. Experiment
  13. Experimental Setting
  14. Datasets and Models.
  15. Metrics.
  16. ...and 38 more sections

Key Result

Proposition 1

By Brownian motion assumption, the conditional joint distribution $P(\tilde{{\bm{a}}}|C) = P({a}_1,\cdots,{a}_{d-1} | u_d = C)$ is a multivariate Gaussian distribution as: where $\Sigma = \sigma ({\bm{I}} - \frac{{\bm{J}}}{d}) \in \mathbb{R}^{(d-1)\times (d-1)}$ and ${\bm{J}}$ is all-one matrix.

Figures (17)

  • Figure 1: we propose SAMP to eliminate ambiguity of attributions by path methods, which can precisely pinpoint important pixels and produce clear saliency maps. Quantitative results show a consistent improvement in Deletion/Insertion metrics.
  • Figure 2: (a) Concentration Principle prioritizes attributions (green point $A$) with large distance from mean point $P$. (b) SAMP chooses the directions with max gradient projection (colored in red), and attributions allocated along this path mainly concentrate on salient pixels.
  • Figure 3: Verification of Concentration Principle. (a) Visualizations of intermediate points and corresponding attributions along the path solved by SAMP. (b) The output score curve from the baseline point to the target image.
  • Figure 4: Visualizations on MNIST, CIFAR-10, and ImageNet compared with other methods.
  • Figure 5: Sensitivity-N check for IC.
  • ...and 12 more figures

Theorems & Definitions (9)

  • Definition 1: Concentration Principle
  • Remark
  • Definition 2: Manipulation Path
  • Remark
  • Proposition 1
  • Remark
  • Proposition 2
  • Remark
  • proof