Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Stability for Attribution-based Explanations

Chirag Agarwal, Nari Johnson, Martin Pawelczyk, Satyapriya Krishna, Eshika Saxena, Marinka Zitnik, Himabindu Lakkaraju

TL;DR

This paper addresses the instability of attribution-based explanations in high-stakes domains by introducing Relative Stability, a framework comprising RIS, RRS, and ROS that ties explanation changes to input perturbations, internal representations, and output behavior. The authors provide theoretical bounds linking these metrics and demonstrate, across three real-world datasets and seven explanation methods, that traditional input-focused stability is insufficient while representation- and output-aware metrics reveal more faithful stability patterns, with SmoothGrad often performing best. Together, these contributions offer a principled, model-aware approach to evaluating explanations, with significant implications for reliability and trust in AI systems.

Abstract

As attribution-based explanation methods are increasingly used to establish model trustworthiness in high-stakes situations, it is critical to ensure that these explanations are stable, e.g., robust to infinitesimal perturbations to an input. However, previous works have shown that state-of-the-art explanation methods generate unstable explanations. Here, we introduce metrics to quantify the stability of an explanation and show that several popular explanation methods are unstable. In particular, we propose new Relative Stability metrics that measure the change in output explanation with respect to change in input, model representation, or output of the underlying predictor. Finally, our experimental evaluation with three real-world datasets demonstrates interesting insights for seven explanation methods and different stability metrics.

Rethinking Stability for Attribution-based Explanations

TL;DR

This paper addresses the instability of attribution-based explanations in high-stakes domains by introducing Relative Stability, a framework comprising RIS, RRS, and ROS that ties explanation changes to input perturbations, internal representations, and output behavior. The authors provide theoretical bounds linking these metrics and demonstrate, across three real-world datasets and seven explanation methods, that traditional input-focused stability is insufficient while representation- and output-aware metrics reveal more faithful stability patterns, with SmoothGrad often performing best. Together, these contributions offer a principled, model-aware approach to evaluating explanations, with significant implications for reliability and trust in AI systems.

Abstract

As attribution-based explanation methods are increasingly used to establish model trustworthiness in high-stakes situations, it is critical to ensure that these explanations are stable, e.g., robust to infinitesimal perturbations to an input. However, previous works have shown that state-of-the-art explanation methods generate unstable explanations. Here, we introduce metrics to quantify the stability of an explanation and show that several popular explanation methods are unstable. In particular, we propose new Relative Stability metrics that measure the change in output explanation with respect to change in input, model representation, or output of the underlying predictor. Finally, our experimental evaluation with three real-world datasets demonstrates interesting insights for seven explanation methods and different stability metrics.
Paper Structure (12 sections, 12 equations, 5 figures, 1 table)

This paper contains 12 sections, 12 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Stability Analysis for Evaluating Explanations
  4. Notation and Preliminaries
  5. Existing Definition and Problems
  6. Proposed metric: Relative Stability
  7. Experiments
  8. Datasets and Experimental Setup
  9. Results
  10. Conclusion
  11. Theoretical Interpretation
  12. Implementation Details

Figures (5)

  • Figure 1: Decision boundaries and embeddings of a two-layer neural network predictor $f$ with 100 units trained on the https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_circles.html dataset. The heatmaps (left and middle column) shows the models' confidence for the positive-class (in blue), test set examples ${\mathbf{x}}$ (, ), and a set of perturbed samples ${\mathbf{x}}'$ (). While all perturbed samples $\mathbf{x}'$ are predicted to the same class as $\mathbf{x}'$, the embeddings (right column) for some $\mathbf{x}'$ are far from the embeddings of $\mathbf{x}'$ and similar to the embeddings of Class 0, highlighting the need of incorporating the model behavior using its internal embeddings (Equations \ref{['eq:rrs']},\ref{['eq:ros']}).
  • Figure 2: Theoretical upper bounds for the (log) relative input stability (RIS) computed using the right-hand-side of Equation \ref{['eq:proof_1']} across seven explanation methods for an ANN predictor trained on Adult dataset. Results show that RIS is upper bounded by the product of $L_{1}$ and RRS (relative representation stability), where $L_1$ is the Lipschitz constant between the input and hidden layer of the ANN model. Results for the Compas and German dataset are shown in Appendix \ref{['app:bounds_ann']}.
  • Figure 3: Empirically calculated log stability of relative stability variants (Equations \ref{['eq:ris']}-\ref{['eq:ros']}) across seven explanation methods. Results on the Adult (a), Compas (b), and German (c) dataset trained with ANN predictor show that SmoothGrad generates the most stable explanation across RRS and ROS variants. Results for all datasets trained on Logistic Regression models are shown in Appendix \ref{['fig:lr']}.
  • Figure 4: Empirically calculated log stability of all three relative stability variants (Equations \ref{['eq:ris']}-\ref{['eq:ros']}) across seven explanation methods. Results on the Adult dataset trained with Logistic Regression predictor show that SmoothGrad generates the most stable explanation across representation and output stability variants.
  • Figure 5: Theoretical upper bounds for the (log) relative input stability (RIS) computed using the right-hand-side of Equation \ref{['eq:proof_1']} across seven explanation methods for an ANN predictor trained on the Compas and German credit datasets. Results show that RIS is upper bounded by the product of $L_{1}$ and RRS (relative representation stability), where $L_1$ is the Lipschitz constant between the input and hidden layer of the ANN model.