Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SIM-Shapley: A Stable and Computationally Efficient Approach to Shapley Value Approximation

Wangxuan Fan, Siqi Li, Doudou Zhou, Yohei Okada, Chuan Hong, Molei Liu, Nan Liu

TL;DR

The paper tackles the high computational cost of Shapley-value based feature attribution in high-dimensional models. It introduces SIM-Shapley, a stochastic iterative momentum method with an ℓ2 penalty that reframes KernelSHAP estimation as a mini-batch stochastic optimization, achieving linear $Q$-convergence with rate $\rho = t \cdot \frac{\lambda}{\alpha+\lambda}$. Two stability enhancements—negative-sampling rejection and initialization bias correction—further improve convergence without harming attribution quality. Empirically, SIM-Shapley reaches up to 85% speedups while maintaining accuracy across classification, regression, and image tasks (including MNIST and MIMIC-IV-ED), and scales to global explanations with strong agreement to SAGE. The approach is model- and imputer-agnostic, compatible with KernelSHAP enhancements, and extends to broader sample-average-approximation objectives.

Abstract

Explainable artificial intelligence (XAI) is essential for trustworthy machine learning (ML), particularly in high-stakes domains such as healthcare and finance. Shapley value (SV) methods provide a principled framework for feature attribution in complex models but incur high computational costs, limiting their scalability in high-dimensional settings. We propose Stochastic Iterative Momentum for Shapley Value Approximation (SIM-Shapley), a stable and efficient SV approximation method inspired by stochastic optimization. We analyze variance theoretically, prove linear $Q$-convergence, and demonstrate improved empirical stability and low bias in practice on real-world datasets. In our numerical experiments, SIM-Shapley reduces computation time by up to 85% relative to state-of-the-art baselines while maintaining comparable feature attribution quality. Beyond feature attribution, our stochastic mini-batch iterative framework extends naturally to a broader class of sample average approximation problems, offering a new avenue for improving computational efficiency with stability guarantees. Code is publicly available at https://github.com/nliulab/SIM-Shapley.

SIM-Shapley: A Stable and Computationally Efficient Approach to Shapley Value Approximation

TL;DR

The paper tackles the high computational cost of Shapley-value based feature attribution in high-dimensional models. It introduces SIM-Shapley, a stochastic iterative momentum method with an ℓ2 penalty that reframes KernelSHAP estimation as a mini-batch stochastic optimization, achieving linear -convergence with rate . Two stability enhancements—negative-sampling rejection and initialization bias correction—further improve convergence without harming attribution quality. Empirically, SIM-Shapley reaches up to 85% speedups while maintaining accuracy across classification, regression, and image tasks (including MNIST and MIMIC-IV-ED), and scales to global explanations with strong agreement to SAGE. The approach is model- and imputer-agnostic, compatible with KernelSHAP enhancements, and extends to broader sample-average-approximation objectives.

Abstract

Explainable artificial intelligence (XAI) is essential for trustworthy machine learning (ML), particularly in high-stakes domains such as healthcare and finance. Shapley value (SV) methods provide a principled framework for feature attribution in complex models but incur high computational costs, limiting their scalability in high-dimensional settings. We propose Stochastic Iterative Momentum for Shapley Value Approximation (SIM-Shapley), a stable and efficient SV approximation method inspired by stochastic optimization. We analyze variance theoretically, prove linear -convergence, and demonstrate improved empirical stability and low bias in practice on real-world datasets. In our numerical experiments, SIM-Shapley reduces computation time by up to 85% relative to state-of-the-art baselines while maintaining comparable feature attribution quality. Beyond feature attribution, our stochastic mini-batch iterative framework extends naturally to a broader class of sample average approximation problems, offering a new avenue for improving computational efficiency with stability guarantees. Code is publicly available at https://github.com/nliulab/SIM-Shapley.
Paper Structure (37 sections, 2 theorems, 49 equations, 5 figures, 10 tables, 3 algorithms)

This paper contains 37 sections, 2 theorems, 49 equations, 5 figures, 10 tables, 3 algorithms.

Table of Contents

  1. Background
  2. Preliminaries
  3. Shapley Value as Weighted Least Squares
  4. Monte Carlo Estimation and Practical Challenges
  5. From Sample Average Approximation to Stochastic Iteration
  6. Proposed Method: SIM-Shapley
  7. Stochastic Iteration with Momentum
  8. Variance and Convergence
  9. Stability Improvement
  10. Preventing Negative Sampling
  11. Correcting Initialization Bias
  12. Experiments
  13. Baseline Methods and Experimental Setup
  14. Local Explanations
  15. Classification Tasks
  16. ...and 22 more sections

Key Result

Theorem 1

Let $\hat{\bm{\beta}}$ denote the estimator obtained from KernelSHAP in equation eq4 with $m$ samples of $z$, and let $\bm{\beta}^{(n)}$ be the SIM-Shapley iterates with fixed momentum $t \in (0,1)$. As $m \to \infty$, for fixed momentum $t \in (0,1)$ and any feature index $i$ and any iteration $n \

Figures (5)

  • Figure 1: Bias-regularization trade-off for SIM-Shapley on the Bank dataset. Bias represents the $\ell_2$ distance from ground-truth SV as $\lambda$ varies from 0 (no regularization) to 1.
  • Figure 1: Comparison of consistency across different Shapley value estimators. All axes represent estimated Shapley values. (a) Comparison between two SAGE estimates obtained via permutation and kernel regression. (b) Comparison between the SAGE estimate and the SIM-Shapley estimate. Experiments are conducted on the Bank dataset under identical settings. (c) Comparison between SIM-Shapley and its stable version. All SV estimates stem from Bike dataset with global explanation.
  • Figure 2: Boxplot of local explanation. Comparison of Consistency (Pearson's Correlation) and Running Time(s).
  • Figure 3: Boxplot of global explanation. Comparison of Consistency (Pearson's Correlation) and Running Time(s).
  • Figure 4: Comparison of convergence time(s). Compare global and local interpretability methods on standard machine learning datasets.

Theorems & Definitions (6)

  • Theorem 1: Variance Contraction
  • Theorem 2
  • Remark 1
  • proof
  • proof
  • proof