Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Additivity: Sparse Isotonic Shapley Regression toward Nonlinear Explainability

Jialai She

TL;DR

This work tackles the limitations of standard Shapley attributions when payoffs are non-additive due to non-Gaussianity, feature dependence, or domain-specific losses, and when high-dimensional sparsity is required. It introduces Sparse Isotonic Shapley Regression (SISR), which learns a data-driven monotone transformation to restore additivity in the transformed domain while enforcing an $\ell_0$ sparsity constraint; the optimization combines Pool-Adjacent-Violators isotonic regression with normalized hard-thresholding and comes with global convergence guarantees. The authors prove that the transformation can be accurately recovered in many settings and demonstrate through extensive experiments on regression, logistic regression, and tree ensembles that SISR stabilizes attributions across payoff schemes, correctly filters irrelevant features, and avoids rank and sign distortions seen with conventional Shapley methods. They also discuss extensions to nonlinear Shapley GLMs and emphasize that nonlinear payoff distortions can arise from feature Irrelevance and dependencies, making nonlinear explainability essential for robust interpretation in practice.

Abstract

Shapley values, a gold standard for feature attribution in Explainable AI, face two primary challenges. First, the canonical Shapley framework assumes that the worth function is additive, yet real-world payoff constructions--driven by non-Gaussian distributions, heavy tails, feature dependence, or domain-specific loss scales--often violate this assumption, leading to distorted attributions. Secondly, achieving sparse explanations in high dimensions by computing dense Shapley values and then applying ad hoc thresholding is prohibitively costly and risks inconsistency. We introduce Sparse Isotonic Shapley Regression (SISR), a unified nonlinear explanation framework. SISR simultaneously learns a monotonic transformation to restore additivity--obviating the need for a closed-form specification--and enforces an L0 sparsity constraint on the Shapley vector, enhancing computational efficiency in large feature spaces. Its optimization algorithm leverages Pool-Adjacent-Violators for efficient isotonic regression and normalized hard-thresholding for support selection, yielding implementation ease and global convergence guarantees. Analysis shows that SISR recovers the true transformation in a wide range of scenarios and achieves strong support recovery even in high noise. Moreover, we are the first to demonstrate that irrelevant features and inter-feature dependencies can induce a true payoff transformation that deviates substantially from linearity. Experiments in regression, logistic regression, and tree ensembles demonstrate that SISR stabilizes attributions across payoff schemes, correctly filters irrelevant features while standard Shapley values suffer severe rank and sign distortions. By unifying nonlinear transformation estimation with sparsity pursuit, SISR advances the frontier of nonlinear explainability, providing a theoretically grounded and practical attribution framework.

Beyond Additivity: Sparse Isotonic Shapley Regression toward Nonlinear Explainability

TL;DR

This work tackles the limitations of standard Shapley attributions when payoffs are non-additive due to non-Gaussianity, feature dependence, or domain-specific losses, and when high-dimensional sparsity is required. It introduces Sparse Isotonic Shapley Regression (SISR), which learns a data-driven monotone transformation to restore additivity in the transformed domain while enforcing an sparsity constraint; the optimization combines Pool-Adjacent-Violators isotonic regression with normalized hard-thresholding and comes with global convergence guarantees. The authors prove that the transformation can be accurately recovered in many settings and demonstrate through extensive experiments on regression, logistic regression, and tree ensembles that SISR stabilizes attributions across payoff schemes, correctly filters irrelevant features, and avoids rank and sign distortions seen with conventional Shapley methods. They also discuss extensions to nonlinear Shapley GLMs and emphasize that nonlinear payoff distortions can arise from feature Irrelevance and dependencies, making nonlinear explainability essential for robust interpretation in practice.

Abstract

Shapley values, a gold standard for feature attribution in Explainable AI, face two primary challenges. First, the canonical Shapley framework assumes that the worth function is additive, yet real-world payoff constructions--driven by non-Gaussian distributions, heavy tails, feature dependence, or domain-specific loss scales--often violate this assumption, leading to distorted attributions. Secondly, achieving sparse explanations in high dimensions by computing dense Shapley values and then applying ad hoc thresholding is prohibitively costly and risks inconsistency. We introduce Sparse Isotonic Shapley Regression (SISR), a unified nonlinear explanation framework. SISR simultaneously learns a monotonic transformation to restore additivity--obviating the need for a closed-form specification--and enforces an L0 sparsity constraint on the Shapley vector, enhancing computational efficiency in large feature spaces. Its optimization algorithm leverages Pool-Adjacent-Violators for efficient isotonic regression and normalized hard-thresholding for support selection, yielding implementation ease and global convergence guarantees. Analysis shows that SISR recovers the true transformation in a wide range of scenarios and achieves strong support recovery even in high noise. Moreover, we are the first to demonstrate that irrelevant features and inter-feature dependencies can induce a true payoff transformation that deviates substantially from linearity. Experiments in regression, logistic regression, and tree ensembles demonstrate that SISR stabilizes attributions across payoff schemes, correctly filters irrelevant features while standard Shapley values suffer severe rank and sign distortions. By unifying nonlinear transformation estimation with sparsity pursuit, SISR advances the frontier of nonlinear explainability, providing a theoretically grounded and practical attribution framework.

Paper Structure

This paper contains 13 sections, 2 theorems, 40 equations, 9 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction and Motivation
  2. Proposed Method
  3. Optimization Algorithm
  4. Data-Driven Insights
  5. Domain Adaptation
  6. Sparsity Recovery
  7. $R^2$-Payoffs in Regression/Logistic Regression
  8. Prostate Cancer
  9. Boston Housing
  10. Bank Credit
  11. Diabetes
  12. Discussion and Extensions
  13. Conclusion

Key Result

Theorem 1

Let $\mathcal{H}(\cdot; s)$ denote the hard-thresholding operator associated with cardinality $s$, defined as follows: for a vector $y \in \mathbb{R}^p$, $\mathcal{H}(y; s) = z$ where $z_i = y_i$ if $|y_i|$ is among the $s$ largest entries of $|y_1|, \ldots, |y_p|$, and $z_i = 0$ otherwise, and the the vector obtained by normalized hard-thresholding, is a global optimizer.

Figures (9)

  • Figure 1: Estimated monotonic transformation $\hat{T}(\nu)$ (in blue) versus the true transformation $T^*$ (in red), for $p=10$ under 6 different functional forms for $T^*$: the fifth root, square root, normal distribution, tangent, exponential, and logarithmic transformations.
  • Figure 2: Estimated monotonic transformation $\hat{T}(\nu)$ (left) and comparison between $\hat{\gamma}$ vs $\hat{T}(\beta^*)$ (right, showing an almost perfect correlation of 1.00) for $p=20$ under a winner-takes-all setting.
  • Figure 3: Computational time versus sparsity level.
  • Figure 4: Estimated monotonic transformation $\hat{T}(\nu)$ using regression-based $R^2$ and logistic regression-based pseudo-$R^2$ as the coalition worth function, for $p=8, 15$.
  • Figure 5: Estimated monotonic transformation $\hat{T}(\nu)$ across varying sparsity levels ($s=15, 8, 2$, top to down) and feature correlation strengths ($\theta = 0.9, 0.5, 0$, left to right). The RIC criterion identifies $s=6$ as optimal.
  • ...and 4 more figures

Theorems & Definitions (5)

  • Remark 1: Univariate $T$-Mappings for Multivariate Structure
  • Theorem 1
  • proof
  • Theorem 2
  • proof