Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Computing Exact Shapley Values in Polynomial Time for Product-Kernel Methods

Majid Mohammadi, Siu Lun Chau, Krikamol Muandet

TL;DR

The paper addresses the interpretability of kernel-based models by enabling exact, polynomial-time Shapley attributions for product-kernel methods through a novel functional baseline value function. It derives a recursive Shapley computation using elementary symmetric polynomials, achieving scalable, exact explanations. The framework extends to kernel-based statistical discrepancies, providing MMD and HSIC attributions. Empirical results show substantial efficiency gains over sampling-based methods and robust attribution performance across predictive and distributional tasks.

Abstract

Kernel methods are widely used in machine learning due to their flexibility and expressiveness. However, their black-box nature poses significant challenges to interpretability, limiting their adoption in high-stakes applications. Shapley value-based feature attribution techniques, such as SHAP and kernel method-specific adaptation like RKHS-SHAP, offer a promising path toward explainability. Yet, computing exact Shapley values is generally intractable, leading existing methods to rely on approximations and thereby incur unavoidable error. In this work, we introduce PKeX-Shapley, a novel algorithm that utilizes the multiplicative structure of product kernels to enable the exact computation of Shapley values in polynomial time. The core of our approach is a new value function, the functional baseline value function, specifically designed for product-kernel models. This value function removes the influence of a feature subset by setting its functional component to the least informative state. Crucially, it allows a recursive thus efficient computation of Shapley values in polynomial time. As an important additional contribution, we show that our framework extends beyond predictive modeling to statistical inference. In particular, it generalizes to popular kernel-based discrepancy measures such as the Maximum Mean Discrepancy (MMD) and the Hilbert-Schmidt Independence Criterion (HSIC), thereby providing new tools for interpretable statistical inference.

Computing Exact Shapley Values in Polynomial Time for Product-Kernel Methods

TL;DR

The paper addresses the interpretability of kernel-based models by enabling exact, polynomial-time Shapley attributions for product-kernel methods through a novel functional baseline value function. It derives a recursive Shapley computation using elementary symmetric polynomials, achieving scalable, exact explanations. The framework extends to kernel-based statistical discrepancies, providing MMD and HSIC attributions. Empirical results show substantial efficiency gains over sampling-based methods and robust attribution performance across predictive and distributional tasks.

Abstract

Kernel methods are widely used in machine learning due to their flexibility and expressiveness. However, their black-box nature poses significant challenges to interpretability, limiting their adoption in high-stakes applications. Shapley value-based feature attribution techniques, such as SHAP and kernel method-specific adaptation like RKHS-SHAP, offer a promising path toward explainability. Yet, computing exact Shapley values is generally intractable, leading existing methods to rely on approximations and thereby incur unavoidable error. In this work, we introduce PKeX-Shapley, a novel algorithm that utilizes the multiplicative structure of product kernels to enable the exact computation of Shapley values in polynomial time. The core of our approach is a new value function, the functional baseline value function, specifically designed for product-kernel models. This value function removes the influence of a feature subset by setting its functional component to the least informative state. Crucially, it allows a recursive thus efficient computation of Shapley values in polynomial time. As an important additional contribution, we show that our framework extends beyond predictive modeling to statistical inference. In particular, it generalizes to popular kernel-based discrepancy measures such as the Maximum Mean Discrepancy (MMD) and the Hilbert-Schmidt Independence Criterion (HSIC), thereby providing new tools for interpretable statistical inference.

Paper Structure

This paper contains 44 sections, 9 theorems, 65 equations, 8 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background on Shapley values
  3. Exact Shapley value for product-kernel learning methods
  4. A new functional baseline value function for product-kernel methods
  5. A recursive formulation for computing Shapley values
  6. Explaining kernel-based statistical discrepancies
  7. Distributing the discrepancy: Explaining the MMD
  8. Distributing the dependence: Explaining the HSIC
  9. Experiments
  10. Effectiveness of recursion and value function in Local Explanations
  11. Explaining distribution discrepancy using MMD with PKeX-Shapley
  12. Explaining HSIC with PKeX-Shapley: Feature selection case study
  13. Product kernels in kernel methods
  14. Radial Basis Function (RBF) Kernels as Product Kernels
  15. Automatic Relevance Determination (ARD) in Gaussian Processes
  16. ...and 29 more sections

Key Result

Proposition 2

$\nu_{\pmb x}(\mathcal{S})$ is the orthogonal projection of $f$ to $\mathbb{H}_{\mathcal{S}} := \{g \in \mathcal{H}: g(\pmb x) = g(\pmb x') \text{ when } \pmb x_{\mathcal{S}} = \pmb x'_{\mathcal{S}}\}$, the subspace of functions in $\mathcal{H}$ depending only on features in $\mathcal{S}$.

Figures (8)

  • Figure 1: Execution time of brute-force and our PKeX-Shapley with a 300s budget (see details in Appendix \ref{['apx:exp computation']}).
  • Figure 2: Relative estimation error of regression-based Shapley values versus the exact recursive values, shown across coalition sample sizes for feature dimensions $d=10,20,30,50$.
  • Figure 3: Recovery rate of true active features by each method on the three synthetic tasks.
  • Figure 4: Per‐instance explanation time (mean $\pm$ standard deviation) for each method with 500 and 1000 coalition samples.
  • Figure 5: KDE plots of Shapley values for the synthetic dataset. Variables $1$–$10$ reduce MMD via negative contributions, while $11$–$20$ increase it via positive contributions.
  • ...and 3 more figures

Theorems & Definitions (13)

  • Definition 1
  • Proposition 2
  • Theorem 3
  • Lemma 4
  • Definition 5
  • Proposition 6
  • Definition 7
  • Proposition 8
  • Theorem 9: Orthogonal projection onto the $\mathcal{S}$-only subspace
  • Proof 1
  • ...and 3 more