An Odd Estimator for Shapley Values

TL;DR

The paper addresses the challenge of efficiently estimating Shapley values in high-dimensional settings where exact computation is intractable. It introduces OddSHAP, a two-stage estimator that isolates the odd component of the value function using a Fourier basis and sparse regression, achieving polynomial-time computation with a budgeted number of samples. The authors prove that Shapley values depend only on the odd component and show that paired sampling orthogonalizes the regression, providing a theoretical justification for this widely used heuristic. Empirically, OddSHAP achieves state-of-the-art estimation accuracy across deep learning and tabular value functions, with favorable runtime and interaction-sparsity behavior, indicating practical impact for reliable feature valuation and model explanations.

Abstract

The Shapley value is a ubiquitous framework for attribution in machine learning, encompassing feature importance, data valuation, and causal inference. However, its exact computation is generally intractable, necessitating efficient approximation methods. While the most effective and popular estimators leverage the paired sampling heuristic to reduce estimation error, the theoretical mechanism driving this improvement has remained opaque. In this work, we provide an elegant and fundamental justification for paired sampling: we prove that the Shapley value depends exclusively on the odd component of the set function, and that paired sampling orthogonalizes the regression objective to filter out the irrelevant even component. Leveraging this insight, we propose OddSHAP, a novel consistent estimator that performs polynomial regression solely on the odd subspace. By utilizing the Fourier basis to isolate this subspace and employing a proxy model to identify high-impact interactions, OddSHAP overcomes the combinatorial explosion of higher-order approximations. Through an extensive benchmark evaluation, we find that OddSHAP achieves state-of-the-art estimation accuracy.

Figures (11)

• Figure 1: OddSHAP balances expressive power with efficiency by only fitting the odd component of the value function.
• Figure 2: Approximation quality measured by MSE (median with interquartile range) for Shapley value estimators with paired sampling.
• Figure 3: Approximation quality measured by MSE (median with interquartile range) and total runtime in seconds for a simulated setting with $0.001$s per subset evaluation. Similar plots for other datasets appear in \ref{['appx_sec_further_results']}.
• Figure 4: MSE ratio between OddSHAP and LeverageSHAP as a function of the number of selected interactions in OddSHAP, computed under a fixed budget of 10,000 samples.
• Figure 5: Runtime measured in seconds (median and IQR band) for varying budgets ($m$). OddSHAP has similar runtime compared with RegressionMSR, but higher runtime than LeverageSHAP.

Theorems & Definitions (11)

• Theorem 2.1: Linear Regression charnes1988extremal
• Theorem 2.2: Polynomial Regression fumagalli2026polyshap
• proof
• Theorem 3.2: Even-Odd Separation via Paired Sampling
• Corollary 3.3
• Lemma 3.4: Unanimity and Fourier Equivalence
• Theorem 3.5: Fourier Regression
• proof : Proof of Theorem \ref{['thm:separation']}
• proof
• proof : Proof of Lemma \ref{['lemma:equivalence']}