The Distributional Uncertainty of the SHAP score in Explainable Machine Learning

TL;DR

SHAP scores depend on an underlying entity population distribution, which is typically unknown in practice. This paper treats SHAP as a polynomial over an uncertainty region of product distributions, enabling the computation of SHAP intervals via maxima and minima over hyperrectangles. It proves that key problems for these region-based SHAP scores are NP-hard or NP-complete, even for binary decision trees, and confirms related hardness for ambiguity and dominance questions, while showing polynomial-time tractability for certain classifier families. An empirical study on the California Housing data demonstrates that feature rankings can vary with distributional assumptions and that shrinking the uncertainty region reduces this sensitivity. Overall, the work offers a principled, robust perspective on attribution under distributional uncertainty and lays groundwork for more reliable local explanations.

Abstract

Attribution scores reflect how important the feature values in an input entity are for the output of a machine learning model. One of the most popular attribution scores is the SHAP score, which is an instantiation of the general Shapley value used in coalition game theory. The definition of this score relies on a probability distribution on the entity population. Since the exact distribution is generally unknown, it needs to be assigned subjectively or be estimated from data, which may lead to misleading feature scores. In this paper, we propose a principled framework for reasoning on SHAP scores under unknown entity population distributions. In our framework, we consider an uncertainty region that contains the potential distributions, and the SHAP score of a feature becomes a function defined over this region. We study the basic problems of finding maxima and minima of this function, which allows us to determine tight ranges for the SHAP scores of all features. In particular, we pinpoint the complexity of these problems, and other related ones, showing them to be NP-complete. Finally, we present experiments on a real-world dataset, showing that our framework may contribute to a more robust feature scoring.

Figures (3)

• Figure 1: SHAP intervals for all features and a fixed entity, over two different sampling percentages. When $p=0.4\%$ it is clear that, according to the SHAP score, the features median_income and ocean_proximity are the two most relevant, but there is an uncertainty on which one of the two is the most important. When the sampling percentage increases to $p=6.4\%$ the SHAP intervals become disjoint, and we can be certain that ocean_proximity is the most relevant feature. Observe that the same kind of uncertainty exists regarding the third ranked feature.
• Figure 2: Evolution of the SHAP intervals for features median_income and ocean_proximity considering the same entity as in Figure \ref{['fig:shap_intervals']} and the different sampling percentages.
• Figure 3: Number of entities whose feature ranking may vary depending on the chosen distribution inside the uncertainty hyperrectangle, for each sampling percentage. The Top $k$ line indicates whether there was a change in the ranking of the top $k$ features, ignoring changes in the rest of the ranking. As expected, sensitivity of the ranking is more common when the sampling percentage is smaller.

Theorems & Definitions (23)

• Example 1
• Definition 1: SHAP score
• Example 2
• Example 3
• Definition 2: SHAP polynomial
• Proposition 1
• proof
• Proposition 2
• Theorem 3
• Corollary 4