How to safely discard features based on aggregate SHAP values

TL;DR

This paper interrogates the soundness of discarding features based on aggregate SHAP values and shows that zero or near-zero global SHAP scores can be misleading when evaluated only on the data support. By introducing the extended distribution $\mu^*$, defined as the product of feature marginals, the authors prove two main results: (i) zero aggregate SHAP values over $supp(\mu^*)$ imply that a feature can be safely discarded, and (ii) a robust bound for near-zero aggregates, extended to KernelSHAP via data scrambling to sample from $\mu^*$. The work develops the Shapley Lie Algebra to analyze SHAP value operators and proves their key algebraic properties, including invertibility and solvability, which underpin the main theorems. Practically, the paper provides a simple column-scrambling technique to generate $\mu^*$-distributed data for KernelSHAP, enabling sound feature-discard decisions in real-world settings and offering new theoretical insights into SHAP and KernelSHAP behavior.

Abstract

SHAP is one of the most popular local feature-attribution methods. Given a function f and an input x, it quantifies each feature's contribution to f(x). Recently, SHAP has been increasingly used for global insights: practitioners average the absolute SHAP values over many data points to compute global feature importance scores, which are then used to discard unimportant features. In this work, we investigate the soundness of this practice by asking whether small aggregate SHAP values necessarily imply that the corresponding feature does not affect the function. Unfortunately, the answer is no: even if the i-th SHAP value is 0 on the entire data support, there exist functions that clearly depend on Feature i. The issue is that computing SHAP values involves evaluating f on points outside of the data support, where f can be strategically designed to mask its dependence on Feature i. To address this, we propose to aggregate SHAP values over the extended support, which is the product of the marginals of the underlying distribution. With this modification, we show that a small aggregate SHAP value implies that we can safely discard the corresponding feature. We then extend our results to KernelSHAP, the most popular method to approximate SHAP values in practice. We show that if KernelSHAP is computed over the extended distribution, a small aggregate value justifies feature removal. This result holds independently of whether KernelSHAP accurately approximates true SHAP values, making it one of the first theoretical results to characterize the KernelSHAP algorithm itself. Our findings have both theoretical and practical implications. We introduce the Shapley Lie algebra, which offers algebraic insights that may enable a deeper investigation of SHAP and we show that randomly permuting each column of the data matrix enables safely discarding features based on aggregate SHAP and KernelSHAP values.

Figures (4)

• Figure 1: Example of a function where the aggregate SHAP value of Feature $1$ is $0$, yet the function depends on this feature.(a): Function $f:\mathop{\mathrm{\mathbb{R}}}\nolimits^2 \to \mathop{\mathrm{\mathbb{R}}}\nolimits$, supported on a ring with the color depicting the function value. The function clearly depends on both Features $1$ and $2$. (b): Point-wise SHAP values $\phi_1(\mu, f,x)$ of Feature $1$ are constantly $0$ on the support. This provides the counter-example we have been looking for. (c) and (d): Function and SHAP values on the extended support. Here the SHAP values are not constantly $0$ any more, illustrating the direction towards resolving the issue of the counter-example.
• Figure 2: Example of a function where the aggregate SHAP value of Feature $1$ is $0$, yet the function depends on this feature on a $3 \times 3$-grid.(a): Function $f:\mathop{\mathrm{\mathbb{R}}}\nolimits^2 \to \mathop{\mathrm{\mathbb{R}}}\nolimits$, supported on only $4$ of the grid cells with the color depicting the function value. The function clearly depends on both Features $1$ and $2$. (b): Point-wise SHAP values $\phi_1(\mu, f,x)$ of Feature $1$ are constantly $0$ on the support. (c) and (d): Function and SHAP values on the extended support. Here the SHAP values are not constantly $0$ any more.
• Figure 3: Example of a function where the aggregate SHAP value of Feature $1$ is $0$, yet the function depends on this feature on a $4 \times 4$-grid.(a): Function $f:\mathop{\mathrm{\mathbb{R}}}\nolimits^2 \to \mathop{\mathrm{\mathbb{R}}}\nolimits$, supported on only $8$ of the grid cells with the color depicting the function value. The function clearly depends on both Features $1$ and $2$. (b): Point-wise SHAP values $\phi_1(\mu, f,x)$ of Feature $1$ are constantly $0$ on the support. (c) and (d): Function and SHAP values on the extended support. Here the SHAP values are not constantly $0$ any more.
• Figure 4: Example of a function where the aggregate SHAP value of Feature $1$ is $0$ on the whole extended support and the function does not depend on this feature.(a): Function $f:\mathop{\mathrm{\mathbb{R}}}\nolimits^2 \to \mathop{\mathrm{\mathbb{R}}}\nolimits$, supported on a ring with the color depicting the function value. The function solely depends on Feature $2$. (b): Point-wise SHAP values $\phi_1(\mu, f,x)$ of Feature $1$ are constantly $0$ on the support. (c) and (d): Function and SHAP values on the extended support. Here the SHAP values are constantly $0$ on the extended support as well.

Theorems & Definitions (41)

• Definition 1: Value Function
• Definition 2: SHAP Values
• Definition 3: Aggregate SHAP Values
• Definition 4: Determined Function / Discarding Features
• Definition 5: Extended distribution and extended support
• Theorem 6: Discarding features based on constant-zero SHAP values on extended support
• Definition 7: Determined Function Space
• Lemma 8: Value operator
• Definition 9: SHAP operator
• Lemma 10: Properties of $A_i$ and $B_i$