SHAP-based Explanations are Sensitive to Feature Representation

TL;DR

The paper investigates how SHAP-based local explanations for tabular data are influenced by upstream feature representations, revealing that simple data engineering like bucketization and categorical encoding can drastically alter reported feature importance and rankings. It demonstrates a Bayes-optimized attack to manipulate SHAP ranks while preserving model fidelity, illustrating practical risks for audits and fairness assessments. The work argues for robust explanation frameworks that account for data preprocessing and advocates governance practices for data engineering in explainability. Overall, it highlights a critical vulnerability in widely used post-hoc explanations and outlines directions to improve transparency and auditability in AI systems.

Abstract

Local feature-based explanations are a key component of the XAI toolkit. These explanations compute feature importance values relative to an ``interpretable'' feature representation. In tabular data, feature values themselves are often considered interpretable. This paper examines the impact of data engineering choices on local feature-based explanations. We demonstrate that simple, common data engineering techniques, such as representing age with a histogram or encoding race in a specific way, can manipulate feature importance as determined by popular methods like SHAP. Notably, the sensitivity of explanations to feature representation can be exploited by adversaries to obscure issues like discrimination. While the intuition behind these results is straightforward, their systematic exploration has been lacking. Previous work has focused on adversarial attacks on feature-based explainers by biasing data or manipulating models. To the best of our knowledge, this is the first study demonstrating that explainers can be misled by standard, seemingly innocuous data engineering techniques.

Figures (12)

• Figure 1: A hypothetical lending example: For a given classifier model, if we bucketize the age feature to generate a local explanation of the outcome with SHAP, then the importance of age for Ann decreases compared to using the raw value of the feature. Intuitively, this happens because $age=30$ is infrequent---and low---in this hypothetical dataset, while $age < 50$ appears to be typical, both on its own and in combination with education.
• Figure 2: SHAP values of features before (a) and after (b) bucketization for a fixed individual from the ACS Income dataset. Note that the classifier model and SHAP explainer remain fixed; the only modification to the individual's features from (a) $\rightarrow$ (b) was the bucketization of age. In (a), age is encoded as a continuous feature and is deemed most important by SHAP, with a rank of 1 and a feature weight of 0.99. In (b), the age feature was bucketized into 12 equi-width intervals over its active domain, using the median age to represent observations within each interval. This decreased the feature weight to 0.37, demoting age to the 5th rank in importance.
• Figure 3: The figures above represent different ways to bucketize a continuous or high-dimensional ordinal feature like age. In each sub-figure, the feature is represented as a lineplot with values from 0 to 99. Each circle represents an age value for a single observation from the dataset, and the observations are the same across the sub-figures. For equi-width buckets (a), the domain is divided into buckets of equal width. For equi-depth buckets (b), buckets are created that all contain an approximately equal number of observations (this is equivalent to equi-width buckets over the percentile values of the feature). Sub-figure (c) shows how custom buckets may be created by the method described in Section \ref{['sec:manipulation']} to manipulate the SHAP rank of a feature.
• Figure 4: Number of buckets versus average feature importance weight and rank of the age feature on ACS Income. For each plot, we compare uniform (equi-width) and quantile (equi-depth) histograms.
• Figure 5: Frequencies of rank shifts of the age feature when using (a) 5 equi-width, (b) 5 equi-depth, (c) 10 equi-width, or (d) 10 equi-depth buckets on ACS Income. The frequencies are shown on a log scale for better readability. Negative rank shifts represent "demotion" of age where the rank values increase (e.g., from Rank 1 to 10) making age less important, while positive shifts represent rank "promotion" where the importance of age increases.