Feature Importance Depends on Properties of the Data: Towards Choosing the Correct Explanations for Your Data and Decision Trees based Models

TL;DR

The paper examines when local explanations for tree-based models yield reliable feature-importance estimates by benchmarking multiple explainers on synthetic data with controlled properties and real-world binary classification datasets. It introduces a Bayesian-network based synthetic data framework that provides ground-truth feature importances φ^* relative to the joint distribution $P(X,Y)$, and evaluates methods including LIME, LSurro, KShap, SShap, Tshap, and TI. Key findings show substantial disparities in magnitude and sign of attributions across methods and a clear sensitivity to data properties; SHAP variants are generally consistent and fast, while local surrogates can be highly stable but may overestimate uninformative features, and TI can be brittle in noisy settings. The results yield practical guidance on selecting explainability methods based on data characteristics and highlight avenues for targeted future work in deeper method-specific analyses and broader data domains.

Abstract

In order to ensure the reliability of the explanations of machine learning models, it is crucial to establish their advantages and limits and in which case each of these methods outperform. However, the current understanding of when and how each method of explanation can be used is insufficient. To fill this gap, we perform a comprehensive empirical evaluation by synthesizing multiple datasets with the desired properties. Our main objective is to assess the quality of feature importance estimates provided by local explanation methods, which are used to explain predictions made by decision tree-based models. By analyzing the results obtained from synthetic datasets as well as publicly available binary classification datasets, we observe notable disparities in the magnitude and sign of the feature importance estimates generated by these methods. Moreover, we find that these estimates are sensitive to specific properties present in the data. Although some model hyper-parameters do not significantly influence feature importance assignment, it is important to recognize that each method of explanation has limitations in specific contexts. Our assessment highlights these limitations and provides valuable insight into the suitability and reliability of different explanatory methods in various scenarios.

Figures (10)

• Figure 1: Bayesian networks that we use as a schema to generate synthetic data, illustrating one full and three partial factorizations of $P(X,Y)$.
• Figure 2: Synthetic data XOR. $X \sim \mathcal{N}(\mu, \Sigma)$. Each dataset expresses a different combination of properties.
• Figure 3: Synthetic data NOT. $X \sim \mathcal{N}(\mu, \Sigma)$. Each dataset expresses a different combination of properties.
• Figure 4: Normalized feature importance estimates of the XOR datasets. These feature importance estimates are obtained for the decision trees trained on datasets with 1 000 instances.
• Figure 5: Normalized feature importance estimates of the NOT datasets. These feature importance estimates are obtained for the decision trees trained on datasets with 1 000 instances.