Explainability of Machine Learning Models under Missing Data
Tuan L. Vo, Thu Nguyen, Luis M. Lopez-Ramos, Hugo L. Hammer, Michael A. Riegler, Pal Halvorsen
TL;DR
This work addresses the interplay between missing data handling and explainability by studying how imputation methods affect SHAP-based attributions. It combines theoretical analysis under MCAR with large-scale experiments across regression and classification tasks, comparing six imputation methods and XGBoost's direct-missing-data capability. The findings show that imputation choice can substantially alter Shapley values, and that methods delivering the best predictive MSE do not always preserve the original feature importance structure, underscoring the need to tailor imputation to analysis goals. Practically, the paper provides guidance for practitioners on selecting imputation strategies compatible with their data characteristics and explainability objectives, and highlights the potential of DIMV and similar approaches to better preserve explanations.
Abstract
Missing data is a prevalent issue that can significantly impair model performance and explainability. This paper briefly summarizes the development of the field of missing data with respect to Explainable Artificial Intelligence and experimentally investigates the effects of various imputation methods on SHAP (SHapley Additive exPlanations), a popular technique for explaining the output of complex machine learning models. Next, we compare different imputation strategies and assess their impact on feature importance and interaction as determined by Shapley values. Moreover, we also theoretically analyze the effects of missing values on Shapley values. Importantly, our findings reveal that the choice of imputation method can introduce biases that could lead to changes in the Shapley values, thereby affecting the explainability of the model. Moreover, we also show that a lower test prediction MSE (Mean Square Error) does not necessarily imply a lower MSE in Shapley values and vice versa. Also, while XGBoost (eXtreme Gradient Boosting) is a method that could handle missing data directly, using XGBoost directly on missing data can seriously affect explainability compared to imputing the data before training XGBoost. This study provides a comprehensive evaluation of imputation methods in the context of model explanations, offering practical guidance for selecting appropriate techniques based on dataset characteristics and analysis objectives. The results underscore the importance of considering imputation effects to ensure robust and reliable insights from machine learning models.