Directly Handling Missing Data in Linear Discriminant Analysis for Enhancing Classification Accuracy and Interpretability
Tuan L. Vo, Uyen Dang, Thu Nguyen
TL;DR
This work tackles the challenge of missing data in Linear Discriminant Analysis while preserving model explainability. It introduces WLDA, which integrates a diagonal weighted missing matrix to penalize missing entries and enables parameter estimation directly on incomplete data without imputation. The authors derive the theoretical properties of WLDA, including its decision boundaries and the expectation, variance, and bias of the classification score under missingness, and demonstrate empirical improvements over imputation-based baselines across several datasets. The method also provides interpretable mechanisms, such as correlation visualization and Shapley-valued feature contributions, key for trustworthy deployment in sensitive domains.
Abstract
As the adoption of Artificial Intelligence (AI) models expands into critical real-world applications, ensuring the explainability of these models becomes paramount, particularly in sensitive fields such as medicine and finance. Linear Discriminant Analysis (LDA) remains a popular choice for classification due to its interpretable nature, derived from its capacity to model class distributions and enhance class separation through linear combinations of features. However, real-world datasets often suffer from incomplete data, posing substantial challenges for both classification accuracy and model interpretability. In this paper, we introduce a novel and robust classification method, termed Weighted missing Linear Discriminant Analysis (WLDA), which extends LDA to handle datasets with missing values without the need for imputation. Our approach innovatively incorporates a weight matrix that penalizes missing entries, thereby refining parameter estimation directly on incomplete data. This methodology not only preserves the interpretability of LDA but also significantly enhances classification performance in scenarios plagued by missing data. We conduct an in-depth theoretical analysis to establish the properties of WLDA and thoroughly evaluate its explainability. Experimental results across various datasets demonstrate that WLDA consistently outperforms traditional methods, especially in challenging environments where missing values are prevalent in both training and test datasets. This advancement provides a critical tool for improving classification accuracy and maintaining model transparency in the face of incomplete data.