Spectrally-Corrected and Regularized Linear Discriminant Analysis for Spiked Covariance Model

TL;DR

The paper addresses high-dimensional binary classification where $p$ and $n$ are large and potentially comparable, and proposes SRLDA, a discriminant analysis method that uses a spectrally corrected covariance aligned with a spiked covariance model $\Sigma = \sigma^2(\mathbf I_p + \sum_j \lambda_j \mathbf v_j \mathbf v_j^T)$ together with a regularized inverse $\widetilde{\mathbf H}$. It derives deterministic equivalents for the misclassification rate and identifies closed-form optimal reg parameters $\gamma_1^*, \gamma_2^*$, yielding an asymptotically optimal SRLDA classifier, with extensions to intercept optimization (OI-SRLDA) and to multi-class settings via generalized Fisher discriminant analysis. The approach leverages large-dimensional random matrix theory to ensure consistency and tractability, including practical spike estimation and bias correction. Empirical results on simulated data and real datasets (e.g., MNIST and LFW) show SRLDA achieving higher accuracy and better dimensionality reduction than RLDA and ILDA, while remaining computationally more efficient than SVM/KNN/CNN methods, and capable of scalable multi-class discrimination.

Abstract

This paper proposes an improved linear discriminant analysis called spectrally-corrected and regularized LDA (SRLDA). This method integrates the design ideas of the sample spectrally-corrected covariance matrix and the regularized discriminant analysis. With the support of a large-dimensional random matrix analysis framework, it is proved that SRLDA has a linear classification global optimal solution under the spiked model assumption. According to simulation data analysis, the SRLDA classifier performs better than RLDA and ILDA and is closer to the theoretical classifier. Experiments on different data sets show that the SRLDA algorithm performs better in classification and dimensionality reduction than currently used tools.