Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Group Quadratic Discriminant Analysis via Projection

Yuchao Wang, Tianying Wang

TL;DR

MGQDA addresses high-dimensional multi-group classification with group-specific covariance structures by projecting predictors onto a joint discriminant subspace and applying a QDA rule on the projected features. It constructs group-wise projection bases from generalized eigenvectors and enforces sparse, shared and group-specific variable selection via a convex penalty, enabling scalable discrimination without direct high-dimensional covariance estimation. Theoretical results establish variable selection consistency under standard assumptions, and simulations plus a DepMap gene-expression application show competitive misclassification rates and interpretable, group-specific markers. The method provides a practical, scalable tool for high-dimensional multi-group problems in genomics and related fields where covariance heterogeneity matters.

Abstract

Multi-group classification arises in many prediction and decision-making problems, including applications in epidemiology, genomics, finance, and image recognition. Although classification methods have advanced considerably, much of the literature focuses on binary problems, and available extensions often provide limited flexibility for multi-group settings. Recent work has extended linear discriminant analysis to multiple groups, but more general methods are still needed to handle complex structures such as nonlinear decision boundaries and group-specific covariance patterns. We develop Multi-Group Quadratic Discriminant Analysis (MGQDA), a method for multi-group classification built on quadratic discriminant analysis. MGQDA projects high-dimensional predictors onto a lower-dimensional subspace, which enables accurate classification while capturing nonlinearity and heterogeneity in group-specific covariance structures. We derive theoretical guarantees, including variable selection consistency, to support the reliability of the procedure. In simulations and a gene-expression application, MGQDA achieves competitive or improved predictive performance compared with existing methods while selecting group-specific informative variables, indicating its practical value for high-dimensional multi-group classification problems. Supplementary materials for this article are available online.

Multi-Group Quadratic Discriminant Analysis via Projection

TL;DR

MGQDA addresses high-dimensional multi-group classification with group-specific covariance structures by projecting predictors onto a joint discriminant subspace and applying a QDA rule on the projected features. It constructs group-wise projection bases from generalized eigenvectors and enforces sparse, shared and group-specific variable selection via a convex penalty, enabling scalable discrimination without direct high-dimensional covariance estimation. Theoretical results establish variable selection consistency under standard assumptions, and simulations plus a DepMap gene-expression application show competitive misclassification rates and interpretable, group-specific markers. The method provides a practical, scalable tool for high-dimensional multi-group problems in genomics and related fields where covariance heterogeneity matters.

Abstract

Multi-group classification arises in many prediction and decision-making problems, including applications in epidemiology, genomics, finance, and image recognition. Although classification methods have advanced considerably, much of the literature focuses on binary problems, and available extensions often provide limited flexibility for multi-group settings. Recent work has extended linear discriminant analysis to multiple groups, but more general methods are still needed to handle complex structures such as nonlinear decision boundaries and group-specific covariance patterns. We develop Multi-Group Quadratic Discriminant Analysis (MGQDA), a method for multi-group classification built on quadratic discriminant analysis. MGQDA projects high-dimensional predictors onto a lower-dimensional subspace, which enables accurate classification while capturing nonlinearity and heterogeneity in group-specific covariance structures. We derive theoretical guarantees, including variable selection consistency, to support the reliability of the procedure. In simulations and a gene-expression application, MGQDA achieves competitive or improved predictive performance compared with existing methods while selecting group-specific informative variables, indicating its practical value for high-dimensional multi-group classification problems. Supplementary materials for this article are available online.
Paper Structure (12 sections, 7 theorems, 13 equations, 3 figures)

This paper contains 12 sections, 7 theorems, 13 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Methodology
  3. Background and Motivation
  4. Proposed Method
  5. Sparse Estimation
  6. Optimization Algorithm
  7. Theoretical Results
  8. Simulation Studies
  9. Simulation Settings
  10. Results
  11. Real Data Application
  12. Discussion

Key Result

Proposition 2.1

We have $\Sigma_B = \Gamma\Gamma^{\top}$ and $\widehat{\Sigma}_B = \widehat{\Gamma}\widehat{\Gamma}^{\top}$, where $\Gamma\in\mathbb{R}^{p\times(G-1)}$. Further, the $r$th column of $\Gamma$ has the form $\Gamma_r = \left[\sqrt{\pi_{r + 1}}\{\sum_{i = 1}^r \pi_i (\mu_i - \mu_{i + 1})\}\right]/\sqrt{

Figures (3)

  • Figure 1: Misclassification error rates for MGQDA and competing methods across all simulation settings, based on 100 replications. The horizontal line in each panel marks the median error of MGQDA.
  • Figure 2: TPR for MGQDA, MGSDA, pLDA, and SLDA across all simulation settings, based on 100 replications. The red horizontal line in each panel marks the median TPR of MGQDA.
  • Figure 4: Classification performance and variable selection on the DepMap gene expression dataset for MGQDA and competing methods. Left: classification error rates across 100 replications (red horizontal line: MGQDA median error). Middle: number of selected variables across replications for sparse methods. Right: MGQDA-selected variable counts by experiment; the $x$-axis indexes experiments and the $y$-axis gives the number of variables selected in the projection basis.

Theorems & Definitions (9)

  • Proposition 2.1
  • Proposition 2.2
  • Proposition 2.3
  • Remark 1
  • Theorem 1
  • Theorem 2
  • Corollary 1
  • Theorem 3
  • Remark 2