Degree-heterogeneous Latent Class Analysis for High-dimensional Discrete Data

TL;DR

This work develops a degree-heterogeneous latent class framework (DhLCM) for high-dimensional discrete data and introduces HeteroClustering, a scalable spectral method that uses heteroskedastic PCA followed by ell2 normalization to remove degree effects for exact clustering under weak signal-to-noise conditions. It then provides a comprehensive estimation and inference pipeline for the high-dimensional item-parameter matrix Θ, including identifiability conditions, a simple estimator, sharp error bounds, distributional limits, and global/multiple testing procedures with valid error control. The approach is validated through extensive simulations and three real-data applications (Senate voting, genetic SNP variation, and single-cell sequencing), demonstrating accurate clustering, interpretable markers, and reliable inference. The work also extends to Binomial and Poisson data, discusses lower bounds, and offers supplementary results and proofs, highlighting the method’s robustness and broad applicability to discrete, high-dimensional problems. Overall, DhLCM with HeteroClustering delivers a theory-backed, scalable framework for uncovering qualitative and quantitative heterogeneity in complex discrete data, with practical impact on biomarker discovery and latent class characterization.

Abstract

The latent class model is a widely used mixture model for multivariate discrete data. Besides the existence of qualitatively heterogeneous latent classes, real data often exhibit additional quantitative heterogeneity nested within each latent class. The modern latent class analysis also faces extra challenges, including the high-dimensionality, sparsity, and heteroskedastic noise inherent in discrete data. Motivated by these phenomena, we introduce the Degree-heterogeneous Latent Class Model and propose an easy-to-implement HeteroClustering algorithm for it. HeteroClustering uses heteroskedastic PCA with $\ell_2$ normalization to remove degree effects and perform clustering in the top singular subspace of the data matrix. We establish the result of exact clustering under minimal signal-to-noise conditions. We further investigate the estimation and inference of the high-dimensional continuous item parameters in the model, which are crucial to interpreting and finding useful markers for latent classes. We provide comprehensive procedures for global testing and multiple testing of these parameters with valid error controls. The superior performance of our methods is demonstrated through extensive simulations and applications to three diverse real-world datasets from political voting records, genetic variations, and single-cell sequencing.

Figures (13)

• Figure 1: Scatter plots of singular subspace embeddings given by HeteroPCA. Left: first and second singular vectors of the Senate voting data. Middle: first and second singular vectors of the single-cell data. Right: fourth and sixth singular vectors of the genetic variation data.
• Figure 2: Clustering error boxplots for Bernoulli model with (left) and without (right) degree heterogeneity, with $N=200, J=1000, K=10$ and 500 replications. From left to right in each figure are $\ell_2$ normalization, SCORE normalization, and no normalization.
• Figure 3: Boxplots comparing SVD and HeteroPCA in terms of the Frobenius norm error of $\widehat{\mathbf{U}}$ (left), clustering error (middle), and the maximum absolute error of $\widehat{\mathbf{\Theta}}$ (right) for the Bernoulli model (upper row) and the Poisson model (lower row).
• Figure 4: Q-Q plots of $p$-values for testing the null hypothesis $H_0:\theta_{j,1}=\theta_{j,2}=\cdots \theta_{j,K}$ in the Bernoulli model. $H_0$ is true for feature $1$ (upper row) and false for feature $2$ (lower row).
• Figure S.1: Clustering error and computation time in seconds comparing JML, MML, and our proposed method, when the ground truth model does not have degree heterogeneity.

Theorems & Definitions (33)

• Lemma 1
• Theorem 1
• Remark 1
• Theorem 2
• Definition 1: $\left( \mathbf{\Omega}, \mathbf{\Theta}\right)$-identifiable
• Proposition 1
• Theorem 3
• Theorem 4
• Theorem 5
• Remark 2