Covariate-assisted Grade of Membership Models via Shared Latent Geometry
Zhiyu Xu, Yuqi Gu
TL;DR
The paper tackles latent structure recovery in multivariate categorical data when auxiliary covariates are available, criticizing joint-likelihood approaches for their computational burden and misspecification sensitivity.It introduces a covariate-assisted GoM framework that leverages a shared low-rank simplex geometry between responses and covariates via a likelihood-free spectral estimator, incorporating a balance parameter $\alpha$ and heteroskedastic PCA (HeteroPCA).Theoretical results show weaker identifiability requirements than covariate-free GoM and provide finite-sample, entrywise error bounds for mixed membership scores $\boldsymbol{\Pi}$ and item/covariate parameters $\boldsymbol{\Theta},\mathbf{M}$, with faster convergence in high dimensions when covariates are informative.Empirical evidence from simulations and a TIMSS educational assessment application demonstrates improved accuracy, computational efficiency, and interpretability, supported by an open-source code release.
Abstract
The grade of membership model is a flexible latent variable model for analyzing multivariate categorical data through individual-level mixed membership scores. In many modern applications, auxiliary covariates are collected alongside responses and encode information about the same latent structure. Traditional approaches to incorporating such covariates typically rely on fully specified joint likelihoods, which are computationally intensive and sensitive to misspecification. We introduce a covariate-assisted grade of membership model that integrates response and covariate information by exploiting their shared low-rank simplex geometry, rather than modeling their joint distribution. We propose a likelihood-free spectral estimation procedure that combines heterogeneous data sources through a balance parameter controlling their relative contribution. To accommodate high-dimensional and heteroskedastic noise, we employ heteroskedastic principal component analysis before performing simplex-based geometric recovery. Our theoretical analysis establishes weaker identifiability conditions than those required in the covariate-free model, and further derives finite-sample, entrywise error bounds for both mixed membership scores and item parameters. These results demonstrate that auxiliary covariates can provably improve latent structure recovery, yielding faster convergence rates in high-dimensional regimes. Simulation studies and an application to educational assessment data illustrate the computational efficiency, statistical accuracy, and interpretability gains of the proposed method. The code for reproducing these results is open-source and available at \texttt{https://github.com/Toby-X/Covariate-Assisted-GoM}