Asymptotic Normality of Generalized Low-Rank Matrix Sensing via Riemannian Geometry
Osbert Bastani
TL;DR
The paper addresses uncertainty quantification for generalized low-rank matrix sensing under a broad convex loss by removing degeneracy caused by rotational symmetry. It introduces a Riemannian-geometry framework, parameterizing M^* as \\bar{\\theta}^*\\bar{\\theta}^{*\\top} and working on the quotient manifold Θ = \\bar{\\Theta} / O(k) to obtain a non-degenerate Hessian on the horizontal space. The main result shows that, with appropriate local conditions and regularity, the estimator satisfies \\sqrt{n}(\\phi^0-\\phi^*) \\xrightarrow\\{D} \\mathcal{N}(0,(H^*)^{-1}) within the non-degenerate subspace, enabling asymptotic normality and uncertainty quantification for generalized low-rank sensing. The work combines Taylor expansions on manifolds, spectral bounds, and concentration analysis to extend classical MLE asymptotics to structured matrix models, with potential impact on statistical inference for low-rank representations in diverse settings.
Abstract
We prove an asymptotic normality guarantee for generalized low-rank matrix sensing -- i.e., matrix sensing under a general convex loss $\bar\ell(\langle X,M\rangle,y^*)$, where $M\in\mathbb{R}^{d\times d}$ is the unknown rank-$k$ matrix, $X$ is a measurement matrix, and $y^*$ is the corresponding measurement. Our analysis relies on tools from Riemannian geometry to handle degeneracy of the Hessian of the loss due to rotational symmetry in the parameter space. In particular, we parameterize the manifold of low-rank matrices by $\barθ\barθ^\top$, where $\barθ\in\mathbb{R}^{d\times k}$. Then, assuming the minimizer of the empirical loss $\barθ^0\in\mathbb{R}^{d\times k}$ is in a constant size ball around the true parameters $\barθ^*$, we prove $\sqrt{n}(φ^0-φ^*)\xrightarrow{D}N(0,(H^*)^{-1})$ as $n\to\infty$, where $φ^0$ and $φ^*$ are representations of $\barθ^*$ and $\barθ^0$ in the horizontal space of the Riemannian quotient manifold $\mathbb{R}^{d\times k}/\text{O}(k)$, and $H^*$ is the Hessian of the true loss in the same representation.