Near optimal sample complexity for matrix and tensor normal models via geodesic convexity
Cole Franks, Rafael Oliveira, Akshay Ramachandran, Michael Walter
TL;DR
We address estimating Kronecker-structured covariance factors in matrix and tensor normal Gaussian models, formulating the MLE on a geodesically convex manifold endowed with the Fisher information metric. The key technical advance is proving strong geodesic convexity of the negative log-likelihood via quantum-expander analysis, which yields near-optimal nonasymptotic sample complexity and error rates in Fisher-Rao and Thompson distances, valid without conditioning or initialization assumptions. The flip-flop algorithm is analyzed as a geodesic-descent method that converges linearly to the MLE with high probability, providing a practically efficient route to optimal estimation. Lower bounds extend classical Gaussian results to the Kronecker-structured setting, showing near-minimax optimality, while the tensor/matrix normal results are nearly tight up to logarithmic factors. Together, these results establish a coherent, condition-number-free theory for high-dimensional Kronecker-structured covariance estimation with scalable computation.
Abstract
The matrix normal model, i.e., the family of Gaussian matrix-variate distributions whose covariance matrices are the Kronecker product of two lower dimensional factors, is frequently used to model matrix-variate data. The tensor normal model generalizes this family to Kronecker products of three or more factors. We study the estimation of the Kronecker factors of the covariance matrix in the matrix and tensor normal models. For the above models, we show that the maximum likelihood estimator (MLE) achieves nearly optimal nonasymptotic sample complexity and nearly tight error rates in the Fisher-Rao and Thompson metrics. In contrast to prior work, our results do not rely on the factors being well-conditioned or sparse, nor do we need to assume an accurate enough initial guess. For the matrix normal model, all our bounds are minimax optimal up to logarithmic factors, and for the tensor normal model our bounds for the largest factor and for overall covariance matrix are minimax optimal up to constant factors provided there are enough samples for any estimator to obtain constant Frobenius error. In the same regimes as our sample complexity bounds, we show that the flip-flop algorithm, a practical and widely used iterative procedure to compute the MLE, converges linearly with high probability. Our main technical insight is that, given enough samples, the negative log-likelihood function is strongly geodesically convex in the geometry on positive-definite matrices induced by the Fisher information metric. This strong convexity is determined by the expansion of certain random quantum channels.