Low Rank Matrix Completion via Robust Alternating Minimization in Nearly Linear Time
Yuzhou Gu, Zhao Song, Junze Yin, Lichen Zhang
TL;DR
The paper tackles low-rank matrix completion by developing a robust alternating minimization framework that tolerates approximate updates while preserving sample complexity. It introduces a subspace-approaching inductive argument and a perturbation theory for matrix incoherence to control errors, then leverages a sketching-based preconditioner to solve weighted mult-response regressions in near-linear time, achieving a total runtime of $\widetilde{O}(|\Omega|k)$. The method matches prior sample complexity results while significantly improving computational efficiency, and it positions alternating minimization as a practical, scalable approach for matrix completion. The work also clarifies the relationship to recent advances (e.g., \cite{kllst23}) and emphasizes practical implementability through high-accuracy regression solvers and fast verification of entries. Overall, the framework bridges theory and practice by enabling fast, approximate updates without sacrificing convergence guarantees or recovery quality.
Abstract
Given a matrix $M\in \mathbb{R}^{m\times n}$, the low rank matrix completion problem asks us to find a rank-$k$ approximation of $M$ as $UV^\top$ for $U\in \mathbb{R}^{m\times k}$ and $V\in \mathbb{R}^{n\times k}$ by only observing a few entries specified by a set of entries $Ω\subseteq [m]\times [n]$. In particular, we examine an approach that is widely used in practice -- the alternating minimization framework. Jain, Netrapalli, and Sanghavi [JNS13] showed that if $M$ has incoherent rows and columns, then alternating minimization provably recovers the matrix $M$ by observing a nearly linear in $n$ number of entries. While the sample complexity has been subsequently improved [GLZ17], alternating minimization steps are required to be computed exactly. This hinders the development of more efficient algorithms and fails to depict the practical implementation of alternating minimization, where the updates are usually performed approximately in favor of efficiency. In this paper, we take a major step towards a more efficient and error-robust alternating minimization framework. To this end, we develop an analytical framework for alternating minimization that can tolerate a moderate amount of errors caused by approximate updates. Moreover, our algorithm runs in time $\widetilde O(|Ω| k)$, which is nearly linear in the time to verify the solution while preserving the sample complexity. This improves upon all prior known alternating minimization approaches which require $\widetilde O(|Ω| k^2)$ time.