Nonnegative Low-rank Matrix Recovery Can Have Spurious Local Minima
Richard Y. Zhang
TL;DR
This work shows that the benign nonconvex geometry often seen in RIP-based low-rank recovery does not survive the imposition of nonnegativity constraints. The authors construct explicit counterexamples, including a rank-1, fully observed case with $δ=0$ that remains benign, but demonstrate that any $δ>0$, higher ground-truth rank $r^{\star}>1$, or overparameterization $r\ge r^{\star}$ yields spurious local minima even in simple instances. A positive result holds for the case $δ=0$ and $r^{\star}=1$, where no spurious local minima exist in both symmetric and asymmetric factorizations. The findings indicate that stability-based explanations for algorithmic success in nonnegative low-rank recovery are insufficient, motivating new analytical frameworks for projected/ constrained methods in this setting.
Abstract
Low-rank matrix recovery is well-known to exhibit benign nonconvexity under the restricted isometry property (RIP): every second-order critical point is globally optimal, so local methods provably recover the ground truth. Motivated by the strong empirical performance of projected gradient methods for nonnegative low-rank recovery problems, we investigate whether this benign geometry persists when the factor matrices are constrained to be elementwise nonnegative. In the simple setting of a rank-1 nonnegative ground truth, we confirm that benign nonconvexity holds in the fully-observed case with RIP constant $δ=0$. This benign nonconvexity, however, is unstable. It fails to extend to the partially-observed case with any arbitrarily small RIP constant $δ>0$, and to higher-rank ground truths $r^{\star}>1$, regardless of how much the search rank $r\ge r^{\star}$ is overparameterized. Together, these results undermine the standard stability-based explanation for the empirical success of nonconvex methods and suggest that fundamentally different tools are needed to analyze nonnegative low-rank recovery.