Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Low-rank Solutions of Linear Matrix Equations via Procrustes Flow

Stephen Tu, Ross Boczar, Max Simchowitz, Mahdi Soltanolkotabi, Benjamin Recht

TL;DR

This work introduces Procrustes Flow, a two-phase nonconvex algorithm for recovering a low-rank matrix from linear measurements under Restricted Isometry Property (RIP). It combines a low-rank projection-based initialization with gradient-descent refinements for both PSD and rectangular cases, and proves geometric convergence once the initial estimate lies in a neighborhood of the true factors. In the PSD setting, with $\delta_{6r}\le 1/10$ and Gaussian-type measurements, the method achieves initialization within a constant radius and converges at a rate $\bigl(1- \tfrac{8}{25}\tfrac{\mu}{\kappa}\bigr)^{\tau/2}$; in the rectangular case, a lifted formulation yields analogous guarantees with $\delta_{6r}\le 1/25$. The results imply near-optimal sample complexity, $m=\Omega((n_1+n_2)r)$ for Gaussian ensembles, and demonstrate a scalable, nonconvex approach with strong convergence guarantees for both PSD and general low-rank matrix recovery scenarios.

Abstract

In this paper we study the problem of recovering a low-rank matrix from linear measurements. Our algorithm, which we call Procrustes Flow, starts from an initial estimate obtained by a thresholding scheme followed by gradient descent on a non-convex objective. We show that as long as the measurements obey a standard restricted isometry property, our algorithm converges to the unknown matrix at a geometric rate. In the case of Gaussian measurements, such convergence occurs for a $n_1 \times n_2$ matrix of rank $r$ when the number of measurements exceeds a constant times $(n_1+n_2)r$.

Low-rank Solutions of Linear Matrix Equations via Procrustes Flow

TL;DR

This work introduces Procrustes Flow, a two-phase nonconvex algorithm for recovering a low-rank matrix from linear measurements under Restricted Isometry Property (RIP). It combines a low-rank projection-based initialization with gradient-descent refinements for both PSD and rectangular cases, and proves geometric convergence once the initial estimate lies in a neighborhood of the true factors. In the PSD setting, with and Gaussian-type measurements, the method achieves initialization within a constant radius and converges at a rate ; in the rectangular case, a lifted formulation yields analogous guarantees with . The results imply near-optimal sample complexity, for Gaussian ensembles, and demonstrate a scalable, nonconvex approach with strong convergence guarantees for both PSD and general low-rank matrix recovery scenarios.

Abstract

In this paper we study the problem of recovering a low-rank matrix from linear measurements. Our algorithm, which we call Procrustes Flow, starts from an initial estimate obtained by a thresholding scheme followed by gradient descent on a non-convex objective. We show that as long as the measurements obey a standard restricted isometry property, our algorithm converges to the unknown matrix at a geometric rate. In the case of Gaussian measurements, such convergence occurs for a matrix of rank when the number of measurements exceeds a constant times .

Paper Structure

This paper contains 24 sections, 17 theorems, 112 equations, 2 algorithms.

Table of Contents

  1. Introduction
  2. Algorithms
  3. Initialization via low-rank projected gradients
  4. Successive refinement via gradient descent -- positive semidefinite case
  5. Successive refinement via gradient descent -- general case
  6. Main Results
  7. Quadratic measurements
  8. Bilinear measurements
  9. Related work
  10. Proofs
  11. Preliminaries
  12. Proof of convergence of gradient descent updates (Equation \ref{['convthm']})
  13. Proof of the regularity condition for the function $F$ (Lemma \ref{['lemma:expected_f_rc_cond']})
  14. Proof of gradient concentration (Lemma \ref{['lemma:concentration:gradient']})
  15. Proof of Lipschitz gradients around optimal solution (Lemma \ref{['lemma:concentration:gradientnorm']})
  16. ...and 9 more sections

Key Result

Theorem 3.2

Let $\bm{M}\in\mathbb{R}^{n\times n}$ be an arbitrary rank-$r$ symmetric positive semidefinite matrix with singular values $\sigma_1(\bm{M}) \geq \sigma_2(\bm{M}) \geq ... \geq \sigma_r(\bm{M}) > 0$ and condition number $\kappa = \sigma_1(\bm{M})/\sigma_r(\bm{M})$. Assume $\bm{M} = \bm{XX}^\mathsf{T Furthermore, take a constant step size $\mu_\tau=\mu$ for all $\tau=1,2,\ldots$, with $\mu \le 36/4

Theorems & Definitions (21)

  • Definition 3.1: Restricted Isometry Property (RIP) candes2005decodingrecht10
  • Theorem 3.2
  • Theorem 3.3
  • Lemma 3.4
  • Lemma 5.1
  • Lemma 5.2
  • Lemma 5.3
  • proof
  • Lemma 5.4
  • Definition 5.5
  • ...and 11 more