Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence Analysis for Rectangular Matrix Completion Using Burer-Monteiro Factorization and Gradient Descent

Qinqing Zheng, John Lafferty

TL;DR

The paper analyzes a nonconvex approach to rectangular matrix completion by lifting the target matrix to a positive semidefinite form and optimizing a factored representation with gradient descent. It introduces a spectral-initialized, projected gradient method on the lifted factor Z, with a regularizer to align column spaces and an incoherence constraint. Under a near-optimal sampling regime, the authors prove geometric convergence to the true lifted factor Z*, providing explicit conditions on m, κ, and μ. Empirical results on synthetic data corroborate the theory, showing scalability and competitive performance relative to existing methods, and suggesting the method scales to large problems with favorable sample complexity.

Abstract

We address the rectangular matrix completion problem by lifting the unknown matrix to a positive semidefinite matrix in higher dimension, and optimizing a nonconvex objective over the semidefinite factor using a simple gradient descent scheme. With $O( μr^2 κ^2 n \max(μ, \log n))$ random observations of a $n_1 \times n_2$ $μ$-incoherent matrix of rank $r$ and condition number $κ$, where $n = \max(n_1, n_2)$, the algorithm linearly converges to the global optimum with high probability.

Convergence Analysis for Rectangular Matrix Completion Using Burer-Monteiro Factorization and Gradient Descent

TL;DR

The paper analyzes a nonconvex approach to rectangular matrix completion by lifting the target matrix to a positive semidefinite form and optimizing a factored representation with gradient descent. It introduces a spectral-initialized, projected gradient method on the lifted factor Z, with a regularizer to align column spaces and an incoherence constraint. Under a near-optimal sampling regime, the authors prove geometric convergence to the true lifted factor Z*, providing explicit conditions on m, κ, and μ. Empirical results on synthetic data corroborate the theory, showing scalability and competitive performance relative to existing methods, and suggesting the method scales to large problems with favorable sample complexity.

Abstract

We address the rectangular matrix completion problem by lifting the unknown matrix to a positive semidefinite matrix in higher dimension, and optimizing a nonconvex objective over the semidefinite factor using a simple gradient descent scheme. With random observations of a -incoherent matrix of rank and condition number , where , the algorithm linearly converges to the global optimum with high probability.

Paper Structure

This paper contains 20 sections, 14 theorems, 84 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Semidefinite Lifting, Factorization, and Gradient Descent
  3. Main Result: Convergence Analysis
  4. Proof Sketch
  5. Related Work
  6. Experiments
  7. Computational Complexity
  8. Runtime Comparison
  9. Sample Complexity
  10. Conclusion
  11. Technical Lemmas
  12. Initialization
  13. Proof of Lemma \ref{['thm:init']}
  14. Proof of Corollary \ref{['coro:proj_const']}
  15. Regularity Condition
  16. ...and 5 more sections

Key Result

Theorem 1

Suppose that $X^\star$ is of rank $r$, with condition number $\kappa = \sigma^\star_1/\sigma^\star_r$, and $\mu$-incoherent as defined in Definition defn:incoh. Suppose further that we observe $m$ entries of $X^\star$ chosen uniformly at random. Let $Y^\star = Z^\star {Z^\star}^\top$ be the lifted m then with probability at least $1 - c_1 n^{-c_2}$ the iterates of Algorithm alg:gd1 converge to $Z^

Figures (2)

  • Figure 1: (a) Runtime comparison where $X^\star$ is $4000 \times 2000$ and of rank $3$. $199057$ entries are observed. (b) Magnified plots to compare other methods except nuclear.
  • Figure 2: (a) Runtime growth of AltMin, trustRegresion, GD and SVP. (b) Sample complexity of gradient scheme.

Theorems & Definitions (22)

  • Definition 1
  • Theorem 1
  • Definition 2
  • Theorem 2
  • Lemma 1
  • Corollary 1
  • Definition 3
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • ...and 12 more