Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast and Provable Nonconvex Robust Matrix Completion

Yichen Fu, Tianming Wang, Ke Wei

TL;DR

This work addresses robust matrix completion by proposing ARMC, a fast nonconvex method that uses tangent-space projections to update the low-rank component and soft-thresholding (or related) strategies for the sparse part. Leveraging a leave-one-out analysis, the authors prove entrywise linear convergence of ARMC to the ground-truth $L^{\star}$ under incoherence, sparse-outlier, and sub-Gaussian-noise assumptions, with improved sample complexity relative to convex approaches. Empirical results on synthetic and real data demonstrate ARMC’s superior computational efficiency and accuracy, including phase transitions and stability under noise. The approach thereby offers a practical, theory-backed tool for robust matrix completion in settings with missing data, adversarial outliers, and noise.

Abstract

This paper studies the robust matrix completion problem and a computationally efficient non-convex method called ARMC has been proposed. This method is developed by introducing subspace projection to a singular value thresholding based method when updating the low rank part. Numerical experiments on synthetic and real data show that ARMC is superior to existing non-convex RMC methods. Through a refined analysis based on the leave-one-out technique, we have established the theoretical guarantee for ARMC subject to both sparse outliers and stochastic noise. The established bounds for the sample complexity and outlier sparsity are better than those established for a convex approach that also considers both outliers and stochastic noise.

Fast and Provable Nonconvex Robust Matrix Completion

TL;DR

This work addresses robust matrix completion by proposing ARMC, a fast nonconvex method that uses tangent-space projections to update the low-rank component and soft-thresholding (or related) strategies for the sparse part. Leveraging a leave-one-out analysis, the authors prove entrywise linear convergence of ARMC to the ground-truth under incoherence, sparse-outlier, and sub-Gaussian-noise assumptions, with improved sample complexity relative to convex approaches. Empirical results on synthetic and real data demonstrate ARMC’s superior computational efficiency and accuracy, including phase transitions and stability under noise. The approach thereby offers a practical, theory-backed tool for robust matrix completion in settings with missing data, adversarial outliers, and noise.

Abstract

This paper studies the robust matrix completion problem and a computationally efficient non-convex method called ARMC has been proposed. This method is developed by introducing subspace projection to a singular value thresholding based method when updating the low rank part. Numerical experiments on synthetic and real data show that ARMC is superior to existing non-convex RMC methods. Through a refined analysis based on the leave-one-out technique, we have established the theoretical guarantee for ARMC subject to both sparse outliers and stochastic noise. The established bounds for the sample complexity and outlier sparsity are better than those established for a convex approach that also considers both outliers and stochastic noise.
Paper Structure (26 sections, 14 theorems, 114 equations, 4 figures, 1 table, 1 algorithm)

This paper contains 26 sections, 14 theorems, 114 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Motivation and Main Contributions
  3. Notation and Organization
  4. Algorithm and Main Result
  5. Accelerated Robust Matrix Completion Method
  6. Theoretical Guarantee of ARMC
  7. Numerical Experiments
  8. Synthetic Data
  9. Phase Transition
  10. Computational Time
  11. Stability against Additive Noise
  12. Real Data
  13. Proof Outline
  14. Proof of Theorem \ref{['thm:induction']}
  15. Base Case
  16. ...and 11 more sections

Key Result

Theorem 2.6

Suppose $\beta_1$ and $\beta_2$ in Algorithm Alg1 satisfy where $C_{\emph{init}}\geq 1$ and $C_N^{(1)}>0$ are two constants ($C_N^{(1)}$ is specified in equation eq:noise_infty). Let $C_{\emph{thresh}}:=(K+B)\cdot C_{\emph{init}}$, where $K$ and $B$ are parameters in Assumption assump4. Assume for some sufficiently large constant $C_{\emph{sample}}>0$ and some sufficiently small constant $c_{\em

Figures (4)

  • Figure 1: Empirical phase transitions for the tested algorithms with $n=1000$ and $r=5$. Left: Success rates for $\alpha=0.15$ and $p\in\{0.02,0.04,\cdots,0.26\}$ with $\kappa\in\{1,5\}$; Right: Success rates for $p=0.2$ and $\alpha\in\{0.2,0.21,\cdots,0.55\}$ with $\kappa=2$.
  • Figure 2: Computational efficiency of the tested algorithms with $r=10$, $\kappa=2$ and $\alpha\in\{0.1,0.2\}$. Left: Total runtime comparisons; Middle: Runtime per iteration comparisons; Right: Iteration counts comparisons.
  • Figure 3: Reconstruction stability of ARMC with respect to different noise levels in the observed samples. Left: Results with $r=5$ and $\alpha\in\{0.1,0.2\}$; Right: Results with $\alpha=0.1$ and $r\in\{5,10\}$. Here the relative reconstruction error is the noise-to-signal ratio calculated with $L_{\text{out}}-L^{\star}$ and $L^{\star}$.
  • Figure 4: Performances of the three algorithms on the test video with $p=0.04$. The first column shows one selected frame from the test video and the samples on that frame from a random trail. The second, third and fourth column show the separated background and foreground by ARMC, RMC and RPCA-GD, respectively, along with their averaged runtime and PIQE values over 10 trails.

Theorems & Definitions (16)

  • Remark 2.5
  • Theorem 2.6
  • Theorem 3.1
  • Remark 3.2
  • Lemma 4.1
  • Lemma 4.2
  • Lemma 4.3
  • Lemma 4.4
  • Lemma 4.5
  • Lemma 4.6
  • ...and 6 more