Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leave-One-Out Analysis for Nonconvex Robust Matrix Completion with General Thresholding Functions

Tianming Wang, Ke Wei

TL;DR

This work tackles robust matrix completion where a low-rank matrix is partially observed and corrupted by sparse outliers. It develops a simple nonconvex algorithm that alternates a projected SVP step for the low-rank part with a general thresholding step for the outliers, and analyzes it via a novel leave-one-out framework. The authors prove linear convergence for a broad class of thresholding functions (including soft-thresholding and SCAD) without relying on sample splitting or explicit incoherence projections, and they improve the noiseless matrix completion sample complexity compared to prior results. The results advance theoretical guarantees for nonconvex robust completion and have practical implications for scalable recovery with sparse corruptions, under standard incoherence and random observation assumptions.

Abstract

We study the problem of robust matrix completion (RMC), where the partially observed entries of an underlying low-rank matrix is corrupted by sparse noise. Existing analysis of the non-convex methods for this problem either requires the explicit but empirically redundant regularization in the algorithm or requires sample splitting in the analysis. In this paper, we consider a simple yet efficient nonconvex method which alternates between a projected gradient step for the low-rank part and a thresholding step for the sparse noise part. Inspired by leave-one out analysis for low rank matrix completion, it is established that the method can achieve linear convergence for a general class of thresholding functions, including for example soft-thresholding and SCAD. To the best of our knowledge, this is the first leave-one-out analysis on a nonconvex method for RMC. Additionally, when applying our result to low rank matrix completion, it improves the sampling complexity of existing result for the singular value projection method.

Leave-One-Out Analysis for Nonconvex Robust Matrix Completion with General Thresholding Functions

TL;DR

This work tackles robust matrix completion where a low-rank matrix is partially observed and corrupted by sparse outliers. It develops a simple nonconvex algorithm that alternates a projected SVP step for the low-rank part with a general thresholding step for the outliers, and analyzes it via a novel leave-one-out framework. The authors prove linear convergence for a broad class of thresholding functions (including soft-thresholding and SCAD) without relying on sample splitting or explicit incoherence projections, and they improve the noiseless matrix completion sample complexity compared to prior results. The results advance theoretical guarantees for nonconvex robust completion and have practical implications for scalable recovery with sparse corruptions, under standard incoherence and random observation assumptions.

Abstract

We study the problem of robust matrix completion (RMC), where the partially observed entries of an underlying low-rank matrix is corrupted by sparse noise. Existing analysis of the non-convex methods for this problem either requires the explicit but empirically redundant regularization in the algorithm or requires sample splitting in the analysis. In this paper, we consider a simple yet efficient nonconvex method which alternates between a projected gradient step for the low-rank part and a thresholding step for the sparse noise part. Inspired by leave-one out analysis for low rank matrix completion, it is established that the method can achieve linear convergence for a general class of thresholding functions, including for example soft-thresholding and SCAD. To the best of our knowledge, this is the first leave-one-out analysis on a nonconvex method for RMC. Additionally, when applying our result to low rank matrix completion, it improves the sampling complexity of existing result for the singular value projection method.
Paper Structure (14 sections, 16 theorems, 181 equations, 2 figures, 1 algorithm)

This paper contains 14 sections, 16 theorems, 181 equations, 2 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Main Contributions
  3. Assumptions and Notations
  4. Nonconvex RMC with General Thresholding Functions
  5. Algorithm and Main Result
  6. Numerical Experiments
  7. Proof Outline
  8. Initialization and Base Case
  9. Induction Steps
  10. Conclusion
  11. Useful Lemmas and Supplementary Proofs
  12. Useful Lemmas
  13. Proof for Lemma \ref{['lem:L_infinity']}
  14. Proof for Lemma \ref{['lem:thresh']}

Key Result

Theorem 1

Suppose that Alg. Alg1 is performed with a thresholding function satisfying properties P1P2P3, and set $\frac{\mu r}{n}\sigma^{\star}_1\leq\beta\leq C_{\emph{init}}\cdot\frac{\mu r}{n}\sigma^{\star}_1$ for some constant $C_{\emph{init}}\geq 1$. Let $C_{\emph{thresh}}:=(K+B)\cdot C_{\emph{init}}$. If for some sufficiently large constant $C_{\emph{sample}}>0$, and some sufficiently small constant $c

Figures (2)

  • Figure 1: Empirical phase transition comparisons for RMC-SOFT and RMC-SCAD: rank $r$ vs. outlier probability $\alpha$. Tests are conducted for fixed $n=1000$, $p=0.3$, $r\in\{1,\cdots,25\}$, and $\alpha\in\{0.025,0.05,\cdots,0.625\}$. The white color corresponds to success with probability 1, and the black color corresponds to success with probability 0.
  • Figure 2: Empirical phase transition comparisons for RMC-SOFT, RMC-SCAD, R-RMC, and RPCA-GD: sample probability $p$ vs. rank $r$.

Theorems & Definitions (22)

  • Theorem 1
  • Lemma 1
  • Lemma 2
  • Theorem 2
  • proof : Proof of Theorem \ref{['thm:noiseless']}
  • Lemma 3
  • proof
  • Lemma 4: Chen2015
  • Lemma 5: Ding2020
  • Lemma 6: Ma2019
  • ...and 12 more