Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deeply Learned Robust Matrix Completion for Large-scale Low-rank Data Recovery

HanQin Cai, Chandra Kundu, Jialin Liu, Wotao Yin

TL;DR

This work tackles large-scale robust matrix completion by formulating LRMC, a scalable non-convex method that writes the target data as a low-rank factor $\bm{L}\bm{R}^\top$ plus a sparse outlier $\bm{S}$ and optimizes with a differentiable objective under partial observations. A key innovation is replacing expensive sparsity steps with learnable soft-thresholding via deep unfolding, together with scaled gradient updates for the low-rank factors; a novel FRMNN model further extends to effectively infinite iterations. The authors prove a recovery guarantee in a special case and show LRMC achieves linear convergence with a per-iteration cost of $\mathcal{O}(p n^2 r + n r^2)$, independent of outlier sparsity, while enabling fast inference on real tasks such as video background subtraction, ultrasound imaging, face modeling, and cloud removal. Empirically, LRMC outperforms state-of-the-art methods in speed and robustness across synthetic and diverse real datasets, highlighting its scalability and practical impact for large-scale robust data recovery.

Abstract

Robust matrix completion (RMC) is a widely used machine learning tool that simultaneously tackles two critical issues in low-rank data analysis: missing data entries and extreme outliers. This paper proposes a novel scalable and learnable non-convex approach, coined Learned Robust Matrix Completion (LRMC), for large-scale RMC problems. LRMC enjoys low computational complexity with linear convergence. Motivated by the proposed theorem, the free parameters of LRMC can be effectively learned via deep unfolding to achieve optimum performance. Furthermore, this paper proposes a flexible feedforward-recurrent-mixed neural network framework that extends deep unfolding from fix-number iterations to infinite iterations. The superior empirical performance of LRMC is verified with extensive experiments against state-of-the-art on synthetic datasets and real applications, including video background subtraction, ultrasound imaging, face modeling, and cloud removal from satellite imagery.

Deeply Learned Robust Matrix Completion for Large-scale Low-rank Data Recovery

TL;DR

This work tackles large-scale robust matrix completion by formulating LRMC, a scalable non-convex method that writes the target data as a low-rank factor plus a sparse outlier and optimizes with a differentiable objective under partial observations. A key innovation is replacing expensive sparsity steps with learnable soft-thresholding via deep unfolding, together with scaled gradient updates for the low-rank factors; a novel FRMNN model further extends to effectively infinite iterations. The authors prove a recovery guarantee in a special case and show LRMC achieves linear convergence with a per-iteration cost of , independent of outlier sparsity, while enabling fast inference on real tasks such as video background subtraction, ultrasound imaging, face modeling, and cloud removal. Empirically, LRMC outperforms state-of-the-art methods in speed and robustness across synthetic and diverse real datasets, highlighting its scalability and practical impact for large-scale robust data recovery.

Abstract

Robust matrix completion (RMC) is a widely used machine learning tool that simultaneously tackles two critical issues in low-rank data analysis: missing data entries and extreme outliers. This paper proposes a novel scalable and learnable non-convex approach, coined Learned Robust Matrix Completion (LRMC), for large-scale RMC problems. LRMC enjoys low computational complexity with linear convergence. Motivated by the proposed theorem, the free parameters of LRMC can be effectively learned via deep unfolding to achieve optimum performance. Furthermore, this paper proposes a flexible feedforward-recurrent-mixed neural network framework that extends deep unfolding from fix-number iterations to infinite iterations. The superior empirical performance of LRMC is verified with extensive experiments against state-of-the-art on synthetic datasets and real applications, including video background subtraction, ultrasound imaging, face modeling, and cloud removal from satellite imagery.
Paper Structure (18 sections, 11 theorems, 62 equations, 7 figures, 6 tables, 1 algorithm)

This paper contains 18 sections, 11 theorems, 62 equations, 7 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work and main contributions
  3. Notation
  4. Proposed Approach
  5. Iterative updates
  6. Initialization
  7. Computational complexity
  8. Theoretical results
  9. Parameter learning
  10. Numerical experiments
  11. Synthetic datasets
  12. Real datasets
  13. Proofs
  14. Proof of Theorem \ref{['thm:main_theorem']}
  15. Auxiliary lemmas
  16. ...and 3 more sections

Key Result

Theorem 2

Suppose that $\bm{X}_\star$ is a rank-$r$ matrix with $\mu$-incoherence and $\bm{S}_\star$ is an $\alpha$-sparse matrix with $\alpha\leq\frac{1}{10^4\mu r^{3/2}\kappa}$. Let $p=1$, i.e., $\Omega=[n_1]\times[n_2]$. If we set the thresholding values $\zeta_0=\|\bm{X}_\star\|_\infty$ and $\zeta_k=\|{\b with the step sizes $\eta_k=\eta\in[\frac{1}{4},\frac{8}{9}]$.

Figures (7)

  • Figure 1: A high-level structure comparison between classic FNN-based deep unfolding (top) and proposed FRMNN-based deep unfolding (bottom). In the diagrams, $\mathcal{L}_k$ denotes the $k$-th layer of FNN and $\overline{\mathcal{L}}$ is a layer of RNN.
  • Figure 2: Convergence comparison for FNN-based, RNN-based, and FRMNN-based learning.
  • Figure 3: Convergence comparison for LRMC and ScaledGD with varying outlier sparsity $\alpha$. Problem dimension $n=3000$ and rank $r=5$.
  • Figure 4: Runtime comparison for LRMC and ScaledGD for varying outlier sparsity $\alpha$. Problem dimension $n=3000$ and rank $r=5$. Left: Single step runtime averaged over 100 iterations, excluding initialization. Right: Total runtime. All algorithms halt when $\|\bm{X} - \bm{X_\star}\|_\text{F} / \|\bm{X_\star}\|_\text{F} < 10^{-6}$.
  • Figure 5: Visual results for video background subtraction. The first column is the original frames. The second and third columns are the foreground and background output by LRMC with $p = 10\%$. The fourth and fifth columns are the outputs by LRMC with $p = 100\%$.
  • ...and 2 more figures

Theorems & Definitions (24)

  • Remark 1
  • Theorem 2: Guaranteed recovery
  • proof
  • Theorem 1: Local linear convergence
  • Theorem 2: Guaranteed initialization
  • Lemma 3
  • proof
  • proof : Proof of Theorem \ref{['thm:main_theorem']}
  • Lemma 4
  • proof
  • ...and 14 more