Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Certifying optimality in nonconvex robust PCA

Pinxi Gong, Lexiao Lai, Jianhao Ma

TL;DR

The paper addresses robust PCA via a scalable, nonsmooth factorization with the objective f(X,Y)=\|XY^\top - M\|_1 for M=L+S, under incoherence and random sparse corruption. It proves that all ground-truth rank-$r$ factorizations with k≥r are Clarke critical with high probability, and that the geometry is a sharp local minimum when k=r but a strict saddle when k>r. The core technique reduces first-order optimality to a restricted $\ell_1$ operator-norm bound on the corruption projector, established through a detailed empirical-process and chaining analysis. These results explain why simple first-order methods can converge rapidly in practice and differentiate between exact and overparameterized regimes. Potential extensions include robust matrix completion and deeper investigation into the activity of strict saddles.

Abstract

Robust principal component analysis seeks to recover a low-rank matrix from fully observed data with sparse corruptions. A scalable approach fits a low-rank factorization by minimizing the sum of entrywise absolute residuals, leading to a nonsmooth and nonconvex objective. Under standard incoherence conditions and a random model for the corruption support, we study factorizations of the ground-truth rank-$r$ matrix with both factors of rank $r$. With high probability, every such factorization is a Clarke critical point. We also characterize the local geometry: when the factorization rank equals $r$, these solutions are sharp local minima; when it exceeds $r$, they are strict saddle points.

Certifying optimality in nonconvex robust PCA

TL;DR

The paper addresses robust PCA via a scalable, nonsmooth factorization with the objective f(X,Y)=\|XY^\top - M\|_1 for M=L+S, under incoherence and random sparse corruption. It proves that all ground-truth rank- factorizations with k≥r are Clarke critical with high probability, and that the geometry is a sharp local minimum when k=r but a strict saddle when k>r. The core technique reduces first-order optimality to a restricted operator-norm bound on the corruption projector, established through a detailed empirical-process and chaining analysis. These results explain why simple first-order methods can converge rapidly in practice and differentiate between exact and overparameterized regimes. Potential extensions include robust matrix completion and deeper investigation into the activity of strict saddles.

Abstract

Robust principal component analysis seeks to recover a low-rank matrix from fully observed data with sparse corruptions. A scalable approach fits a low-rank factorization by minimizing the sum of entrywise absolute residuals, leading to a nonsmooth and nonconvex objective. Under standard incoherence conditions and a random model for the corruption support, we study factorizations of the ground-truth rank- matrix with both factors of rank . With high probability, every such factorization is a Clarke critical point. We also characterize the local geometry: when the factorization rank equals , these solutions are sharp local minima; when it exceeds , they are strict saddle points.
Paper Structure (9 sections, 13 theorems, 119 equations, 2 figures, 1 table)

This paper contains 9 sections, 13 theorems, 119 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Empirical motivation: convergence and scalability
  3. Main results
  4. Proof of \ref{['thm:main']}
  5. Certifying first-order optimality
  6. Proof of \ref{['thm::first-order-optimality']}
  7. Determination of second-order optimality
  8. Conclusion and future directions
  9. Auxiliary lemmas

Key Result

Theorem 1

Let Assumption assumption:standing hold with where $c>0$ is a sufficiently small universal constant. Then there exists a universal constant $C>0$ such that, with probability at least $1-\exp\{-Cr\log^2(mn)\}$, any $(X^{\star},Y^{\star})\in \mathcal{L}$ is a (Clarke) critical point of $f$. Moreover, it is a local minimum when $k = r$, and a str

Figures (2)

  • Figure 1: Convergence of the subgradient method with small initialization for solving Equation \ref{['eq:obj']}. Each entry of $S$ is nonzero with probability $p = 0.1$. In all experiments, we adopt a robust exponentially decaying step-size schedule: the step size is reduced by a factor of $0.5$ whenever the objective value fails to decrease for $10$ consecutive iterations.
  • Figure 2: Recovery success-rate heatmaps (in %) for robust PCA on synthetic low-rank plus sparse matrices. The matrices are of size $100$ by $80$. Each grid point averages over $100$ independent trials; The success criterion is $\|L_{\mathrm{est}}-L\|_2/\|L\|_2 \le 10^{-3}$.

Theorems & Definitions (23)

  • Theorem 1
  • Remark 1: On global optimality of true solutions
  • Lemma 1
  • proof
  • Theorem 2
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • Lemma 4
  • ...and 13 more