Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Global convergence of the subgradient method for robust signal recovery

Zesheng Cai, Lexiao Lai, Tiansheng Li

TL;DR

This work analyzes the subgradient method for factorized robust signal recovery tasks (robust PCA, robust phase retrieval, robust matrix sensing), which yield nonsmooth, nonconvex objectives with possible unbounded sublevel sets. It develops a global convergence framework for locally Lipschitz semialgebraic objectives, proving that bounded continuous-time subgradient trajectories imply bounded subgradient sequences and convergence to a critical point when step sizes scale as $\alpha/(k+1)$. The framework is validated for robust PCA, phase retrieval, and matrix sensing, with a mild nondegeneracy condition needed for the sensing case. Additionally, in the rank-one symmetric robust PCA setting, the authors show that, for almost every initialization, the subgradient method avoids spurious critical points and converges to a global minimum under small nonsummable step sizes, yielding a practical global convergence guarantee. Overall, the results provide global convergence guarantees and landscape-based guarantees for a broad class of robust recovery problems using simple subgradient dynamics with modest step-size schemes.

Abstract

We study the subgradient method for factorized robust signal recovery problems, including robust PCA, robust phase retrieval, and robust matrix sensing. These objectives are nonsmooth and nonconvex, and may have unbounded sublevel sets, so standard arguments for analyzing first-order optimization algorithms based on descent and coercivity do not apply. For locally Lipschitz semialgebraic objectives, we develop a convergence framework under the assumption that continuous-time subgradient trajectories are bounded: for sufficiently small step sizes of order \(1/k\), any subgradient sequence remains bounded and converges to a critical point. We verify this trajectory boundedness assumption for the robust objectives by adapting and extending existing trajectory analyses, requiring only a mild nondegeneracy condition in the matrix sensing case. Finally, for rank-one symmetric robust PCA, we show that the subgradient method avoids spurious critical points for almost every initialization, and therefore converges to a global minimum under the same step-size regime.

Global convergence of the subgradient method for robust signal recovery

TL;DR

This work analyzes the subgradient method for factorized robust signal recovery tasks (robust PCA, robust phase retrieval, robust matrix sensing), which yield nonsmooth, nonconvex objectives with possible unbounded sublevel sets. It develops a global convergence framework for locally Lipschitz semialgebraic objectives, proving that bounded continuous-time subgradient trajectories imply bounded subgradient sequences and convergence to a critical point when step sizes scale as . The framework is validated for robust PCA, phase retrieval, and matrix sensing, with a mild nondegeneracy condition needed for the sensing case. Additionally, in the rank-one symmetric robust PCA setting, the authors show that, for almost every initialization, the subgradient method avoids spurious critical points and converges to a global minimum under small nonsummable step sizes, yielding a practical global convergence guarantee. Overall, the results provide global convergence guarantees and landscape-based guarantees for a broad class of robust recovery problems using simple subgradient dynamics with modest step-size schemes.

Abstract

We study the subgradient method for factorized robust signal recovery problems, including robust PCA, robust phase retrieval, and robust matrix sensing. These objectives are nonsmooth and nonconvex, and may have unbounded sublevel sets, so standard arguments for analyzing first-order optimization algorithms based on descent and coercivity do not apply. For locally Lipschitz semialgebraic objectives, we develop a convergence framework under the assumption that continuous-time subgradient trajectories are bounded: for sufficiently small step sizes of order , any subgradient sequence remains bounded and converges to a critical point. We verify this trajectory boundedness assumption for the robust objectives by adapting and extending existing trajectory analyses, requiring only a mild nondegeneracy condition in the matrix sensing case. Finally, for rank-one symmetric robust PCA, we show that the subgradient method avoids spurious critical points for almost every initialization, and therefore converges to a global minimum under the same step-size regime.
Paper Structure (11 sections, 14 theorems, 95 equations, 2 figures)

This paper contains 11 sections, 14 theorems, 95 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Notations
  3. From bounded subgradient trajectories to converging subgradient sequences
  4. Avoidance of spurious critical points in rank-one symmetric robust PCA
  5. Proofs of main results
  6. Proof of Theorem \ref{['thm:bounded_diameter']}
  7. Proof of Theorem \ref{['thm:sym_avoid']}
  8. Proof of Proposition \ref{['prop:u_i_zero']}
  9. Omitted proofs
  10. Proof of Proposition \ref{['prop:phase']}
  11. Proof of Proposition \ref{['prop:sensing']}

Key Result

Lemma 2.2

Let $f:\mathbb R^n\to \mathbb R$ be locally Lipschitz semialgebraic. If $x:[0,\infty)\to \mathbb R^n$ is a subgradient trajectory of $f$, then $f\circ x$ is differentiable for almost everywhere on $[0,\infty)$ with

Figures (2)

  • Figure 1: We apply the subgradient method with step size of order $1/k$ to minimize objectives that arise in robust signal recovery problems: the top panel shows the evolution of the objective values, and the bottom panel shows the evolution of the norms of the iterates.
  • Figure 2: Minimizing $f(x):=\tfrac{1}{2}\|xx^\top-uu^\top\|_1$ with $u=(0,1)$ by the subgradient method. Red subgradient sequences are initialized at $(1,0.3)$ (left) and $(1,0.6)$ (right) respectively. Stars mark the global minima $\pm u$.

Theorems & Definitions (22)

  • Example 1.1: Robust principal component analysis
  • Example 1.2: Robust phase retrieval
  • Example 1.3: Robust matrix sensing
  • Definition 2.1
  • Lemma 2.2: davis2020stochastic
  • Proposition 2.3: josz2023certifying
  • Proposition 2.4
  • Proposition 2.5
  • Lemma 2.7: josz2023global
  • Theorem 2.8
  • ...and 12 more