Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust principal component analysis with rank and cardinality regularization under matrix factorization

Wenjing Li, Wei Bian, Kim-Chuan Toh

TL;DR

A broadly applicable equivalent nonconvex relaxation framework for the constrained factorization model in the sense of global minimizers and stationary points with strong optimality conditions (called strong stationary points) is constructed.

Abstract

Robust principal component analysis is an important representative method in data analysis. It is usually viewed as an optimization problem involving the rank and $\ell_0$-norm of matrices. In this paper, we study the rank and $\ell_0$ regularized optimization problem and its matrix factorization problem. We establish their equivalences on global minimizers and stationary points, respectively. Furthermore, we construct a broadly applicable equivalent nonconvex relaxation framework for the constrained factorization model in the sense of global minimizers and stationary points with strong optimality conditions (called strong stationary points). For the general factorization problem with lower semicontinuous regularizers and a loss function whose gradient is locally Lipschitz, we propose a novel proximal gradient-based algorithm based on joint and alternating calculation with convergence to its limiting-critical points. The algorithm can attain the stationary points of the original problem and its adaptive counterpart can attain the strong stationary points of the factorization problem.

Robust principal component analysis with rank and cardinality regularization under matrix factorization

TL;DR

A broadly applicable equivalent nonconvex relaxation framework for the constrained factorization model in the sense of global minimizers and stationary points with strong optimality conditions (called strong stationary points) is constructed.

Abstract

Robust principal component analysis is an important representative method in data analysis. It is usually viewed as an optimization problem involving the rank and -norm of matrices. In this paper, we study the rank and regularized optimization problem and its matrix factorization problem. We establish their equivalences on global minimizers and stationary points, respectively. Furthermore, we construct a broadly applicable equivalent nonconvex relaxation framework for the constrained factorization model in the sense of global minimizers and stationary points with strong optimality conditions (called strong stationary points). For the general factorization problem with lower semicontinuous regularizers and a loss function whose gradient is locally Lipschitz, we propose a novel proximal gradient-based algorithm based on joint and alternating calculation with convergence to its limiting-critical points. The algorithm can attain the stationary points of the original problem and its adaptive counterpart can attain the strong stationary points of the factorization problem.
Paper Structure (10 sections, 16 theorems, 61 equations, 4 figures, 2 tables, 4 algorithms)

This paper contains 10 sections, 16 theorems, 61 equations, 4 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Equivalences and properties of problems (\ref{['m_s_l']}), (\ref{['s_l_nc']}) and (\ref{['s_l']})
  3. Equivalent continuous relaxations of problem (\ref{['s_l']})
  4. Joint alternating proximal gradient algorithm
  5. General factorization case
  6. Specific factorization case
  7. Numerical experiments
  8. Synthetic data
  9. Real data
  10. Conclusions

Key Result

Proposition 1

Problems (m_s_l) and (s_l_nc) own the same optimal function value and their global minimizers can be transformed into each other, i.e.

Figures (4)

  • Figure 1: Relations among problems (\ref{['s_l_nc']}), (\ref{['m_s_l']}), (\ref{['s_l']}) and (\ref{['m_s_l_b']}), where 'OP-value' means 'optimal value', and '(Cor-)GM-set' means 'the set of (corresponding matrix products of) global minimizers'.
  • Figure 2: Relations between problems (\ref{['m_s_l']}) and (\ref{['s_l_nc']}).
  • Figure 3: Function curve diagrams for $\theta(\cdot)$ satisfying (\ref{['theta_cond']}) and its scaled forms $\theta(\cdot/0.5)$ and $\theta(\cdot/1.5)$, where the red, blue and green curves represent three applicable alternatives to $\theta$ when $t\in(0,1)$.
  • Figure 4: Relation diagram of the optimality conditions for problems (\ref{['s_l']}) and (\ref{['s_l_r']}), where 'GM/LM-set' means 'the set of global/local minimizers', '(S-)Stat-set' means 'the set of (strong-)stationary points', and 'GM', 'OP-value', 'OP-con' and 'S-Stat' means 'global minimizer', 'optimal value', 'optimal condition' and 'strong stationary point', respectively.

Theorems & Definitions (38)

  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Proposition 3
  • proof
  • Proposition 4
  • proof
  • Definition 1
  • Theorem 1
  • ...and 28 more