Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence of the majorized PAM method with subspace correction for low-rank composite factorization model

Ting Tao, Yitian Qian, Shaohua Pan

TL;DR

This work develops and analyzes a subspace-corrected majorized proximal alternating minimization method (MPAMSC) for low-rank composite factorization with column sparsity penalties. By majorizing the smooth part and applying subspace corrections to ensure closed-form proximal subproblems, the authors prove subsequence convergence under the KL property and full convergence under a mild proximal-assumption framework, with implications similar to softImpute-ALS. Theoretical results are complemented by numerical experiments on one-bit matrix completion, where MPAMSC demonstrates competitive accuracy and efficiency against PALM variants and a Hybrid AMM approach. The contributions yield a first convergence certificate for a subspace-corrected AM method in this class of problems, offering a principled and robust tool for matrix completion and related factorization tasks.

Abstract

This paper focuses on the convergence certificates of the majorized proximal alternating minimization (PAM) method with subspace correction, proposed in \cite{TaoQianPan22} for the column $\ell_{2,0}$-norm regularized factorization model and now extended to a class of low-rank composite factorization models from matrix completion. The convergence analysis of this PAM method becomes extremely challenging because a subspace correction step is introduced to every proximal subproblem to ensure a closed-form solution. We establish the full convergence of the iterate sequence and column subspace sequences of factor pairs generated by the PAM, under the KL property of the objective function and a condition that holds automatically for the column $\ell_{2,0}$-norm function. Numerical comparison with the popular proximal alternating linearized minimization (PALM) method is conducted on one-bit matrix completion problems, which indicates that the PAM with subspace correction has an advantage in seeking lower relative error within less time.

Convergence of the majorized PAM method with subspace correction for low-rank composite factorization model

TL;DR

This work develops and analyzes a subspace-corrected majorized proximal alternating minimization method (MPAMSC) for low-rank composite factorization with column sparsity penalties. By majorizing the smooth part and applying subspace corrections to ensure closed-form proximal subproblems, the authors prove subsequence convergence under the KL property and full convergence under a mild proximal-assumption framework, with implications similar to softImpute-ALS. Theoretical results are complemented by numerical experiments on one-bit matrix completion, where MPAMSC demonstrates competitive accuracy and efficiency against PALM variants and a Hybrid AMM approach. The contributions yield a first convergence certificate for a subspace-corrected AM method in this class of problems, offering a principled and robust tool for matrix completion and related factorization tasks.

Abstract

This paper focuses on the convergence certificates of the majorized proximal alternating minimization (PAM) method with subspace correction, proposed in \cite{TaoQianPan22} for the column -norm regularized factorization model and now extended to a class of low-rank composite factorization models from matrix completion. The convergence analysis of this PAM method becomes extremely challenging because a subspace correction step is introduced to every proximal subproblem to ensure a closed-form solution. We establish the full convergence of the iterate sequence and column subspace sequences of factor pairs generated by the PAM, under the KL property of the objective function and a condition that holds automatically for the column -norm function. Numerical comparison with the popular proximal alternating linearized minimization (PALM) method is conducted on one-bit matrix completion problems, which indicates that the PAM with subspace correction has an advantage in seeking lower relative error within less time.
Paper Structure (20 sections, 17 theorems, 93 equations, 5 figures, 2 tables, 2 algorithms)

This paper contains 20 sections, 17 theorems, 93 equations, 5 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Main contribution
  4. Notation
  5. Preliminaries
  6. Stationary points of problem \ref{['prob']}
  7. Relation between column $\ell_{2,p}$-norm and Schatten $p$-norm
  8. Kurdyka-Łojasiewicz property
  9. A majorized PAM with subspace correction
  10. Convergence analysis of Algorithm \ref{['MPAMSC']}
  11. Subsequence convergence
  12. Full convergence of iterate sequence
  13. Numerical experiments
  14. One-bit matrix completions with noise
  15. Implementation details of Algorithm \ref{['MPAMSC']}
  16. ...and 5 more sections

Key Result

Lemma 1

Fix any $\gamma>0$ and $u\in\mathbb{R}^n$. Let $S^*$ be the solution set of the problem Then, $S^*=\{0\}$ when $u=0$; otherwise $S^*=\frac{u}{\|u\|}\mathcal{P}_{\!\lambda/\gamma^2}\theta(\|u\|/\gamma)$.

Figures (5)

  • Figure 1: The number of iterations and running time of the two methods with $\theta=\theta_1$
  • Figure 2: Curves of objective value and running time for the three solvers under Case I with different $\theta$
  • Figure 3: Curves of objective value and running time for the three solvers under Case II with different $\theta$
  • Figure 4: Curves of relative error and rank of the three solvers under Case I with $\theta=\theta_1$
  • Figure 5: Curves of relative error and rank of the three solvers under Case II with $\theta=\theta_4$ and $p=\frac{1}{2}$

Theorems & Definitions (40)

  • Lemma 1
  • Definition 1
  • Lemma 2
  • proof
  • Definition 2
  • Lemma 3
  • proof
  • Proposition 1
  • proof
  • Definition 3
  • ...and 30 more