Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Low-rank Identification via Accelerated Iteratively Reweighted Nuclear Norm Minimization

Hao Wang, Ye Wang, Xiangyu Yang

TL;DR

The paper tackles nonconvex low-rank matrix optimization via the Schatten-$p$ norm regularization by proposing Extrapolated Iteratively Reweighted Nuclear Norm with Rank Identification (EIRNRI). The method combines extrapolation, adaptive per-singular-value smoothing, and a weighted nuclear-norm surrogate to identify the solution rank in finite iterations and reduce to a smooth problem on a low-rank manifold, with proven global convergence and KL-based local rates. Its main contributions include a rank-identification property, an adaptive perturbation strategy that preserves ascending weights and enables a closed-form subproblem solution, and strong empirical performance on synthetic and real data. This approach offers a scalable and theoretically grounded framework for nonconvex low-rank optimization with automatic rank recovery and accelerated convergence, enhancing practical applicability in matrix completion and related tasks.

Abstract

This paper considers the problem of minimizing the sum of a smooth function and the Schatten-$p$ norm of the matrix. Our contribution involves proposing accelerated iteratively reweighted nuclear norm methods designed for solving the nonconvex low-rank minimization problem. Two major novelties characterize our approach. Firstly, the proposed method possesses a rank identification property, enabling the provable identification of the "correct" rank of the stationary point within a finite number of iterations. Secondly, we introduce an adaptive updating strategy for smoothing parameters. This strategy automatically fixes parameters associated with zero singular values as constants upon detecting the "correct" rank while quickly driving the rest of the parameters to zero. This adaptive behavior transforms the algorithm into one that effectively solves smooth problems after a few iterations, setting our work apart from existing iteratively reweighted methods for low-rank optimization. We prove the global convergence of the proposed algorithm, guaranteeing that every limit point of the iterates is a critical point. Furthermore, a local convergence rate analysis is provided under the Kurdyka-Łojasiewicz property. We conduct numerical experiments using both synthetic and real data to showcase our algorithm's efficiency and superiority over existing methods.

Efficient Low-rank Identification via Accelerated Iteratively Reweighted Nuclear Norm Minimization

TL;DR

The paper tackles nonconvex low-rank matrix optimization via the Schatten- norm regularization by proposing Extrapolated Iteratively Reweighted Nuclear Norm with Rank Identification (EIRNRI). The method combines extrapolation, adaptive per-singular-value smoothing, and a weighted nuclear-norm surrogate to identify the solution rank in finite iterations and reduce to a smooth problem on a low-rank manifold, with proven global convergence and KL-based local rates. Its main contributions include a rank-identification property, an adaptive perturbation strategy that preserves ascending weights and enables a closed-form subproblem solution, and strong empirical performance on synthetic and real data. This approach offers a scalable and theoretically grounded framework for nonconvex low-rank optimization with automatic rank recovery and accelerated convergence, enhancing practical applicability in matrix completion and related tasks.

Abstract

This paper considers the problem of minimizing the sum of a smooth function and the Schatten- norm of the matrix. Our contribution involves proposing accelerated iteratively reweighted nuclear norm methods designed for solving the nonconvex low-rank minimization problem. Two major novelties characterize our approach. Firstly, the proposed method possesses a rank identification property, enabling the provable identification of the "correct" rank of the stationary point within a finite number of iterations. Secondly, we introduce an adaptive updating strategy for smoothing parameters. This strategy automatically fixes parameters associated with zero singular values as constants upon detecting the "correct" rank while quickly driving the rest of the parameters to zero. This adaptive behavior transforms the algorithm into one that effectively solves smooth problems after a few iterations, setting our work apart from existing iteratively reweighted methods for low-rank optimization. We prove the global convergence of the proposed algorithm, guaranteeing that every limit point of the iterates is a critical point. Furthermore, a local convergence rate analysis is provided under the Kurdyka-Łojasiewicz property. We conduct numerical experiments using both synthetic and real data to showcase our algorithm's efficiency and superiority over existing methods.
Paper Structure (17 sections, 13 theorems, 108 equations, 6 figures, 3 tables, 2 algorithms)

This paper contains 17 sections, 13 theorems, 108 equations, 6 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contribution
  4. Notation and Preliminaries
  5. Proposed Extrapolated Iteratively Reweighed Nuclear Norm Algorithm with Rank Identification
  6. A Weighted Nuclear Norm Surrogate with Extrapolation
  7. Subproblem Solution
  8. An Adaptive Updating Strategy for Perturbation e
  9. Convergence Analysis
  10. Local Properties: Stable Support and Adaptively Reweighting
  11. Rank Identification
  12. Global Convergence
  13. Convergence Analysis Under KL Property
  14. Numerical Experiments
  15. Synthetic Data
  16. ...and 2 more sections

Key Result

Lemma 1.4

Let $\varphi:\mathbb{R}^{n}\to\mathbb{R}$ be an absolutely symmetric function, meaning $\varphi (x_{1},\cdots,x_{n}) = \varphi(|x_{\pi (1)}|,\cdots,|x_{\pi(n)}|)$ holds for any permutation $\pi$ of $[n]$, and let $\bm{\sigma}(X)$ be the singular values of a matrix $X\in\mathbb{R}^{m\times n}$($n \le with $U\mathrm{diag} (\bm{\sigma} (X))V^{\top}$ being the SVD of $X$.

Figures (6)

  • Figure 1: A sample example to show the rank-identification property.
  • Figure 2: The number of problems that achieve model identification for different $r^{*}$. For each $r^{*}$, 2000 problems are generated and set $\lambda = 10^{-1}\|X^{*}\|_{\infty}$ for those problems. We set different $\epsilon = 10^{-3}$ for PIRNN and the default values for our algorithm.
  • Figure 3: Comparison of matrix recovery on synthetic data with initial point $X^{0}$ satisfies ${\mathrm{Rank}}(X^{0}) = r^*$, initial parameters satisfy $\epsilon_{i}^{0} = \epsilon = 10^{-3}$.
  • Figure 4: The performance of EIRNRI with different $\alpha$ for three kinds of problems.
  • Figure 5: The performance of different methods in image recovery with a random mask(SR= $0.8$). (a) Original image, ${\mathrm{Rank}}(X) = 300$; (b) Low-rank image, ${\mathrm{Rank}}(X^{*}) = 30$; (c) Noised picture; (d) PIRNN: ${\mathrm{Rank}}(X) = 30$, PSNR=$28.855$; (e) AIRNN: ${\mathrm{Rank}}(X) = 36$, PSNR=$28.854$; (f) EIRNRI: ${\mathrm{Rank}}(X) = 30$, PSNR=$28.855$; (g) Sc$p$: ${\mathrm{Rank}}(X) = 187$, PSNR=$28.971$; (h) FGSR$p$: ${\mathrm{Rank}}(X) = 187$, PSNR=$21.806$;
  • ...and 1 more figures

Theorems & Definitions (33)

  • Definition 1.1: Rank identification property
  • Definition 1.2: Simultaneous ordered SVD
  • Definition 1.3: Subdifferentials
  • Lemma 1.4: Limiting subdifferential of singular value function
  • Proposition 1.5
  • Theorem 1.6: Nonsmooth versions of Fermat's rule
  • Definition 1.7: Critical point
  • Definition 1.8: Desingularizing function
  • Definition 1.9: KL property
  • Definition 1.10: Uniform KL property
  • ...and 23 more