Guarantees of a Preconditioned Subgradient Algorithm for Overparameterized Asymmetric Low-rank Matrix Recovery

TL;DR

The paper tackles robust recovery of asymmetric low-rank matrices from corrupted measurements by factorizing ${\bm{X}}={\bm{L}}{\bm{R}}^\top$ with overparameterization ($d\ge r$) and proposing the Overparameterized Preconditioned Subgradient Algorithm (OPSA) for non-smooth losses such as $\ell_1$. It develops a specialized distance metric with alignment, establishes linear convergence under rank-$d$ restricted Lipschitz continuity and sharpness, and derives iteration-complexity results for matrix sensing in both noiseless and outlier-settings, with rates independent of the true condition number. The work includes rigorous theoretical results and extensive experiments showing robust performance under varying overparameterization levels, outlier densities, and condition numbers, highlighting OPSA’s practical viability for robust, scalable low-rank recovery. Overall, OPSA advances non-convex, overparameterized, preconditioned methods to asymmetric, unknown-rank settings with non-smooth losses, supported by provable convergence and empirical validation, including robust matrix sensing and video background subtraction demos.

Abstract

In this paper, we focus on a matrix factorization-based approach to recover low-rank {\it asymmetric} matrices from corrupted measurements. We propose an {\it Overparameterized Preconditioned Subgradient Algorithm (OPSA)} and provide, for the first time in the literature, linear convergence rates independent of the rank of the sought asymmetric matrix in the presence of gross corruptions. Our work goes beyond existing results in preconditioned-type approaches addressing their current limitation, i.e., the lack of convergence guarantees in the case of {\it asymmetric matrices of unknown rank}. By applying our approach to (robust) matrix sensing, we highlight its merits when the measurement operator satisfies a mixed-norm restricted isometry property. Lastly, we present extensive numerical experiments that validate our theoretical results and demonstrate the effectiveness of our approach for different levels of overparameterization and outlier corruptions.

Figures (9)

• Figure 1: Values of $\frac{\|\mathcal{A}({\bm{X}})\|_1}{\|{\bm{X}}\|_F}$ for Gaussian map $\mathcal{A}:\mathbb{R}^{500\times500}\rightarrow\mathbb{R}^{5000}$, where $\mathrm{rank}({\bm{X}})=10$. Each point represents one result of 500 random trials. Blue and red dash lines are the lower and upper uniform bounds, respectively.
• Figure 2: Performance comparison between OPSA (top) and ScaledSM (bottom) with different overparameterization $d$, where $n,r,\kappa,\lambda,\textnormal{outlier} = 100,5,20,2,10\%$.
• Figure 3: Performance comparison between OPSA (top) and ScaledSM (bottom) with different overparameterization $d$, where $n,r,\kappa,\lambda,\textnormal{outlier} = 100,10,20,2,10\%$.
• Figure 4: OPSA performance with different condition numbers $\kappa$, where $n,\lambda,\textnormal{outlier}=100,2,10\%$. Top: $r,d=5,10$. Bottom: $r,d=10,20$.
• Figure 5: OPSA performance with different $\lambda$, where $n,\kappa,\textnormal{outlier}=100,20,10\%$. Top: $r,d=5,10$. Bottom: $r,d=10,20$.

Theorems & Definitions (20)

• Remark 4.1
• Lemma 5.3
• Theorem 5.4: Convergence of OPSA
• proof
• Remark 5.5
• Remark 5.6
• Remark 5.7
• Definition 5.8: Mixed-norm RIP
• Proposition 5.9: Lipschitz Continuity and Restricted Sharpness--No Outliers
• Corollary 5.10: Iteration Complexity--No Outliers