Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Low solution rank of the matrix LASSO under RIP with consequences for rank-constrained algorithms

Andrew D. McRae

TL;DR

This work proves that nuclear-norm regularized matrix LASSO solutions under RIP have low rank, with a concrete bound tying the solution rank to the ground-truth rank. It then shows that low-rank constrained methods, including a projected proximal gradient descent and a Burer-Monteiro factorization, not only converge but also possess benign global landscapes under the same RIP regime, yielding global optimality from broad initializations. The results connect convex recovery guarantees to efficient nonconvex algorithms, enabling practical, scalable rank-aware procedures for matrix sensing with noisy measurements. The theoretical contributions advance our understanding of when low-rank structure emerges in LASSO solutions and how to leverage it algorithmically for reliable recovery.

Abstract

We show that solutions to the popular convex matrix LASSO problem (nuclear-norm--penalized linear least-squares) have low rank under similar assumptions as required by classical low-rank matrix sensing error bounds. Although the purpose of the nuclear norm penalty is to promote low solution rank, a proof has not yet (to our knowledge) been provided outside very specific circumstances. Furthermore, we show that this result has significant theoretical consequences for nonconvex rank-constrained optimization approaches. Specifically, we show that if (a) the ground truth matrix has low rank, (b) the (linear) measurement operator has the matrix restricted isometry property (RIP), and (c) the measurement error is small enough relative to the nuclear norm penalty, then the (unique) LASSO solution has rank (approximately) bounded by that of the ground truth. From this, we show (a) that a low-rank--projected proximal gradient descent algorithm will converge linearly to the LASSO solution from any initialization, and (b) that the nonconvex landscape of the low-rank Burer-Monteiro--factored problem formulation is benign in the sense that all second-order critical points are globally optimal and yield the LASSO solution.

Low solution rank of the matrix LASSO under RIP with consequences for rank-constrained algorithms

TL;DR

This work proves that nuclear-norm regularized matrix LASSO solutions under RIP have low rank, with a concrete bound tying the solution rank to the ground-truth rank. It then shows that low-rank constrained methods, including a projected proximal gradient descent and a Burer-Monteiro factorization, not only converge but also possess benign global landscapes under the same RIP regime, yielding global optimality from broad initializations. The results connect convex recovery guarantees to efficient nonconvex algorithms, enabling practical, scalable rank-aware procedures for matrix sensing with noisy measurements. The theoretical contributions advance our understanding of when low-rank structure emerges in LASSO solutions and how to leverage it algorithmically for reliable recovery.

Abstract

We show that solutions to the popular convex matrix LASSO problem (nuclear-norm--penalized linear least-squares) have low rank under similar assumptions as required by classical low-rank matrix sensing error bounds. Although the purpose of the nuclear norm penalty is to promote low solution rank, a proof has not yet (to our knowledge) been provided outside very specific circumstances. Furthermore, we show that this result has significant theoretical consequences for nonconvex rank-constrained optimization approaches. Specifically, we show that if (a) the ground truth matrix has low rank, (b) the (linear) measurement operator has the matrix restricted isometry property (RIP), and (c) the measurement error is small enough relative to the nuclear norm penalty, then the (unique) LASSO solution has rank (approximately) bounded by that of the ground truth. From this, we show (a) that a low-rank--projected proximal gradient descent algorithm will converge linearly to the LASSO solution from any initialization, and (b) that the nonconvex landscape of the low-rank Burer-Monteiro--factored problem formulation is benign in the sense that all second-order critical points are globally optimal and yield the LASSO solution.
Paper Structure (19 sections, 8 theorems, 77 equations)

This paper contains 19 sections, 8 theorems, 77 equations.

Table of Contents

  1. Introduction and main result
  2. Guarantees for nonconvex rank-constrained optimization
  3. Projected spectral algorithm
  4. Burer-Monteiro factorization
  5. Combining the results
  6. Related work
  7. Estimator rank bounds in matrix sensing
  8. Analogous results in sparse recovery
  9. Low-rank optimization for matrix recovery
  10. Proof of main rank bound
  11. Preliminary facts
  12. Subgradient condition
  13. The idealized nuclear norm subgradient
  14. A matrix error bound
  15. Final rank bounds
  16. ...and 4 more sections

Key Result

Theorem 1

Suppose we have measurements of the form eq:measmodel. Let $r^* = \operatorname{rank}(M_*)$, and suppose $\scrA$ has $(2r^*, \delta^*)$-RIP for some $\delta^* > 0$. Choose $\lambda > 0$, and suppose that Then, with this choice of $\lambda$, eq:opt_orig has a unique solution $\Mhat$ that satisfies

Theorems & Definitions (10)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Corollary 1
  • Lemma 1
  • Lemma 2
  • Definition 1
  • Lemma 3
  • Lemma 4
  • proof : Proof of \ref{['lem:PPGDtoLandscape']}