Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the distance to low-rank matrices in the maximum norm

Stanislav Budzinskiy

TL;DR

This work studies how close a large matrix with small spectral norm can be to a rank-$r$ matrix in the max norm, refining and correcting prior max-norm bounds and deriving new, coherence- and spikiness-dependent estimates. The authors establish two main results: a Johnson–Lindenstrauss–based bound showing $\|X-Y\|_{\max} \le \frac{\varepsilon}{3}ig(\frac{k}{m}\mu_{\mathrm{col}}+\frac{k}{n}\mu_{\mathrm{row}}+\frac{\gamma_X}{\sqrt{mn}}\big)\|X\|_2$ and a tighter Hanson–Wright–based bound $\|X-Y\|_{\max} \le \varepsilon\frac{k}{\sqrt{mn}}\sqrt{\mu_{\mathrm{col}}\mu_{\mathrm{row}}}\,\|X\|_2$, under incoherence and low spikiness assumptions. A corrected version of the previous result clarifies the rank dependence, and the paper together with numerical experiments using alternating projections demonstrates the practical tightness and phase-transition behavior of these bounds. The numerical study across identity, random, banded, and orthogonally factorized matrices reveals how the distance scales with rank, spectral norm, and matrix class, informing both theory and algorithm design for max-norm low-rank approximation. Overall, the results delineate when and how strong max-norm low-rank approximations are attainable and quantify the pivotal role of spectral norm and coherence in this non-unitarily invariant setting.

Abstract

Every sufficiently big matrix with small spectral norm has a nearby low-rank matrix if the distance is measured in the maximum norm (Udell & Townsend, SIAM J Math Data Sci, 2019). We use the Hanson--Wright inequality to improve the estimate of the distance for matrices with incoherent column and row spaces. In numerical experiments with several classes of matrices we study how well the theoretical upper bound describes the approximation errors achieved with the method of alternating projections.

On the distance to low-rank matrices in the maximum norm

TL;DR

This work studies how close a large matrix with small spectral norm can be to a rank- matrix in the max norm, refining and correcting prior max-norm bounds and deriving new, coherence- and spikiness-dependent estimates. The authors establish two main results: a Johnson–Lindenstrauss–based bound showing and a tighter Hanson–Wright–based bound , under incoherence and low spikiness assumptions. A corrected version of the previous result clarifies the rank dependence, and the paper together with numerical experiments using alternating projections demonstrates the practical tightness and phase-transition behavior of these bounds. The numerical study across identity, random, banded, and orthogonally factorized matrices reveals how the distance scales with rank, spectral norm, and matrix class, informing both theory and algorithm design for max-norm low-rank approximation. Overall, the results delineate when and how strong max-norm low-rank approximations are attainable and quantify the pivotal role of spectral norm and coherence in this non-unitarily invariant setting.

Abstract

Every sufficiently big matrix with small spectral norm has a nearby low-rank matrix if the distance is measured in the maximum norm (Udell & Townsend, SIAM J Math Data Sci, 2019). We use the Hanson--Wright inequality to improve the estimate of the distance for matrices with incoherent column and row spaces. In numerical experiments with several classes of matrices we study how well the theoretical upper bound describes the approximation errors achieved with the method of alternating projections.
Paper Structure (12 sections, 9 theorems, 52 equations, 4 figures)

This paper contains 12 sections, 9 theorems, 52 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Contributions and outline
  3. Proof based on the Johnson--Lindenstrauss lemma
  4. Proof based on the Hanson--Wright inequality
  5. Numerical experiments
  6. Known algorithms
  7. Alternating projections
  8. Identity matrices
  9. Random uniform matrices
  10. Banded random uniform matrices
  11. Products of random matrices with orthonormal columns
  12. Discussion

Key Result

Theorem 1.1

Let $X \in \mathbb{R}^{n \times n}$ be symmetric positive semidefinite. For $\varepsilon \in (0,1)$, there exists a matrix $Y \in \mathbb{R}^{n \times n}$ such that

Figures (4)

  • Figure 1: Low-rank approximation errors in the maximum norm obtained with the method of alternating projections for the class of identity matrices: (left) fixed matrix size $n$, (right) fixed approximation rank $r$.
  • Figure 2: Low-rank approximation errors in the maximum norm obtained with the method of alternating projections for the class of random uniform matrices: (left) fixed matrix size $n$, (right) fixed approximation rank $r$.
  • Figure 3: Properties of $2000 \times 2000$ banded random uniform matrices: (left) spectral norm as a function of band width $b$; (right) low-rank approximation errors in the maximum norm obtained with the method of alternating projections for fixed approximation rank $r$.
  • Figure 4: Properties of $2000 \times 2000$ matrices formed as products of $2000 \times k$ random matrices with orthonormal columns and normalized to unit maximum norm: (left) spectral norm as a function of rank $k$; (right) low-rank approximation errors in the maximum norm obtained with the method of alternating projections for fixed approximation rank $r$.

Theorems & Definitions (16)

  • Theorem 1.1: alon2013approximate
  • Theorem 1.2: udell2019big
  • Theorem 2.1: dasgupta2003elementary
  • Corollary 2.2
  • proof
  • Theorem 2.3
  • proof
  • Remark 2.4
  • Remark 2.5
  • Theorem 3.1: rudelson2013hanson
  • ...and 6 more