Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Preserving Extreme Singular Values with One Oblivious Sketch

John M. Mango, Ronald Katende

TL;DR

The paper investigates whether a single oblivious sketch can uniformly bound the largest and smallest nonzero singular values of every rank-$r$ matrix. It presents a constructive route that pairs a sparse embedding with a deterministic geometric balancing operator to achieve constant-factor control of all nonzero singular values under mild coherence or spectral-gap assumptions, yielding a perfectly conditioned sketch ($\kappa(SA)=1$). In contrast, it proves an oblivious lower bound showing that $s = O(r\log r)$ is insufficient to uniformly preserve both extremes for all rank-$r$ matrices, highlighting a fundamental gap between data-dependent balancing and fully oblivious sketches. Numerical experiments corroborate the theory: balancing improves conditioning and accelerates iterative solvers on well-behaved data, while adversarial/coherent inputs reveal the predicted limits. Overall, the work delineates the boundary between achievable extreme-value preservation via data-dependent techniques and the inherent limitations of oblivious sketches for conditioning-sensitive tasks in numerical linear algebra.

Abstract

We study when a single linear sketch can control the largest and smallest nonzero singular values of every rank-$r$ matrix. Classical oblivious embeddings require $s=Θ(r/\varepsilon^{2})$ for $(1\pm\varepsilon)$ distortion, but this does not yield constant-factor control of extreme singular values or condition numbers. We formalize a conjecture that $s=O(r\log r)$ suffices for such preservation. On the constructive side, we show that combining a sparse oblivious sketch with a deterministic geometric balancing map produces a sketch whose nonzero singular values collapse to a common scale under bounded condition number and coherence. On the negative side, we prove that any oblivious sketch achieving relative $\varepsilon$-accurate singular values for all rank-$r$ matrices must satisfy $s=Ω((r+\log(1/δ))/\varepsilon^{2})$. Numerical experiments on structured matrix families confirm that balancing improves conditioning and accelerates iterative solvers, while coherent or nearly rank-deficient inputs manifest the predicted failure modes.

Preserving Extreme Singular Values with One Oblivious Sketch

TL;DR

The paper investigates whether a single oblivious sketch can uniformly bound the largest and smallest nonzero singular values of every rank- matrix. It presents a constructive route that pairs a sparse embedding with a deterministic geometric balancing operator to achieve constant-factor control of all nonzero singular values under mild coherence or spectral-gap assumptions, yielding a perfectly conditioned sketch (). In contrast, it proves an oblivious lower bound showing that is insufficient to uniformly preserve both extremes for all rank- matrices, highlighting a fundamental gap between data-dependent balancing and fully oblivious sketches. Numerical experiments corroborate the theory: balancing improves conditioning and accelerates iterative solvers on well-behaved data, while adversarial/coherent inputs reveal the predicted limits. Overall, the work delineates the boundary between achievable extreme-value preservation via data-dependent techniques and the inherent limitations of oblivious sketches for conditioning-sensitive tasks in numerical linear algebra.

Abstract

We study when a single linear sketch can control the largest and smallest nonzero singular values of every rank- matrix. Classical oblivious embeddings require for distortion, but this does not yield constant-factor control of extreme singular values or condition numbers. We formalize a conjecture that suffices for such preservation. On the constructive side, we show that combining a sparse oblivious sketch with a deterministic geometric balancing map produces a sketch whose nonzero singular values collapse to a common scale under bounded condition number and coherence. On the negative side, we prove that any oblivious sketch achieving relative -accurate singular values for all rank- matrices must satisfy . Numerical experiments on structured matrix families confirm that balancing improves conditioning and accelerates iterative solvers, while coherent or nearly rank-deficient inputs manifest the predicted failure modes.

Paper Structure

This paper contains 11 sections, 3 theorems, 36 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminaries and related work
  3. Positioning within existing work
  4. Main Theorems
  5. Constructive Route
  6. Impossibility Route
  7. Numerical experiments
  8. Structured and adversarial synthetic matrices
  9. Application to iterative solvers for PDE and graph Laplacian systems
  10. Additional numerical summaries
  11. Reproducibility and Code Availability

Key Result

Theorem 1

Let $A \in \mathbb{R}^{m \times n}$ have rank $r$ and condition number for some fixed $\kappa_{0} \ge 1$. Fix $\varepsilon \in (0,1/4)$ and $\delta \in (0,1/10)$. Let $\Phi \in \mathbb{R}^{s \times m}$ be a sparse oblivious subspace embedding (for example, an OSNAP or CountSketch transform) with for a sufficiently large absolute constant $C>0$. Form and compute its thin singular value decomposi

Figures (2)

  • Figure 1: Empirical success rate for constant factor preservation of both extreme singular values over twenty trials for each combination of matrix type, sketch size $s$, and method
  • Figure 2: Residual norm $\|Ax_k - b\|_2$ as a function of iteration for gradient descent least squares on Poisson and path graph Laplacian systems, comparing the original system, a CountSketch only sketch, and the balanced RLCB sketch

Theorems & Definitions (8)

  • Theorem 1: Geometric balancing sketch
  • proof
  • Remark 1
  • Corollary 2: Implications for Krylov solvers
  • proof
  • Theorem 3: Oblivious lower bound for extreme singular values
  • proof
  • Remark 2