Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Subspace Embeddings: Resolving Nelson-Nguyen Conjecture Up to Sub-Polylogarithmic Factors

Shabarish Chenakkod, Michał Dereziński, Xiaoyu Dong

TL;DR

This work resolves the Nelson–Nguyen conjecture on oblivious subspace embeddings up to sub-polylogarithmic factors by constructing OSNAP-based embeddings with dimension m = 4~O(d/b5^2) and sparsity per column 4~O(b5^{-1} ) that preserve all d-dimensional subspaces with high probability. The authors introduce iterative decoupling, a novel matrix-concentration technique that recursively tightens higher-order trace moments by decoupling powers into products of independent copies, refining Gaussian-universality-based bounds. They develop a sequence of moment estimates culminating in embedding guarantees for OSNAP matrices, and demonstrate implications for fast linear-regression sketches with time 4~O( ext{nnz}(A)ce 8/d). The results unify and surpass prior polylogarithmic gaps, offering a principled, near-optimal pathway to dimension-sparsity trade-offs in subspace embedding, with direct impact on sketching-based linear algebra. Practically, this yields faster sketch-based reductions for large-scale regression and matrix computations, underpinned by a robust probabilistic framework for higher-order moment control.

Abstract

We give a proof of the conjecture of Nelson and Nguyen [FOCS 2013] on the optimal dimension and sparsity of oblivious subspace embeddings, up to sub-polylogarithmic factors: For any $n\geq d$ and $ε\geq d^{-O(1)}$, there is a random $\tilde O(d/ε^2)\times n$ matrix $Π$ with $\tilde O(\log(d)/ε)$ non-zeros per column such that for any $A\in\mathbb{R}^{n\times d}$, with high probability, $(1-ε)\|Ax\|\leq\|ΠAx\|\leq(1+ε)\|Ax\|$ for all $x\in\mathbb{R}^d$, where $\tilde O(\cdot)$ hides only sub-polylogarithmic factors in $d$. Our result in particular implies a new fastest sub-current matrix multiplication time reduction of size $\tilde O(d/ε^2)$ for a broad class of $n\times d$ linear regression tasks. A key novelty in our analysis is a matrix concentration technique we call iterative decoupling, which we use to fine-tune the higher-order trace moment bounds attainable via existing random matrix universality tools [Brailovskaya and van Handel, GAFA 2024].

Optimal Subspace Embeddings: Resolving Nelson-Nguyen Conjecture Up to Sub-Polylogarithmic Factors

TL;DR

This work resolves the Nelson–Nguyen conjecture on oblivious subspace embeddings up to sub-polylogarithmic factors by constructing OSNAP-based embeddings with dimension m = 4~O(d/b5^2) and sparsity per column 4~O(b5^{-1} ) that preserve all d-dimensional subspaces with high probability. The authors introduce iterative decoupling, a novel matrix-concentration technique that recursively tightens higher-order trace moments by decoupling powers into products of independent copies, refining Gaussian-universality-based bounds. They develop a sequence of moment estimates culminating in embedding guarantees for OSNAP matrices, and demonstrate implications for fast linear-regression sketches with time 4~O( ext{nnz}(A)ce 8/d). The results unify and surpass prior polylogarithmic gaps, offering a principled, near-optimal pathway to dimension-sparsity trade-offs in subspace embedding, with direct impact on sketching-based linear algebra. Practically, this yields faster sketch-based reductions for large-scale regression and matrix computations, underpinned by a robust probabilistic framework for higher-order moment control.

Abstract

We give a proof of the conjecture of Nelson and Nguyen [FOCS 2013] on the optimal dimension and sparsity of oblivious subspace embeddings, up to sub-polylogarithmic factors: For any and , there is a random matrix with non-zeros per column such that for any , with high probability, for all , where hides only sub-polylogarithmic factors in . Our result in particular implies a new fastest sub-current matrix multiplication time reduction of size for a broad class of linear regression tasks. A key novelty in our analysis is a matrix concentration technique we call iterative decoupling, which we use to fine-tune the higher-order trace moment bounds attainable via existing random matrix universality tools [Brailovskaya and van Handel, GAFA 2024].

Paper Structure

This paper contains 33 sections, 37 theorems, 362 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction.
  2. Our Results.
  3. Implications for Linear Regression.
  4. Further Related Work.
  5. Main Results and Techniques.
  6. Main Result.
  7. Overview of the Techniques.
  8. High-level proof sketch.
  9. Moment estimates obtained from iterative decoupling.
  10. Towards iterative decoupling: Intuition in a general random matrix model.
  11. First attempt: Naïve full decoupling.
  12. Alternative approach: Moment bounds via Gaussian universality.
  13. Iterative decoupling to fine-tune universality error.
  14. Iterative Decoupling for OSNAP.
  15. Obtaining the Subspace Embedding Guarantee.
  16. ...and 18 more sections

Key Result

Theorem 1.2

For any $n\geq d$ and $\varepsilon \geq d^{-O(1)}$, there is a random $\Pi\in\mathop{\mathrm{\mathbb{R}}}\nolimits^{m\times n}$ with $\tilde{O}(\log(d)/\varepsilon)$Here and throughout, the notation $\tilde{O}(\cdot)$ hides only sub-polylogarithmic factors in $d$, i.e., $\log^{o(1)}(d)$. non-zeros p

Figures (3)

  • Figure 1: Plotting the exponent in front of $q=\log(d/\delta)$ as a function of $k$ in $\lVert(S_1 U)^T(S_2 U)\rVert_{2q \cdot 2^k} \le C_k(K+K^{\alpha_k}(q^{2}R(I_d))^{1-\alpha_k}) \approx O(q^{1+\frac{2}{2k+1}})$ under the condition that $pmpd=Cq^2$ (which holds for the conjectured sparsity).
  • Figure 2: First three steps of iterative decoupling for general random matrices.
  • Figure 3: Overview of the iterative decoupling argument for OSNAP.

Theorems & Definitions (63)

  • Conjecture 1.1: Nelson and Nguyen, FOCS 2013 nelson2013osnap
  • Theorem 1.2: Nelson-Nguyen up to sub-polylogarithmic factors
  • Corollary 1.3: Fast reduction for linear regression
  • Theorem 2.1: High Probability Bounds for the Embedding Error of OSNAP
  • Corollary 2.2
  • Corollary 2.3
  • proof : Proof of Theorem \ref{['t:main']}
  • Proposition 2.4: Theorem 2.9, brailovskaya2022universality
  • Lemma 3.1: Theorem 15.10 from boucheron2013concentration
  • Lemma 3.2: Lemma 6.2.2. in vershynin2018high
  • ...and 53 more