Optimal Oblivious Subspace Embeddings with Near-optimal Sparsity

TL;DR

The paper resolves a central question in randomized linear algebra by constructing an oblivious subspace embedding with optimal dimension $m=\Theta(d/\varepsilon^2)$ while achieving near-optimal sparsity $s=\tilde O(1/\varepsilon)$ per column, nearly matching the Nelson–Nguyen sparsity conjecture. It introduces LESS-IC, a leverage-score–aware sparse embedding that attains these guarantees with fast application times, and extends the approach to LESS-IE and OSE-IE variants for data-aware or independent-entry models. A novel analytical framework combining a decoupling technique with a two-dimensional interpolation (via chain-rule derivatives) bounds trace moments of embedding errors, yielding sharp $1/\varepsilon$ sparsity while preserving subspace geometry with high probability. The contributions include fast subspace-embedding constructions, improved runtimes for matrix approximation and regression tasks, and a versatile universality toolkit that may influence future random-matrix analyses. Overall, the results significantly advance practical dimensionality reduction by achieving optimal dimension alongside practical sparsity and speed.

Abstract

An oblivious subspace embedding is a random $m\times n$ matrix $Π$ such that, for any $d$-dimensional subspace, with high probability $Π$ preserves the norms of all vectors in that subspace within a $1\pmε$ factor. In this work, we give an oblivious subspace embedding with the optimal dimension $m=Θ(d/ε^2)$ that has a near-optimal sparsity of $\tilde O(1/ε)$ non-zero entries per column of $Π$. This is the first result to nearly match the conjecture of Nelson and Nguyen [FOCS 2013] in terms of the best sparsity attainable by an optimal oblivious subspace embedding, improving on a prior bound of $\tilde O(1/ε^6)$ non-zeros per column [Chenakkod et al., STOC 2024]. We further extend our approach to the non-oblivious setting, proposing a new family of Leverage Score Sparsified embeddings with Independent Columns, which yield faster runtimes for matrix approximation and regression tasks. In our analysis, we develop a new method which uses a decoupling argument together with the cumulant method for bounding the edge universality error of isotropic random matrices. To achieve near-optimal sparsity, we combine this general-purpose approach with new traces inequalities that leverage the specific structure of our subspace embedding construction.

Figures (3)

• Figure 1: An example of a column divided into $s=3$ subcolumns with each subcolumn having exactly one non-zero entry in a random position.
• Figure 2: In the LESS-IC distribution, column $j$ is filled with $s_j$ many subcolumns, with the bottom-most subcolumn truncated to fit the size of $\Pi$. Each subcolumn has one non-zero entry. Notice that as the leverage scores decrease, the number of subcolumns decreases and the matrix becomes sparser. However, each column always has at least one non-zero entry.
• Figure 3: Two-dimensional interpolation in $(t_1,t_2)\in[0,1]^2$, decomposed using the chain rule.

Theorems & Definitions (89)

• Definition 1.1
• Conjecture 1.2: Nelson and Nguyen, FOCS 2013 nelson2013osnap
• theorem 1.3: Oblivious Subspace Embedding
• theorem 1.4: Fast Subspace Embedding
• Corollary 1.5: Fast reduction for constrained least squares
• Definition 3.1: OSNAP
• theorem 3.2: Subspace Embedding Guarantee for OSNAP
• Remark 3.3
• Remark 3.4
• Definition 3.5