Universal Matrix Sparsifiers and Fast Deterministic Algorithms for Linear Algebra
Rajarshi Bhattacharjee, Gregory Dexter, Cameron Musco, Archan Ray, Sushant Sachdeva, David P Woodruff
TL;DR
This work introduces universal matrix sparsifiers: a fixed sampling matrix $\mathbf{S}$ with $\|\mathbf{1}-\mathbf{S}\|_2 \le \varepsilon n$ sparsifies every bounded-entry PSD matrix with $s=O(n/\varepsilon^2)$ nonzeros, and extends to non-PSD matrices with $s=\widetilde{O}(n/\varepsilon^4)$, achieving additive spectral-error bounds of $\varepsilon n$ or $\varepsilon \cdot \max(n,\|\mathbf{A}\|_1)$ respectively. The construction leverages Ramanujan expanders to realize $\mathbf{S}$ and yields tight lower bounds up to logarithmic factors, establishing near-optimal deterministic sampling. By derandomizing sparsification and combining it with a deterministic power method and deflation, the paper presents the first fast deterministic algorithms for singular value/vector approximation and PSD testing with subquadratic or subcubic time, surpassing prior randomized sublinear-sample approaches in determinism. It also develops improved bounds for binary-magnitude PSD matrices and connects to spectral graph sparsification, offering a broad framework for deterministic spectral approximation beyond sparse representations. These results collectively advance derandomized, sublinear-entry-query strategies for core linear-algebra tasks with potential wide-reaching impact on deterministic numerical linear algebra and related applications.
Abstract
Let $\mathbf S \in \mathbb R^{n \times n}$ satisfy $\|\mathbf 1-\mathbf S\|_2\leεn$, where $\mathbf 1$ is the all ones matrix and $\|\cdot\|_2$ is the spectral norm. It is well-known that there exists such an $\mathbf S$ with just $O(n/ε^2)$ non-zero entries: we can let $\mathbf S$ be the scaled adjacency matrix of a Ramanujan expander graph. We show that such an $\mathbf S$ yields a $universal$ $sparsifier$ for any positive semidefinite (PSD) matrix. In particular, for any PSD $\mathbf A \in \mathbb{R}^{n\times n}$ with entries bounded in magnitude by $1$, $\|\mathbf A - \mathbf A\circ\mathbf S\|_2 \le εn$, where $\circ$ denotes the entrywise (Hadamard) product. Our techniques also give universal sparsifiers for non-PSD matrices. In this case, letting $\mathbf S$ be the scaled adjacency matrix of a Ramanujan graph with $\tilde O(n/ε^4)$ edges, we have $\|\mathbf A - \mathbf A \circ \mathbf S \|_2 \le ε\cdot \max(n,\|\mathbf A\|_1)$, where $\|\mathbf A\|_1$ is the nuclear norm. We show that the above bounds for both PSD and non-PSD matrices are tight up to log factors. Since $\mathbf A \circ \mathbf S$ can be constructed deterministically, our result for PSD matrices derandomizes and improves upon known results for randomized matrix sparsification, which require randomly sampling ${O}(\frac{n \log n}{ε^2})$ entries. We also leverage our results to give the first deterministic algorithms for several problems related to singular value approximation that run in faster than matrix multiplication time. Finally, if $\mathbf A \in \{-1,0,1\}^{n \times n}$ is PSD, we show that $\mathbf{\tilde A}$ with $\|\mathbf A - \mathbf{\tilde A}\|_2 \le εn$ can be obtained by deterministically reading $\tilde O(n/ε)$ entries of $\mathbf A$. This improves the $1/ε$ dependence on our result for general PSD matrices and is near-optimal.