The Structural Complexity of Matrix-Vector Multiplication

TL;DR

This work resolves a long-standing gap between practical speedups and worst-case hardness for matrix-vector multiplication by focusing on structured inputs. It introduces corrupted VC-dimension as a robust measure of near-structure in real data and shows that matrices with such structure admit near-quadratic preprocessing and subquadratic query times via a Δ-labeled spanning-tree MST framework. The results extend to transposed and non-Boolean matrices (via Pollard pseudodimension) and yield dynamic-data structures with amortized update times that enable subquadratic maintenance of high-accuracy Laplacian solvers, effective resistance, and triangle detection, among others. Collectively, the paper provides both a unifying theory and practical subquadratic algorithms for a broad class of matrix-vector and graph-related problems, offering concrete explanations for observed empirical performance on structured inputs and suggesting directions for further generalizations.

Abstract

We consider the problem of preprocessing an $n\times n$ matrix M, and supporting queries that, for any vector v, returns the matrix-vector product Mv. This problem has been extensively studied in both theory and practice: on one side, practitioners have developed algorithms that are highly efficient in practice, whereas theoreticians have proven that the problem cannot be solved faster than naive multiplication in the worst-case. This lower bound holds even in the average-case, implying that existing average-case analyses cannot explain this gap between theory and practice. Therefore, we study the problem for structured matrices. We show that for $n\times n$ matrices of VC-dimension d, the matrix-vector multiplication problem can be solved with $\tilde{O}(n^2)$ preprocessing and $\tilde O(n^{2-1/d})$ query time. Given the low constant VC-dimensions observed in most real-world data, our results posit an explanation for why the problem can be solved so much faster in practice. Moreover, our bounds hold even if the matrix does not have a low VC-dimension, but is obtained by (possibly adversarially) corrupting at most a subquadratic number of entries of any unknown low VC-dimension matrix. Our results yield the first non-trivial upper bounds for many applications. In previous works, the online matrix-vector hypothesis (conjecturing that quadratic time is needed per query) was used to prove many conditional lower bounds, showing that it is impossible to compute and maintain high-accuracy estimates for shortest paths, Laplacian solvers, effective resistance, and triangle detection in graphs subject to node insertions and deletions in subquadratic time. Yet, via a reduction to our matrix-vector-multiplication result, we show we can maintain the aforementioned problems efficiently if the input is structured, providing the first subquadratic upper bounds in the high-accuracy regime.

Theorems & Definitions (62)

• definition 1: Corrupted VC-dimension d
• Theorem 2
• Theorem 3
• Corollary 4
• Theorem 5
• Theorem 6
• Theorem 7
• Theorem 8
• Theorem 9
• definition 9: $\Delta$-labeled spanning tree of matrix $\mM$