Understanding the Kronecker Matrix-Vector Complexity of Linear Algebra
Raphael A. Meyer, William Swartworth, David P. Woodruff
TL;DR
This work establishes fundamental exponential lower bounds on the Kronecker matrix-vector query model, showing that estimating basic linear-algebra properties of a hidden matrix (such as the trace or top eigenvalue) requires a number of Kronecker-structured queries exponential in the tensor order $q$, even for well-conditioned algorithms. A central insight is near-total orthogonality of random Kronecker-structured vectors, which makes planted signals hard to detect via limited Kronecker measurements. The paper also reveals a sharp alphabet-dependent gap in zero-testing: Gaussian Kronecker queries can test zero-ness with a single query, while small-alphabet Kronecker queries (e.g., Rademacher) incur exponential sample complexity, and it extends these ideas to complex alphabets. Collectively, these results explain why Kronecker-based sketching (e.g., Kronecker JL/Hutchinson) often requires exponential sketch dimensions in $q$ and motivate seeking structure that avoids these worst-case scenarios. The findings have implications for the design of Kronecker-based tensor sketches and for understanding when subgaussian assumptions suffice in Kronecker settings.
Abstract
We study the computational model where we can access a matrix $\mathbf{A}$ only by computing matrix-vector products $\mathbf{A}\mathrm{x}$ for vectors of the form $\mathrm{x} = \mathrm{x}_1 \otimes \cdots \otimes \mathrm{x}_q$. We prove exponential lower bounds on the number of queries needed to estimate various properties, including the trace and the top eigenvalue of $\mathbf{A}$. Our proofs hold for all adaptive algorithms, modulo a mild conditioning assumption on the algorithm's queries. We further prove that algorithms whose queries come from a small alphabet (e.g., $\mathrm{x}_i \in \{\pm1\}^n$) cannot test if $\mathbf{A}$ is identically zero with polynomial complexity, despite the fact that a single query using Gaussian vectors solves the problem with probability 1. In steep contrast to the non-Kronecker case, this shows that sketching $\mathbf{A}$ with different distributions of the same subguassian norm can yield exponentially different query complexities. Our proofs follow from the observation that random vectors with Kronecker structure have exponentially smaller inner products than their non-Kronecker counterparts.