On the Space Complexity of Online Convolution
Joel Daniel Andersson, Amir Yehudayoff
TL;DR
The paper analyzes the space complexity of online convolution in a streaming setting, showing a sharp dichotomy governed by the generating function $G[a]$ of the Toeplitz kernel: rational $G$ of degree $d$ imposes a tight $O(d)$ space bound (with exact characterization $\mathsf{space}_a(t)=d$ once $t\ge d$), while irrational $G$ forces infinite space on infinite streams. It develops a lower-bound mechanism via Hankel-matrix rank and a Borsuk-Ulam argument, and proves that algebraic streaming can be reduced to linear, enabling the lower-bound framework. Finite-horizon behavior is tied to Hankel-rank properties and yields precise results for challenging functions like $G_{1/2}$, where $\mathsf{space}_{G_{1/2}}(t;2t)=t+1$. The work has DP-continual-counting implications, clarifying when sublinear-space streaming is possible and illuminating the space-accuracy tradeoffs for private streaming queries.
Abstract
We study a discrete convolution streaming problem. An input arrives as a stream of numbers $z = (z_0,z_1,z_2,\ldots)$, and at time $t$ our goal is to output $(Tz)_t$ where $T$ is a lower-triangular Toeplitz matrix. We focus on space complexity; the algorithm can store a buffer of $β(t)$ numbers in order to achieve this goal. We characterize space complexity when algorithms perform continuous operations. The matrix $T$ corresponds to a generating function $G(x)$. If $G(x)$ is rational of degree $d$, then it is known that the space complexity is at most $O(d)$. We prove a corresponding lower bound; the space complexity is at least $Ω(d)$. In addition, for irrational $G(x)$, we prove that the space complexity is infinite. We also provide finite-time guarantees. For example, for the generating function $\frac{1}{\sqrt{1-x}}$ that was studied in various previous works in the context of differentially private continual counting, we prove a sharp lower bound on the space complexity; at time $t$, it is at least $Ω(t)$.