Tight Bounds for Heavy-Hitters and Moment Estimation in the Sliding Window Model

TL;DR

This work addresses the problem of estimating $F_p$ moments and identifying heavy-hitters within a sliding window of size $n$ in data streams. It introduces the Strong Estimator and a complementary Difference Estimator to overcome traditional union-bound barriers, enabling near-optimal space in the sliding-window setting. For $1<p\le 2$, the authors achieve a $(1\pm\varepsilon)$-approximation of the window's $F_p$ moment using $\tilde{O}(\frac{1}{\varepsilon^p}\log^2 n + \frac{1}{\varepsilon^2}\log n)$ bits on any fixed window (probability $2/3$), and they show this is tight up to $\log\log n$ and $\log\frac{1}{\varepsilon}$. As a corollary, heavy-hitters over any window can be computed in $O(\frac{1}{\varepsilon^2}\log^2 n)$ space, matching known lower bounds up to polylog factors. The paper also proves a lower bound of $\Omega(\frac{1}{\varepsilon^p}\log^2(\varepsilon U) + \frac{1}{\varepsilon^2}\log(\varepsilon^{1/p} U))$ for $F_p$-estimation in the sliding window model, establishing near-optimality of the results. These contributions advance practical, theoretically tight sliding-window streaming algorithms with potential impact on network monitoring and time-window data analytics, where keeping full history is infeasible.

Abstract

We consider the heavy-hitters and $F_p$ moment estimation problems in the sliding window model. For $F_p$ moment estimation with $1<p\leq 2$, we show that it is possible to give a $(1\pm ε)$ multiplicative approximation to the $F_p$ moment with $2/3$ probability on any given window of size $n$ using $\tilde{O}(\frac{1}{ε^p}\log^2 n + \frac{1}{ε^2}\log n)$ bits of space. We complement this result with a lower bound showing that our algorithm gives tight bounds up to factors of $\log\log n$ and $\log\frac{1}ε.$ As a consequence of our $F_2$ moment estimation algorithm, we show that the heavy-hitters problem can be solved on an arbitrary window using $O(\frac{1}{ε^2}\log^2 n)$ space which is tight.

Theorems & Definitions (56)

• Theorem 1.1
• Theorem 1.2
• Theorem 1.3
• Definition 2.1
• Definition 2.2
• Lemma 2.3
• Lemma 2.4
• Lemma 2.5
• Lemma 2.6
• Lemma 3.2