$L_p$ Sampling in Distributed Data Streams with Applications to Adversarial Robustness
Honghao Lin, Zhao Song, David P. Woodruff, Shenghao Xie, Samson Zhou
TL;DR
This work resolves the long-standing challenge of designing perfect $L_p$ samplers in distributed monitoring for all $p\ge 1$, achieving near-optimal communication $k^{\max(1,p-1)}\cdot\mathrm{polylog}(n)$. The authors leverage a novel two-sided exponential embedding and a suite of techniques—global $L_1$-of-$L_p$ heavy-hitter concepts, a double-exponential scaling, anti-concentration, geometric-mean estimators, and truncated Taylor series—to produce a perfect sampler with robust statistical properties. Building on this primitive, they develop a general difference-estimator framework that yields adversarially robust distributed protocols for central problems like $F_p$ moment estimation, counting, distinct elements, and heavy hitters, achieving near-optimal communication and resilience to adaptive inputs. The paper also presents lower bounds showing optimality of the sampling bounds up to polylog factors and discusses concurrent work on robustness in non-adaptive settings, highlighting the essential role of perfect $L_p$ sampling for robustness. Overall, these advances enable efficient, adversarially robust data analysis in distributed streams with provable optimality guarantees and broad applicability to core data-processing tasks.
Abstract
In the distributed monitoring model, a data stream over a universe of size $n$ is distributed over $k$ servers, who must continuously provide certain statistics of the overall dataset, while minimizing communication with a central coordinator. In such settings, the ability to efficiently collect a random sample from the global stream is a powerful primitive, enabling a wide array of downstream tasks such as estimating frequency moments, detecting heavy hitters, or performing sparse recovery. Of particular interest is the task of producing a perfect $L_p$ sample, which given a frequency vector $f \in \mathbb{R}^n$, outputs an index $i$ with probability $\frac{f_i^p}{\|f\|_p^p}+\frac{1}{\mathrm{poly}(n)}$. In this paper, we resolve the problem of perfect $L_p$ sampling for all $p\ge 1$ in the distributed monitoring model. Specifically, our algorithm runs in $k^{p-1} \cdot \mathrm{polylog}(n)$ bits of communication, which is optimal up to polylogarithmic factors. Utilizing our perfect $L_p$ sampler, we achieve adversarially-robust distributed monitoring protocols for the $F_p$ moment estimation problem, where the goal is to provide a $(1+\varepsilon)$-approximation to $f_1^p+\ldots+f_n^p$. Our algorithm uses $\frac{k^{p-1}}{\varepsilon^2}\cdot\mathrm{polylog}(n)$ bits of communication for all $p\ge 2$ and achieves optimal bounds up to polylogarithmic factors, matching lower bounds by Woodruff and Zhang (STOC 2012) in the non-robust setting. Finally, we apply our framework to achieve near-optimal adversarially robust distributed protocols for central problems such as counting, frequency estimation, heavy-hitters, and distinct element estimation.