Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scalable Differentially Private Sketches under Continual Observation

Rayne Holland

TL;DR

The paper tackles privacy-preserving frequency estimation and heavy hitter detection in data streams under continual observation. It introduces LazySketch, a lazy-update framework that reduces per-update cost while maintaining differential privacy, complemented by a Gaussian Binary Mechanism for private counting. The approach improves throughput up to 250x over previous continual-observation methods and demonstrates solid utility on synthetic and real-world data. This work enables scalable, private streaming analytics without compromising real-time update performance.

Abstract

Linear sketches are fundamental tools in data stream analytics. They are notable for supporting both approximate frequency queries and heavy hitter detection with bounded trade-offs for error and memory. Importantly, on streams that contain sensitive information, linear sketches can be easily privatized with the injection of a suitable amount of noise. This process is efficient in the single release model, where the output is released only at the end of the stream. In this setting, it suffices to add noise to the sketch once. In contrast, in the continual observation model, where the output is released at every time-step, fresh noise needs to be added to the sketch before each release. This creates an additional computational overhead. To address this, we introduce Lazy Sketch, a novel differentially private sketching method that employs lazy updates, perturbing and modifying only a small portion of the sketch at each step. Compared to prior work, we reduce the update complexity by a factor of $O(w)$, where $w$ is the width of the sketch. Experiments demonstrate that our method increases throughput by up to 250x over prior work, making continual observation differential privacy practical for high-speed streaming applications.

Scalable Differentially Private Sketches under Continual Observation

TL;DR

The paper tackles privacy-preserving frequency estimation and heavy hitter detection in data streams under continual observation. It introduces LazySketch, a lazy-update framework that reduces per-update cost while maintaining differential privacy, complemented by a Gaussian Binary Mechanism for private counting. The approach improves throughput up to 250x over previous continual-observation methods and demonstrates solid utility on synthetic and real-world data. This work enables scalable, private streaming analytics without compromising real-time update performance.

Abstract

Linear sketches are fundamental tools in data stream analytics. They are notable for supporting both approximate frequency queries and heavy hitter detection with bounded trade-offs for error and memory. Importantly, on streams that contain sensitive information, linear sketches can be easily privatized with the injection of a suitable amount of noise. This process is efficient in the single release model, where the output is released only at the end of the stream. In this setting, it suffices to add noise to the sketch once. In contrast, in the continual observation model, where the output is released at every time-step, fresh noise needs to be added to the sketch before each release. This creates an additional computational overhead. To address this, we introduce Lazy Sketch, a novel differentially private sketching method that employs lazy updates, perturbing and modifying only a small portion of the sketch at each step. Compared to prior work, we reduce the update complexity by a factor of , where is the width of the sketch. Experiments demonstrate that our method increases throughput by up to 250x over prior work, making continual observation differential privacy practical for high-speed streaming applications.

Paper Structure

This paper contains 19 sections, 7 theorems, 23 equations, 4 figures, 1 table, 5 algorithms.

Table of Contents

  1. Introduction
  2. Our Solution
  3. Paper Outline
  4. Preliminaries
  5. Privacy
  6. Linear Sketching
  7. The Gaussian Binary Mechanism
  8. The $n$-Counters Problem
  9. Lazy Frequency Estimation
  10. The Punctual Sketch
  11. The Lazy Sketch
  12. Properties
  13. Experiments
  14. Implemented Sketches
  15. Datasets and Metrics
  16. ...and 4 more sections

Key Result

Lemma 1

Let $f$ be a function with $\ell_2$-sensitivity $\triangle_2(f)$. For any $\varepsilon \in (0,1)$ and $\delta \in (0,1)$, the mechanism where $\sigma \geq \frac{\triangle_2(f) \cdot \sqrt{2 \ln (1.25/\delta)}}{\varepsilon}$, satisfies $(\varepsilon, \delta)$-differential privacy.

Figures (4)

  • Figure 1: The update procedure for a Count-Min Sketch. It follows that $h_j(x)=i$.
  • Figure 2: The Binary Mechanism. $c_i$ denotes the counter increment at time $i$. The sum released at time $t=7$ is $c_7+c_{5:6}+c_{1:4}+ \gamma_7 + \gamma_{10} + \gamma_{11}$.
  • Figure 3: Update procedure for the $\mathsf{LazySketch}$. At time $t$, column $i = t\mod w$ from $P$ gets pushed to the output sketch.
  • Figure 4: Comparative frequency estimation on synthetic data. $\delta = 0.001$. For Figures \ref{['fig:mem_freq_time']} & \ref{['fig:mem_freq_are']}, memory is increased by changing the width of the data structures. In contrast, for Figure \ref{['fig:depth_freq_are']}, memory is increased by changing the depth of the data structures.

Theorems & Definitions (13)

  • Definition 1: $(\varepsilon, \delta)$-Differential Privacy
  • Lemma 1: Gaussian Mechanism dwork2014algorithmic
  • Lemma 2: Approximation error for $\mathsf{CMS}$ cormode2005improved
  • Lemma 3: Approximation error for $\mathsf{CS}$ charikar2002finding
  • Definition 2: $m$-Neighboring Inputs
  • Theorem 1: Gaussian Binary Mechanism
  • proof
  • Lemma 4
  • proof
  • Theorem 2: $\mathsf{LazySketch}$ Privacy
  • ...and 3 more