Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Keeping a Secret Requires a Good Memory: Space Lower-Bounds for Private Algorithms

Alessandro Epasto, Xin Lyu, Pasin Manurangsi

TL;DR

This paper addresses the memory cost of differential privacy in streaming by proving unconditional space lower bounds for user-level DP. It introduces a multi-player communication game that ties the privacy requirement to contribution capping of over-active users, showing that resolving heavy hitters under DP requires memory scaling with the number of such users. Focusing on CountDistinct in a $(w,k)$-occurrency bounded regime, it proves a lower bound $M \ge \widetilde{\Omega}(\frac{kw}{h^2})$ on the space needed for achieving a $(\tau,\eta)$-approximation, and demonstrates an exponential separation from non-private algorithms; the technique also extends to MaxSelect and Quantile. The results imply that privacy can impose fundamental memory costs even for natural statistical estimation tasks, and the proposed information-theoretic framework may apply to a broad class of private streaming problems with memory-accuracy tradeoffs.

Abstract

We study the computational cost of differential privacy in terms of memory efficiency. While the trade-off between accuracy and differential privacy is well-understood, the inherent cost of privacy regarding memory use remains largely unexplored. This paper establishes for the first time an unconditional space lower bound for user-level differential privacy by introducing a novel proof technique based on a multi-player communication game. Central to our approach, this game formally links the hardness of low-memory private algorithms to the necessity of ``contribution capping'' -- tracking and limiting the users who disproportionately impact the dataset. We demonstrate that winning this communication game requires transmitting information proportional to the number of over-active users, which translates directly to memory lower bounds. We apply this framework, as an example, to the fundamental problem of estimating the number of distinct elements in a stream and we prove that any private algorithm requires almost $\widetildeΩ(T^{1/3})$ space to achieve certain error rates in a promise variant of the problem. This resolves an open problem in the literature (by Jain et al. NeurIPS 2023 and Cummings et al. ICML 2025) and establishes the first exponential separation between the space complexity of private algorithms and their non-private $\widetilde{O}(1)$ counterparts for a natural statistical estimation task. Furthermore, we show that this communication-theoretic technique generalizes to broad classes of problems, yielding lower bounds for private medians, quantiles, and max-select.

Keeping a Secret Requires a Good Memory: Space Lower-Bounds for Private Algorithms

TL;DR

This paper addresses the memory cost of differential privacy in streaming by proving unconditional space lower bounds for user-level DP. It introduces a multi-player communication game that ties the privacy requirement to contribution capping of over-active users, showing that resolving heavy hitters under DP requires memory scaling with the number of such users. Focusing on CountDistinct in a -occurrency bounded regime, it proves a lower bound on the space needed for achieving a -approximation, and demonstrates an exponential separation from non-private algorithms; the technique also extends to MaxSelect and Quantile. The results imply that privacy can impose fundamental memory costs even for natural statistical estimation tasks, and the proposed information-theoretic framework may apply to a broad class of private streaming problems with memory-accuracy tradeoffs.

Abstract

We study the computational cost of differential privacy in terms of memory efficiency. While the trade-off between accuracy and differential privacy is well-understood, the inherent cost of privacy regarding memory use remains largely unexplored. This paper establishes for the first time an unconditional space lower bound for user-level differential privacy by introducing a novel proof technique based on a multi-player communication game. Central to our approach, this game formally links the hardness of low-memory private algorithms to the necessity of ``contribution capping'' -- tracking and limiting the users who disproportionately impact the dataset. We demonstrate that winning this communication game requires transmitting information proportional to the number of over-active users, which translates directly to memory lower bounds. We apply this framework, as an example, to the fundamental problem of estimating the number of distinct elements in a stream and we prove that any private algorithm requires almost space to achieve certain error rates in a promise variant of the problem. This resolves an open problem in the literature (by Jain et al. NeurIPS 2023 and Cummings et al. ICML 2025) and establishes the first exponential separation between the space complexity of private algorithms and their non-private counterparts for a natural statistical estimation task. Furthermore, we show that this communication-theoretic technique generalizes to broad classes of problems, yielding lower bounds for private medians, quantiles, and max-select.
Paper Structure (17 sections, 12 theorems, 19 equations)

This paper contains 17 sections, 12 theorems, 19 equations.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Problem Setting
  4. CountDistinct: the occurrency bounding technique
  5. Our main result
  6. Construction of Hard Instances
  7. Proof of Main Results
  8. Multi-Player Communication Game of Avoiding Heavy Hitters
  9. From DP Streaming Algorithm to Communication Protocol
  10. Correlation Analysis and Rounding Algorithm
  11. Proof of \ref{['thm: main']}
  12. Conclusions
  13. Missing Proofs from \ref{['sec:proof']}
  14. Technical preliminaries
  15. The rounding analysis
  16. ...and 2 more sections

Key Result

Theorem 2.1

Let $\gamma_w, \gamma_k, \gamma_h \in (0, 1)$ be such that $\frac{1}{2} \gamma_w \le \gamma_k \le \gamma_h < \gamma_w$ and $3\gamma_h < 1$, and $T\ge 1$ be sufficiently large. Let $w = T^{\gamma_w}, k = T^{\gamma_k}, h = T^{\gamma_h}$. Suppose $A$ is an $(\varepsilon = 0.5,\delta = \frac{1}{T^2})$-D

Theorems & Definitions (17)

  • Theorem 2.1
  • Corollary 2.2
  • Theorem 2.3
  • Lemma 3.1
  • Lemma 3.2
  • Corollary 3.3
  • proof
  • Lemma 3.4: See, e.g., BunUV18DworkSSUV15
  • Definition 1.1: Subgaussian r.v.
  • Lemma 1.2: Sum of Subgaussian r.v.'s
  • ...and 7 more