Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Individualized Privacy Accounting via Subsampling with Applications in Combinatorial Optimization

Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Adam Sealfon

TL;DR

This work introduces individualized privacy accounting by leveraging subsampling to convert one-sided add-DP guarantees into two-sided, pure-DP guarantees. By applying this amplification to repeated exponential mechanism and repeated above-threshold procedures, the authors derive new pure-DP algorithms with near-tight error for several combinatorial problems, including decomposable monotone submodular maximization under cardinality and matroid constraints, Set Cover, and metric clustering, as well as a pure-DP solution for shifting heavy hitters. The approach yields improved error bounds compared to prior approximate-DP results, and demonstrates a unifying framework that can extend to streaming and other DP settings. The results broaden the practical impact of differential privacy in discrete optimization, offering strong privacy with strong utility across a range of canonical problems.

Abstract

In this work, we give a new technique for analyzing individualized privacy accounting via the following simple observation: if an algorithm is one-sided add-DP, then its subsampled variant satisfies two-sided DP. From this, we obtain several improved algorithms for private combinatorial optimization problems, including decomposable submodular maximization and set cover. Our error guarantees are asymptotically tight and our algorithm satisfies pure-DP while previously known algorithms (Gupta et al., 2010; Chaturvedi et al., 2021) are approximate-DP. We also show an application of our technique beyond combinatorial optimization by giving a pure-DP algorithm for the shifting heavy hitter problem in a stream; previously, only an approximateDP algorithm was known (Kaplan et al., 2021; Cohen & Lyu, 2023).

Individualized Privacy Accounting via Subsampling with Applications in Combinatorial Optimization

TL;DR

This work introduces individualized privacy accounting by leveraging subsampling to convert one-sided add-DP guarantees into two-sided, pure-DP guarantees. By applying this amplification to repeated exponential mechanism and repeated above-threshold procedures, the authors derive new pure-DP algorithms with near-tight error for several combinatorial problems, including decomposable monotone submodular maximization under cardinality and matroid constraints, Set Cover, and metric clustering, as well as a pure-DP solution for shifting heavy hitters. The approach yields improved error bounds compared to prior approximate-DP results, and demonstrates a unifying framework that can extend to streaming and other DP settings. The results broaden the practical impact of differential privacy in discrete optimization, offering strong privacy with strong utility across a range of canonical problems.

Abstract

In this work, we give a new technique for analyzing individualized privacy accounting via the following simple observation: if an algorithm is one-sided add-DP, then its subsampled variant satisfies two-sided DP. From this, we obtain several improved algorithms for private combinatorial optimization problems, including decomposable submodular maximization and set cover. Our error guarantees are asymptotically tight and our algorithm satisfies pure-DP while previously known algorithms (Gupta et al., 2010; Chaturvedi et al., 2021) are approximate-DP. We also show an application of our technique beyond combinatorial optimization by giving a pure-DP algorithm for the shifting heavy hitter problem in a stream; previously, only an approximateDP algorithm was known (Kaplan et al., 2021; Cohen & Lyu, 2023).
Paper Structure (34 sections, 24 theorems, 60 equations, 5 algorithms)

This paper contains 34 sections, 24 theorems, 60 equations, 5 algorithms.

Table of Contents

  1. Introduction
  2. Our Results
  3. Technical Overview
  4. Preliminaries
  5. One-Sided DP and Subsampling
  6. Algorithm I: Repeated EM
  7. Applications
  8. Monotone Submodular Maximization under Cardinality Constraint
  9. Monotone Submodular Maximization under Matroid Constraint
  10. Metric $k$-Means and $k$-Median
  11. Algorithm II: Repeated Above Threshold
  12. Applications
  13. Shifting Heavy Hitters
  14. Set Cover
  15. Conclusion and Open Questions
  16. ...and 19 more sections

Key Result

Theorem 1.2

For any $0 < \varepsilon, \beta, \eta < 1$, there is an $\varepsilon$-DP algorithm for monotone submodular maximization under a cardinality constraint that achieves $\left(1 - \frac{1}{e} - \eta\right)$-approximation and $O\left(\frac{k \log (m/\beta)}{\eta \varepsilon}\right)$-error w.p. $1 - \beta

Theorems & Definitions (44)

  • Definition 1.1: Add-remove DP, dwork2006calibrating
  • Theorem 1.2
  • Theorem 1.3
  • Theorem 1.4
  • Theorem 1.5
  • Theorem 1.6: Informal
  • Definition 1.7: One-Sided DP, KDHM20
  • Theorem 2.1: Chernoff bound; Corollary 2.11 of CNZ-arxiv)
  • Theorem 2.2: MT07
  • Lemma 3.1
  • ...and 34 more