Budget Recycling Differential Privacy

TL;DR

This work tackles the utility-privacy tradeoff in differential privacy by introducing Budget Recycling Differential Privacy (BR-DP), a framework that couples a DP kernel with a probabilistic recycler. BR-DP allocates part of the privacy budget to the kernel and the rest to recycling via a parameter $q$, producing soft-bounded outputs whose acceptance probability within a tolerance $\theta$ is increased. The authors derive a budgeting principle, a tight composition theorem with a linear-time accounting algorithm, and a privacy amplification-by-subsampling analysis to further enhance utility. Empirical results on real datasets show BR-DP achieves higher acceptance rates and reduced privacy leakage after composition compared to conventional DP, demonstrating practical gains for diverse query types. The framework is adaptable to Gaussian and Laplacian kernels and offers a path toward improved utility without compromising formal DP guarantees.

Abstract

Differential Privacy (DP) mechanisms usually {force} reduction in data utility by producing "out-of-bound" noisy results for a tight privacy budget. We introduce the Budget Recycling Differential Privacy (BR-DP) framework, designed to provide soft-bounded noisy outputs for a broad range of existing DP mechanisms. By "soft-bounded," we refer to the mechanism's ability to release most outputs within a predefined error boundary, thereby improving utility and maintaining privacy simultaneously. The core of BR-DP consists of two components: a DP kernel responsible for generating a noisy answer per iteration, and a recycler that probabilistically recycles/regenerates or releases the noisy answer. We delve into the privacy accounting of BR-DP, culminating in the development of a budgeting principle that optimally sub-allocates the available budget between the DP kernel and the recycler. Furthermore, we introduce algorithms for tight BR-DP accounting in composition scenarios, and our findings indicate that BR-DP achieves reduced privacy leakage post-composition compared to DP. Additionally, we explore the concept of privacy amplification via subsampling within the BR-DP framework and propose optimal sampling rates for BR-DP across various queries. We experiment with real data, and the results demonstrate BR-DP's effectiveness in lifting the utility-privacy tradeoff provided by DP mechanisms.

Figures (8)

• Figure 1: Illustration of the Budget Recycling Differential Privacy (BR-DP) framework
• Figure 2: BR-DP noise distribution illustration
• Figure 3: Comparison of $q$ for baseline and optimal approaches. $\epsilon = 3$, $\delta = \delta_y$ with values shown in the figure, $y=10$, $\Delta_f = 1$, $\theta = 1$.
• Figure 4: Numerical comparison of the acceptance rate provided by Gaussian DP and the Gaussian kernel BR-DP under different parameters.
• Figure 5: Composed leakage comparison for BR-DP and DP with Gaussian kernel and Laplacian kernel, respectively. For each plot, $\epsilon = 1$, $\theta = 1$, $\Delta_f = 1$. $\epsilon_y$ and $q$ are derived from Algorithm 3 and Algorithm 2, respectively.

Theorems & Definitions (26)

• Definition 1
• Proposition 1
• Definition 2: Privacy Loss Distribution
• Theorem 1
• Proposition 2
• Theorem 2
• Remark 1
• Proposition 3
• Proposition 4
• proof