Randomized Response with Gradual Release of Privacy Budget

TL;DR

The paper addresses the challenge of gradually relaxing local differential privacy for randomized response while preserving utility. It introduces a rigorous framework that ensures outputs at each relaxation step follow the distribution of an appropriately parameterized $\epsilon_i$-LDP randomized response and that the overall process maintains the latest DP budget under composition. The authors provide closed-form solutions for binary and polychotomous cases, extend the approach to continual relaxation, and prove collusion-proof characteristics. They demonstrate practical applicability by integrating into RAPPOR, enabling mean estimation, and enabling a privacy-budget-aware data market, with empirical validation showing tight adherence to theoretical guarantees and improved utility over naive relaxation methods.

Abstract

An algorithm is developed to gradually relax the Differential Privacy (DP) guarantee of a randomized response. The output from each relaxation maintains the same probability distribution as a standard randomized response with the equivalent DP guarantee, ensuring identical utility as the standard approach. The entire relaxation process is proven to have the same DP guarantee as the most recent relaxed guarantee. The DP relaxation algorithm is adaptable to any Local Differential Privacy (LDP) mechanisms relying on randomized response. It has been seamlessly integrated into RAPPOR, an LDP crowdsourcing string-collecting tool, to optimize the utility of estimating the frequency of collected data. Additionally, it facilitates the relaxation of the DP guarantee for mean estimation based on randomized response. Finally, numerical experiments have been conducted to validate the utility and DP guarantee of the algorithm.

Figures (3)

• Figure 1: (a) illustrates the DP guarantee of repeated noisy samplings with $\epsilon_\alpha = 1$ and varying $\epsilon_\beta$ as indicated in the legend; (b) depicts the variance in the estimated frequency of the original bit $B = 1$ with $\epsilon_\alpha = 1$ and various $\epsilon_\beta$ values specified in the legend. Solid curves represent repeated noisy samplings, while dashed curves depict the DP relaxation of randomized responses; (c) showcases the variance in the estimated frequency of the original bit $B = 1$ derived from randomized responses with DP guarantee $\epsilon$ relaxed from 0.1 to 1.0 linearly. Different curves denote different rounds to complete relaxation, labeled as $K$. Both (b) and (c) use $N = 1$ when calculating the variance.
• Figure 2: (a) displays the mean and confidence interval ($2 \sigma$) of the estimated frequency of the objects being one in the first experiment. Dashed curves represent the theoretical values; (b) illustrates the variance of the estimated frequency for both the DP relaxation of randomized responses and repeated noisy sampling with the same DP guarantee. The dashed curve represents the theoretical variance of DP relaxation. (c) shows the mean and error bars of the error rates for different inference methods inferring the true values of the objects in the first experiment. The dashed line represents the theoretical minimum. LO, MLE, HF, and WHF stand for Last Output, Maximum Likelihood Estimation, Highest Frequency, and Weighted Highest Frequency, respectively.
• Figure 3: (a) - (e) present the mean and confidence interval ($2 \sigma$) of the estimated frequency of the object in the second experiment being one to five, respectively. Dash curves represent the theoretical values; (f) depicts the mean and error bars of the error rates for different inference methods inferring the true values of the objects in the second experiment. The dashed line represents the theoretical minimum. LO, MLE, HF, and WHF stand for Last Output, Maximum Likelihood Estimation, Highest Frequency, and Weighted Highest Frequency, respectively.

Theorems & Definitions (8)

• Theorem 3.1
• Theorem 3.2
• Theorem 3.3
• Theorem 3.4
• Lemma 3.1
• Theorem 3.5
• Theorem 4.1
• Theorem 6.1