Estimating the True Distribution of Data Collected with Randomized Response

TL;DR

This work derives an exact, efficiently computable Maximum Likelihood Estimator (MLE*) for recovering the true data distribution under Randomized Response with local differential privacy. It formalizes the problem, compares likelihood-based methods to expectation-based debiasing, and proves the uniqueness and validity of the MLE*, providing an $O(K\log K)$ algorithm. Empirically, MLE* demonstrates robust performance across diverse data regimes and privacy settings, often achieving the best or near-best fit in terms of Negative Log-Likelihood while maintaining consistency. The results offer practical guidance for practitioners deploying RR and motivate extensions to other LDP frequency-estimation protocols, with a public codebase for reproducibility.

Abstract

Randomized Response (RR) is a protocol designed to collect and analyze categorical data with local differential privacy guarantees. It has been used as a building block of mechanisms deployed by Big tech companies to collect app or web users' data. Each user reports an automatic random alteration of their true value to the analytics server, which then estimates the histogram of the true unseen values of all users using a debiasing rule to compensate for the added randomness. A known issue is that the standard debiasing rule can yield a vector with negative values (which can not be interpreted as a histogram), and there is no consensus on the best fix. An elegant but slow solution is the Iterative Bayesian Update algorithm (IBU), which converges to the Maximum Likelihood Estimate (MLE) as the number of iterations goes to infinity. This paper bypasses IBU by providing a simple formula for the exact MLE of RR and compares it with other estimation methods experimentally to help practitioners decide which one to use.

Figures (10)

• Figure 1: Performance of MLE*, InvP, and InvN across different data distributions ($s \in \{0.01, 1.3, 5.0\}$) for fixed $K=10{,}000$ and $\epsilon=4.0$.
• Figure 2: Performance of MLE*, InvP, and InvN across different data distributions ($s \in \{0.01, 1.3, 5.0\}$) for fixed $K=10{,}000$ and $N=10^6$.
• Figure 3: Performance of MLE*, InvP, and InvN across three different real-world distributions.
• Figure 4: The squared error of the IBU's estimates relative to MLE*, with the number of iterations.
• Figure 5: MSE results for near-uniform distribution ($s = 0.01$). Rows vary domain size $K$, and columns vary privacy level $\epsilon$. The far-left subplot of each row shows the true histogram.

Theorems & Definitions (30)

• Theorem 1
• Theorem 2
• proof
• Theorem 3
• proof
• Theorem 4
• proof
• Theorem 5
• proof
• Lemma 1