Nonparametric extensions of randomized response for private confidence sets

TL;DR

The paper addresses private, nonparametric inference for population means under local differential privacy by developing NPRR, a sequentially interactive generalization of Warner's randomized response for bounded data. It derives explicit locally private confidence sets and confidence sequences for both fixed and time-varying means, including Hoeffding-type and variance-adaptive constructions, and extends to private online A/B testing. The framework combines NPRR with martingale techniques to achieve time-uniform guarantees and sharp finite-sample bounds, while highlighting practical advantages over Laplace-based approaches. The results enable anytime-valid, privacy-preserving sequential inference and testing, with demonstrations in private online experimentation and guidance for parameter choices. The work contributes a cohesive toolkit for private nonparametric sequential inference and sets the stage for further extensions to multivariate settings and adaptive privacy-control strategies.

Abstract

This work derives methods for performing nonparametric, nonasymptotic statistical inference for population means under the constraint of local differential privacy (LDP). Given bounded observations $(X_1, \dots, X_n)$ with mean $μ^\star$ that are privatized into $(Z_1, \dots, Z_n)$, we present confidence intervals (CI) and time-uniform confidence sequences (CS) for $μ^\star$ when only given access to the privatized data. To achieve this, we study a nonparametric and sequentially interactive generalization of Warner's famous ``randomized response'' mechanism, satisfying LDP for arbitrary bounded random variables, and then provide CIs and CSs for their means given access to the resulting privatized observations. For example, our results yield private analogues of Hoeffding's inequality in both fixed-time and time-uniform regimes. We extend these Hoeffding-type CSs to capture time-varying (non-stationary) means, and conclude by illustrating how these methods can be used to conduct private online A/B tests.

Figures (7)

• Figure 1: Widths of private 90%-CIs for the mean of a uniform distribution using our private Hoeffding CI given in \ref{['eq:hoeffding-ci-simple-case']} for various levels of $\varepsilon$ ranging from $\varepsilon = 0.5$ (very private) to $\varepsilon = \infty$ (no privacy). Unsurprisingly, less privacy leads to sharper inference, but notice that inference is still practical, especially for $\varepsilon \geq 2$. For context, Apple uses $\varepsilon \in \{2,4,8\}$ for various data collection tasks on iPhones appleEps. At these levels of privacy, our CIs perform nearly as well as --- and are in some cases indistinguishable from --- the non-private Hoeffding CI.
• Figure 2: An illustration of how a distribution $Q \in \mathcal{Q}_{\mu^\star}^n$ can arise from applying NPRR with $G_t = 4$ to draws from the input distribution $P \in \mathcal{P}_{\mu^\star}^n$. Raw data $X_t$ are discretized into $Y_t$ so that $Y_t$ has finite support but so that $\mu^\star = \mathbb E (X_t) = \mathbb E(Y_t)$. The discrete $Y_t$ are then privatized into $Z_t$ with conditional mean $\mathbb E (Z_t \mid Z_1^{t-1}) = \zeta_t(\mu^\star) = r_t\mu^\star + (1-r_t)/2$ by being mixed with independent uniform noise $\mathcal{U}_t \sim {\mathrm{Unif}\{0, 1/4, 1/2, 3/4, 1\}}$.
• Figure 3: Two $(90\%,2)$-LPCIs: $\dot L_n^H$ given in \ref{['eq:hoeffding-ci-simple-case']} and $\dot L_n^{H+}$ given in \ref{['eq:nprr-h-ci-tighter']} --- i.e. these are $(1-\alpha,\varepsilon)$-LPCIs with $\alpha=0.1$ and $\varepsilon=2$. Notice that the latter can be tighter than the former. Indeed this is because $L_n^{H+}$ is never looser than $L_n^H$ (by definition) but strictly tighter with positive probability.
• Figure 4: Widths of $(90\%,2)$-LPCIs for the mean of a Beta(50, 50) distribution. Hoeffding-based methods (Lap-H and NPRR-H found in \ref{['corollary:laplace-hoeffding-ci']} and \ref{['theorem:hoeffding-nprr-ci']}) do slightly worse than the variance-adaptive ones (NPRR-EB and NPRR-hedged in \ref{['proposition:ldp-eb-ci']} and \ref{['theorem:ldp-hedged-ci']}), but in all cases, CIs that rely on NPRR seem to outperform Lap-H in both small and large $n$ regimes.
• Figure 5: Widths of (90%, 2)-LPCSs for the mean of a Beta(50, 50) distribution. Like Figure \ref{['fig:ci']}, Hoeffding-based methods (Lap-H-CS and NPRR-H-CS found in \ref{['proposition:laplace-hoeffding-cs']} and \ref{['theorem:ldp-hoeffding-cs']}) do worse than the variance-adaptive ones (NPRR-EB-CS and NPRR-GK-CS in \ref{['proposition:ldp-eb-cs']} and \ref{['theorem:ldp-dkelly']}) for large $t$, though NPRR-H-CS does outperform NPRR-EB-CS for small $t$. Nevertheless, in all cases, we find that NPRR-based CSs outperform Lap-H-CS in both small and large $t$ regimes.

Theorems & Definitions (45)

• Theorem 1: satisfies LDP
• Remark 2: Who chooses $\varepsilon_t$, $r_t$, or $G_t$, and how?
• Definition 3: Locally private confidence sets
• Theorem 4: NPRR-H
• Remark 5: Minimax rate optimality of \ref{['eq:hoeffding-ci-simple-case']}
• Remark 6: The relationship between $\varepsilon$ and \ref{['eq:hoeffding-ci-simple-case']} for practical levels of privacy
• Remark 7
• Theorem 8: NPRR-H-CS
• Theorem 9: Confidence sequences for time-varying means
• Corollary 1: Locally private online A/B estimation