RPWithPrior: Label Differential Privacy in Regression
Haixia Liu, Ruifan Huang
TL;DR
This work tackles regression under $\epsilon$-label differential privacy by moving from discretized outputs to a continuous-output framework, RPWithPrior, which jointly models $Y$ and $\tilde{Y}$ and optimizes an interval $[A_1-\zeta, A_2+\zeta]$ to maximize neighborhood preservation while satisfying $\epsilon$-label DP. In the known-prior setting, the authors derive the optimal conditional density with $f_{\tilde{Y}|Y}(\tilde{y}|y)$ equal to $1/\gamma$ on $\mathcal{N}_y$ and $e^{-\epsilon}/\gamma$ on $\mathcal{N}_{\mathcal{I}}\setminus\mathcal{N}_y$, where $\gamma=2\zeta+e^{-\epsilon}(A_2-A_1)$, and provide an efficient algorithm to compute the optimal interval; the method extends to unknown priors via a histogram-based estimator Hist_ε and a budget-splitting scheme $\epsilon_1+\epsilon_2$. Empirical results on three real datasets show that RPWithPrior yields lower test MSE and competitive runtimes compared with Gaussian, Laplace, Staircase, RRonBins, and Unbiased mechanisms, indicating improved privacy-utility trade-offs for continuous regression. The approach eliminates discretization errors common in prior work and offers practical utility under label DP, with clear pathways for known- and unknown-prior applications in privacy-preserving regression. Overall, the work advances label DP for regression by enabling continuous outputs, formal privacy guarantees, and scalable, prior-aware algorithms with strong empirical performance.
Abstract
With the wide application of machine learning techniques in practice, privacy preservation has gained increasing attention. Protecting user privacy with minimal accuracy loss is a fundamental task in the data analysis and mining community. In this paper, we focus on regression tasks under $ε$-label differential privacy guarantees. Some existing methods for regression with $ε$-label differential privacy, such as the RR-On-Bins mechanism, discretized the output space into finite bins and then applied RR algorithm. To efficiently determine these finite bins, the authors rounded the original responses down to integer values. However, such operations does not align well with real-world scenarios. To overcome these limitations, we model both original and randomized responses as continuous random variables, avoiding discretization entirely. Our novel approach estimates an optimal interval for randomized responses and introduces new algorithms designed for scenarios where a prior is either known or unknown. Additionally, we prove that our algorithm, RPWithPrior, guarantees $ε$-label differential privacy. Numerical results demonstrate that our approach gets better performance compared with the Gaussian, Laplace, Staircase, and RRonBins, Unbiased mechanisms on the Communities and Crime, Criteo Sponsored Search Conversion Log, California Housing datasets.