DPSR: Differentially Private Sparse Reconstruction via Multi-Stage Denoising for Recommender Systems
Sarwan Ali
TL;DR
This work addresses the privacy-utility tradeoff in recommender systems by introducing DPSR, a three-stage denoising pipeline that preserves differential privacy through post-processing. DPSR adaptively calibrates noise based on information content, denoises via collaborative filtering, and leverages low-rank matrix completion to recover clean signals. The framework demonstrates superior RMSE and ranking metrics compared to Laplace and Gaussian baselines across privacy budgets, even outperforming non-private baselines at moderate privacy levels due to effective regularization. Theoretical guarantees include ε-differential privacy via post-processing and a utility bound decomposed into privacy noise, approximation error, and inherent data noise, with practical runtimes suitable for typical-scale recommender systems.
Abstract
Differential privacy (DP) has emerged as the gold standard for protecting user data in recommender systems, but existing privacy-preserving mechanisms face a fundamental challenge: the privacy-utility tradeoff inevitably degrades recommendation quality as privacy budgets tighten. We introduce DPSR (Differentially Private Sparse Reconstruction), a novel three-stage denoising framework that fundamentally addresses this limitation by exploiting the inherent structure of rating matrices -- sparsity, low-rank properties, and collaborative patterns. DPSR consists of three synergistic stages: (1) \textit{information-theoretic noise calibration} that adaptively reduces noise for high-information ratings, (2) \textit{collaborative filtering-based denoising} that leverages item-item similarities to remove privacy noise, and (3) \textit{low-rank matrix completion} that exploits latent structure for signal recovery. Critically, all denoising operations occur \textit{after} noise injection, preserving differential privacy through the post-processing immunity theorem while removing both privacy-induced and inherent data noise. Through extensive experiments on synthetic datasets with controlled ground truth, we demonstrate that DPSR achieves 5.57\% to 9.23\% RMSE improvement over state-of-the-art Laplace and Gaussian mechanisms across privacy budgets ranging from $\varepsilon=0.1$ to $\varepsilon=10.0$ (all improvements statistically significant with $p < 0.05$, most $p < 0.001$). Remarkably, at $\varepsilon=1.0$, DPSR achieves RMSE of 0.9823, \textit{outperforming even the non-private baseline} (1.0983), demonstrating that our denoising pipeline acts as an effective regularizer that removes data noise in addition to privacy noise.