The Limits of Differential Privacy in Online Learning
Bo Li, Wei Wang, Peng Ye
TL;DR
This work characterizes the limits of differential privacy in online learning, showing a sharp separation between no DP, pure DP, and approximate DP. It proves that approximate DP is necessary to handle adaptive adversaries, while pure DP forces the number of mistakes to grow at least logarithmically with the horizon for many classes, and even to infinity for infinite horizons in some cases. By linking learnability to the Littlestone dimension and representation dimension, the authors establish universal lower bounds $\Omega(\log T)$ and $\Omega(\mathrm{LD}(\mathcal{H}))\log T$, and they compare these with existing upper bounds, highlighting gaps and guiding future research on private online learning. The results have implications for private online inference in adversarial contexts and clarify the cost of privacy in sequential decision-making tasks.
Abstract
Differential privacy (DP) is a formal notion that restricts the privacy leakage of an algorithm when running on sensitive data, in which privacy-utility trade-off is one of the central problems in private data analysis. In this work, we investigate the fundamental limits of differential privacy in online learning algorithms and present evidence that separates three types of constraints: no DP, pure DP, and approximate DP. We first describe a hypothesis class that is online learnable under approximate DP but not online learnable under pure DP under the adaptive adversarial setting. This indicates that approximate DP must be adopted when dealing with adaptive adversaries. We then prove that any private online learner must make an infinite number of mistakes for almost all hypothesis classes. This essentially generalizes previous results and shows a strong separation between private and non-private settings since a finite mistake bound is always attainable (as long as the class is online learnable) when there is no privacy requirement.