Prediction with Expert Advice under Local Differential Privacy

TL;DR

The paper tackles prediction with expert advice under local differential privacy, introducing two algorithms—RW-AdaBatch and RW-Meta—that leverage limited switching and private meta-learning to improve privacy-utility trade-offs in dynamic settings. RW-AdaBatch provides privacy amplification via permutation-invariant batching with minimal regret overhead, while RW-Meta privately aggregates among data-dependent learners without increasing privacy cost. The authors establish regret bounds and privacy guarantees for both approaches and validate them on real-world COVID-19 hospitalization data, where RW-Meta outperforms both a non-private baseline and a central-DP competitor by substantial margins. This work demonstrates practical, privacy-preserving advancements for repeated prediction tasks in sensitive domains such as healthcare and human mobility.

Abstract

We study the classic problem of prediction with expert advice under the constraint of local differential privacy (LDP). In this context, we first show that a classical algorithm naturally satisfies LDP and then design two new algorithms that improve it: RW-AdaBatch and RW-Meta. For RW-AdaBatch, we exploit the limited-switching behavior induced by LDP to provide a novel form of privacy amplification that grows stronger on easier data, analogous to the shuffle model in offline learning. Drawing on the theory of random walks, we prove that this improvement carries essentially no utility cost. For RW-Meta, we develop a general method for privately selecting between experts that are themselves non-trivial learning algorithms, and we show that in the context of LDP this carries no extra privacy cost. In contrast, prior work has only considered data-independent experts. We also derive formal regret bounds that scale inversely with the degree of independence between experts. Our analysis is supplemented by evaluation on real-world data reported by hospitals during the COVID-19 pandemic; RW-Meta outperforms both the classical baseline and a state-of-the-art \textit{central} DP algorithm by 1.5-3$\times$ on the task of predicting which hospital will report the highest density of COVID patients each week.

Figures (3)

• Figure 1: Equivalent representations of worst-case privacy amplification for RW-AdaBatch with parameters $\mu=1$, $n=25$, $\Delta=\sqrt{n}$, $\alpha=0.01$. The baseline corresponds to the $G_\mu$ tradeoff curve, solid lines correspond to the Analytic upper bound, and dash-dotted lines correspond to Empirical Monte Carlo simulations. The analytic bound is fairly tight, particularly for small $\delta$/low FPR.
• Figure 2: Results of empirical evaluation on COVID-19 hospitalization data, averaged over 100 iterations. The shaded orange regions enclose the maximum and minimum gain of the 12 rolling regression learners, and the dotted black lines represent the maximum cumulative COVID density of any single hospital in the given state. Static regret can be interpreted as the distance between a learner's total gain and the dotted line, while the regret of RW-Meta corresponds to its distance from the top of the orange envelope. Note that all plots share the same $x$ and $y$ axis.
• Figure : RW-FTPLdevroye2013prediction

Theorems & Definitions (16)

• Theorem 1
• Corollary 1
• Corollary 2
• Theorem 2: Informal
• Lemma 1
• Lemma 2: Joint Concavity of Tradeoff Functions wang2024unified
• Theorem 3
• Lemma 3
• proof
• Lemma 4: Borell-TIS