Privacy in Metalearning and Multitask Learning: Modeling and Separations
Maryam Aliakbarpour, Konstantina Bairaktari, Adam Smith, Marika Swanberg, Jonathan Ullman
TL;DR
The paper investigates privacy in personalized learning by framing a taxonomy of private learning frameworks for multitask and metalearning under differential privacy. It proves a key equivalence: private billboard multitask learning implies private metalearning, and it separates DP billboard/JDP/1-out-of-$t$ DP across both estimation and classification tasks using indexed mean estimation and the fingerprinting lemma. The analysis centers on sample complexity and how privacy constraints scale with the number of tasks $t$, dimensions $d$, and privacy parameters, revealing fundamental trade-offs and separations between privacy models. The results provide both upper and lower bounds for indexed mean estimation and indexed classification, clarifying when personalization under privacy is feasible and how information can be safeguarded in the billboard versus individualized-output settings, with implications for private federated or collaborative learning systems.
Abstract
Model personalization allows a set of individuals, each facing a different learning task, to train models that are more accurate for each person than those they could develop individually. The goals of personalization are captured in a variety of formal frameworks, such as multitask learning and metalearning. Combining data for model personalization poses risks for privacy because the output of an individual's model can depend on the data of other individuals. In this work we undertake a systematic study of differentially private personalized learning. Our first main contribution is to construct a taxonomy of formal frameworks for private personalized learning. This taxonomy captures different formal frameworks for learning as well as different threat models for the attacker. Our second main contribution is to prove separations between the personalized learning problems corresponding to different choices. In particular, we prove a novel separation between private multitask learning and private metalearning.