Compressed Private Aggregation for Scalable and Robust Federated Learning over Massive Networks
Natalie Lang, Nir Shlezinger, Rafael G. L. D'Oliveira, Salim El Rouayheb
TL;DR
This paper tackles privacy, robustness, and communication bottlenecks in Federated Learning over massive networks by introducing Compressed Private Aggregation (CPA). CPA uses nested lattice quantizers and random codebooks to encode local updates into very few bits, while applying randomized response to meet $\varepsilon$-LD and constructing a $k$-anonymous representation that resists Byzantine poisoning. The authors prove that CPA preserves the FL convergence rate at order $O(1/t)$ in the amortized sense, with a per-round distortion that diminishes as the number of participating users $K$ grows, and validate these properties with extensive experiments on MNIST and CIFAR-10 showing competitive accuracy against non-private baselines and robustness to malicious participants. Overall, CPA offers a scalable, privacy-preserving, and Byzantine-robust framework for federated learning in large-scale deployments, enabling practical deployments in massive networks with minimal communication overhead.
Abstract
Federated learning (FL) is an emerging paradigm that allows a central server to train machine learning models using remote users' data. Despite its growing popularity, FL faces challenges in preserving the privacy of local datasets, its sensitivity to poisoning attacks by malicious users, and its communication overhead. The latter is additionally considerably dominant in large-scale networks. These limitations are often individually mitigated by local differential privacy (LDP) mechanisms, robust aggregation, compression, and user selection techniques, which typically come at the cost of accuracy. In this work, we present compressed private aggregation (CPA), that allows massive deployments to simultaneously communicate at extremely low bit rates while achieving privacy, anonymity, and resilience to malicious users. CPA randomizes a codebook for compressing the data into a few bits using nested lattice quantizers, while ensuring anonymity and robustness, with a subsequent perturbation to hold LDP. The proposed CPA is proven to result in FL convergence in the same asymptotic rate as FL without privacy, compression, and robustness considerations, while satisfying both anonymity and LDP requirements. These analytical properties are empirically confirmed in a numerical study, where we demonstrate the performance gains of CPA compared with separate mechanisms for compression and privacy for training different image classification models, as well as its robustness in mitigating the harmful effects of malicious users.