Camel: Communication-Efficient and Maliciously Secure Federated Learning in the Shuffle Model of Differential Privacy
Shuangqing Xu, Yifeng Zheng, Zhongyun Hua
TL;DR
Camel delivers a practical, maliciously secure federated learning framework in the shuffle model of differential privacy by combining gradient perturbation under local DP with a lossless, seed-based gradient compression and a secret-shared shuffle across three non-colluding servers. It strengthens security through integrity checks against online selective failure attacks and fusing shuffle amplification with subsampling, yielding tighter per-iteration and overall privacy bounds via Rényi DP composition. The approach achieves substantial communication and runtime efficiency while improving privacy-utility trade-offs over prior shuffle-model FL methods, demonstrated on MNIST and FMNIST. These advances enable scalable, privacy-preserving FL in adversarial environments with rigorous formal guarantees. The work also outlines possible extensions to broader server settings and multi-message shuffles for further privacy gains.
Abstract
Federated learning (FL) has rapidly become a compelling paradigm that enables multiple clients to jointly train a model by sharing only gradient updates for aggregation, without revealing their local private data. In order to protect the gradient updates which could also be privacy-sensitive, there has been a line of work studying local differential privacy (LDP) mechanisms to provide a formal privacy guarantee. With LDP mechanisms, clients locally perturb their gradient updates before sharing them out for aggregation. However, such approaches are known for greatly degrading the model utility, due to heavy noise addition. To enable a better privacy-utility tradeoff, a recently emerging trend is to apply the shuffle model of DP in FL, which relies on an intermediate shuffling operation on the perturbed gradient updates to achieve privacy amplification. Following this trend, in this paper, we present Camel, a new communication-efficient and maliciously secure FL framework in the shuffle model of DP. Camel first departs from existing works by ambitiously supporting integrity check for the shuffle computation, achieving security against malicious adversary. Specifically, Camel builds on the trending cryptographic primitive of secret-shared shuffle, with custom techniques we develop for optimizing system-wide communication efficiency, and for lightweight integrity checks to harden the security of server-side computation. In addition, we also derive a significantly tighter bound on the privacy loss through analyzing the Renyi differential privacy (RDP) of the overall FL process. Extensive experiments demonstrate that Camel achieves better privacy-utility trade-offs than the state-of-the-art work, with promising performance.