No Vandalism: Privacy-Preserving and Byzantine-Robust Federated Learning

TL;DR

NoV tackles the dual challenges of privacy and Byzantine robustness in federated learning by introducing a privacy-preserving model filter and a verifiable aggregation protocol. The model filter uses a hybrid, layer-wise check with magnitude and directional constraints, protected by non-interactive zero-knowledge proofs (NIZK) to keep local updates private. The aggregation protocol combines verifiable secret sharing with verifiable decryption to detect and remove Byzantine attackers while preserving update confidentiality. Empirical results on EMNIST and CIFAR-10 show NoV effectively defends against data and model poisoning, including PGD-based backdoors, with competitive computational and communication overhead compared to existing schemes.

Abstract

Federated learning allows several clients to train one machine learning model jointly without sharing private data, providing privacy protection. However, traditional federated learning is vulnerable to poisoning attacks, which can not only decrease the model performance, but also implant malicious backdoors. In addition, direct submission of local model parameters can also lead to the privacy leakage of the training dataset. In this paper, we aim to build a privacy-preserving and Byzantine-robust federated learning scheme to provide an environment with no vandalism (NoV) against attacks from malicious participants. Specifically, we construct a model filter for poisoned local models, protecting the global model from data and model poisoning attacks. This model filter combines zero-knowledge proofs to provide further privacy protection. Then, we adopt secret sharing to provide verifiable secure aggregation, removing malicious clients that disrupting the aggregation process. Our formal analysis proves that NoV can protect data privacy and weed out Byzantine attackers. Our experiments illustrate that NoV can effectively address data and model poisoning attacks, including PGD, and outperforms other related schemes.

Figures (9)

• Figure 1: The system and threat model for NoV.
• Figure 2: Observations on malicious models and benign models.
• Figure 3: The workflow of the Model aggregation protocol.
• Figure 4: The description of the verifiable privacy-preserving aggregation protocol in NoV.
• Figure 5: Comparison of Euclidean distance among honest and malicious model updates on different models and tasks.