Privacy-Preserving Aggregation for Decentralized Learning with Byzantine-Robustness

TL;DR

SecureDL addresses the privacy risks and convergence challenges of Byzantine-robust decentralized learning by introducing a privacy-preserving aggregation rule based on secure multiparty computation. It combines cosine similarity and L2 normalization with secret sharing, Beaver triples, and a secure comparison protocol to identify and discard malicious updates without exposing individual models. The authors provide convergence guarantees under standard assumptions and perform comprehensive experiments on MNIST, Fashion-MNIST, SVHN, and CIFAR-10, showing strong resilience against diverse Byzantine attacks and favorable scalability versus non-private baselines. The work demonstrates practical privacy-preserving protection for collaborative learning without a central aggregator, offering a meaningful step toward secure, decentralized AI deployments.

Abstract

Decentralized machine learning (DL) has been receiving an increasing interest recently due to the elimination of a single point of failure, present in Federated learning setting. Yet, it is threatened by the looming threat of Byzantine clients who intentionally disrupt the learning process by broadcasting arbitrary model updates to other clients, seeking to degrade the performance of the global model. In response, robust aggregation schemes have emerged as promising solutions to defend against such Byzantine clients, thereby enhancing the robustness of Decentralized Learning. Defenses against Byzantine adversaries, however, typically require access to the updates of other clients, a counterproductive privacy trade-off that in turn increases the risk of inference attacks on those same model updates. In this paper, we introduce SecureDL, a novel DL protocol designed to enhance the security and privacy of DL against Byzantine threats. SecureDL~facilitates a collaborative defense, while protecting the privacy of clients' model updates through secure multiparty computation. The protocol employs efficient computation of cosine similarity and normalization of updates to robustly detect and exclude model updates detrimental to model convergence. By using MNIST, Fashion-MNIST, SVHN and CIFAR-10 datasets, we evaluated SecureDL against various Byzantine attacks and compared its effectiveness with four existing defense mechanisms. Our experiments show that SecureDL is effective even in the case of attacks by the malicious majority (e.g., 80% Byzantine clients) while preserving high training accuracy.

Figures (3)

• Figure 1: Collaborative learning system in a Federated (left) and decentralized fashion.
• Figure 2: The illustration of SecureDL aggregation rule. The model update of $Client_{0}$, denoted as $w_0$, obtains the updates from other clients ($w_1$, $w_2$, and $w_3$). In image A, the protocol discards any model update with a negative cosine value. Image B depicts $Client_0$ normalizing the received models based on its own model's magnitude. ImageC shows the final accepted model update after normalization.
• Figure 3: Implications of varying the fraction of malicious clients in different attack scenarios on MNIST dataset.

Theorems & Definitions (16)

• Theorem 1
• proof : Proof of Theorem \ref{['convergenceproof']}
• Theorem 2: Privacy w.r.t Semi-Honest Behavior
• proof : Proof of Theorem \ref{['securedlproof']}
• Theorem 3: privacy w.r.t semi-honest behavior
• proof
• Theorem 4: privacy w.r.t semi-honest behavior
• proof
• Theorem 5: privacy w.r.t semi-honest behavior
• proof