Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Privacy-Preserving, Dropout-Resilient Aggregation in Decentralized Learning

Ali Reza Ghavamipour, Benjamin Zi Hao Zhao, Fatih Turkmen

TL;DR

This work addresses the privacy and dropout challenges of decentralized learning by introducing three secret-sharing-based aggregation protocols that are dropout-resilient. It combines Shamir's Secret Sharing, LWE-based masking, and pairwise masking with DH key exchange to enable secure, scalable aggregation without a central server. Across MNIST, Fashion-MNIST, SVHN, and CIFAR-10, the methods achieve strong privacy guarantees while maintaining high accuracy, with PPDL-NV and PPDL-PW closely approaching the centralized baseline and PPDL-LWE offering robust privacy with competitive efficiency. The study demonstrates that privacy-preserving, dropout-tolerant decentralized learning can operate efficiently at scale, outperforming traditional secure aggregation approaches in many scenarios and providing practical guidance for deploying DL in privacy-sensitive, unreliable networks.

Abstract

Decentralized learning (DL) offers a novel paradigm in machine learning by distributing training across clients without central aggregation, enhancing scalability and efficiency. However, DL's peer-to-peer model raises challenges in protecting against inference attacks and privacy leaks. By forgoing central bottlenecks, DL demands privacy-preserving aggregation methods to protect data from 'honest but curious' clients and adversaries, maintaining network-wide privacy. Privacy-preserving DL faces the additional hurdle of client dropout, clients not submitting updates due to connectivity problems or unavailability, further complicating aggregation. This work proposes three secret sharing-based dropout resilience approaches for privacy-preserving DL. Our study evaluates the efficiency, performance, and accuracy of these protocols through experiments on datasets such as MNIST, Fashion-MNIST, SVHN, and CIFAR-10. We compare our protocols with traditional secret-sharing solutions across scenarios, including those with up to 1000 clients. Evaluations show that our protocols significantly outperform conventional methods, especially in scenarios with up to 30% of clients dropout and model sizes of up to $10^6$ parameters. Our approaches demonstrate markedly high efficiency with larger models, higher dropout rates, and extensive client networks, highlighting their effectiveness in enhancing decentralized learning systems' privacy and dropout robustness.

Privacy-Preserving, Dropout-Resilient Aggregation in Decentralized Learning

TL;DR

This work addresses the privacy and dropout challenges of decentralized learning by introducing three secret-sharing-based aggregation protocols that are dropout-resilient. It combines Shamir's Secret Sharing, LWE-based masking, and pairwise masking with DH key exchange to enable secure, scalable aggregation without a central server. Across MNIST, Fashion-MNIST, SVHN, and CIFAR-10, the methods achieve strong privacy guarantees while maintaining high accuracy, with PPDL-NV and PPDL-PW closely approaching the centralized baseline and PPDL-LWE offering robust privacy with competitive efficiency. The study demonstrates that privacy-preserving, dropout-tolerant decentralized learning can operate efficiently at scale, outperforming traditional secure aggregation approaches in many scenarios and providing practical guidance for deploying DL in privacy-sensitive, unreliable networks.

Abstract

Decentralized learning (DL) offers a novel paradigm in machine learning by distributing training across clients without central aggregation, enhancing scalability and efficiency. However, DL's peer-to-peer model raises challenges in protecting against inference attacks and privacy leaks. By forgoing central bottlenecks, DL demands privacy-preserving aggregation methods to protect data from 'honest but curious' clients and adversaries, maintaining network-wide privacy. Privacy-preserving DL faces the additional hurdle of client dropout, clients not submitting updates due to connectivity problems or unavailability, further complicating aggregation. This work proposes three secret sharing-based dropout resilience approaches for privacy-preserving DL. Our study evaluates the efficiency, performance, and accuracy of these protocols through experiments on datasets such as MNIST, Fashion-MNIST, SVHN, and CIFAR-10. We compare our protocols with traditional secret-sharing solutions across scenarios, including those with up to 1000 clients. Evaluations show that our protocols significantly outperform conventional methods, especially in scenarios with up to 30% of clients dropout and model sizes of up to parameters. Our approaches demonstrate markedly high efficiency with larger models, higher dropout rates, and extensive client networks, highlighting their effectiveness in enhancing decentralized learning systems' privacy and dropout robustness.
Paper Structure (28 sections, 3 theorems, 2 equations, 3 figures, 2 tables, 4 algorithms)

This paper contains 28 sections, 3 theorems, 2 equations, 3 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Preliminaries
  4. Learning in a Decentralized Setting
  5. Shamir's Secret Sharing
  6. Diffie-Hellman Key Agreement Protocol
  7. Problem Definition
  8. Threat model
  9. Approaches
  10. SA using Shamir's secret sharing
  11. Security Analysis
  12. SA using LWE-Based Masking
  13. Security Analysis
  14. SA with pairwise masking
  15. Security Analysis
  16. ...and 13 more sections

Key Result

Theorem 1

The $PPDL\xspace-NV\xspace$ protocol, which utilizes SSS to distribute model updates as $n$ shares within a finite field $\mathbb{F}_q$ among clients, ensures security against semi-honest adversaries and exhibits resilience to client dropout in the honest majority setting.

Figures (3)

  • Figure 1: Collaborative learning system in a federated (left) and decentralized fashion when Client 4 has dropped out.
  • Figure 2: Implications of increasing the number of clients on the efficiency of protocols.
  • Figure 3: Implications of increasing the number of model parameters on the efficiency of protocols.

Theorems & Definitions (3)

  • Theorem 1
  • Theorem 2
  • Theorem 3