Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Flamingo: Multi-Round Single-Server Secure Aggregation with Applications to Private Federated Learning

Yiping Ma, Jess Woods, Sebastian Angel, Antigoni Polychroniadou, Tal Rabin

TL;DR

It is shown that Flamingo can securely train a neural network on the (Extended) MNIST and CIFAR-100 datasets, and the model converges without a loss in accuracy, compared to a non-private federated learning system.

Abstract

This paper introduces Flamingo, a system for secure aggregation of data across a large set of clients. In secure aggregation, a server sums up the private inputs of clients and obtains the result without learning anything about the individual inputs beyond what is implied by the final sum. Flamingo focuses on the multi-round setting found in federated learning in which many consecutive summations (averages) of model weights are performed to derive a good model. Previous protocols, such as Bell et al. (CCS '20), have been designed for a single round and are adapted to the federated learning setting by repeating the protocol multiple times. Flamingo eliminates the need for the per-round setup of previous protocols, and has a new lightweight dropout resilience protocol to ensure that if clients leave in the middle of a sum the server can still obtain a meaningful result. Furthermore, Flamingo introduces a new way to locally choose the so-called client neighborhood introduced by Bell et al. These techniques help Flamingo reduce the number of interactions between clients and the server, resulting in a significant reduction in the end-to-end runtime for a full training session over prior work. We implement and evaluate Flamingo and show that it can securely train a neural network on the (Extended) MNIST and CIFAR-100 datasets, and the model converges without a loss in accuracy, compared to a non-private federated learning system.

Flamingo: Multi-Round Single-Server Secure Aggregation with Applications to Private Federated Learning

TL;DR

It is shown that Flamingo can securely train a neural network on the (Extended) MNIST and CIFAR-100 datasets, and the model converges without a loss in accuracy, compared to a non-private federated learning system.

Abstract

This paper introduces Flamingo, a system for secure aggregation of data across a large set of clients. In secure aggregation, a server sums up the private inputs of clients and obtains the result without learning anything about the individual inputs beyond what is implied by the final sum. Flamingo focuses on the multi-round setting found in federated learning in which many consecutive summations (averages) of model weights are performed to derive a good model. Previous protocols, such as Bell et al. (CCS '20), have been designed for a single round and are adapted to the federated learning setting by repeating the protocol multiple times. Flamingo eliminates the need for the per-round setup of previous protocols, and has a new lightweight dropout resilience protocol to ensure that if clients leave in the middle of a sum the server can still obtain a meaningful result. Furthermore, Flamingo introduces a new way to locally choose the so-called client neighborhood introduced by Bell et al. These techniques help Flamingo reduce the number of interactions between clients and the server, resulting in a significant reduction in the end-to-end runtime for a full training session over prior work. We implement and evaluate Flamingo and show that it can securely train a neural network on the (Extended) MNIST and CIFAR-100 datasets, and the model converges without a loss in accuracy, compared to a non-private federated learning system.
Paper Structure (49 sections, 7 theorems, 12 equations, 17 figures)

This paper contains 49 sections, 7 theorems, 12 equations, 17 figures.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Target deployment scenario
  4. Communication model
  5. Failure and threat model
  6. Properties
  7. Background
  8. Cryptographic building blocks
  9. The BBGLR protocol
  10. Using BBGLR for federated learning
  11. Efficient Multi-Round Secure Aggregation
  12. High-level ideas
  13. Types of keys at PKI
  14. Setup phase
  15. Collection phase
  16. ...and 34 more sections

Key Result

Theorem 1

Assume that a PKI and a trusted source of randomness exist, and that the DDH assumption holds. Let the dropout rate of decryptors in the setup phase be bounded by $\delta_D$. If $\delta_D + \eta_D < 1/3$, then under the communication model defined in Section s:comm-model, protocol $\Pi_{\text{setup}

Figures (17)

  • Figure 1: Pseudocode for generating graph $G_t$ in round $t$.
  • Figure 2: Workflow of Flamingo. The server first does a setup for all clients in the system. In each round $t$ of training, the server securely aggregates the masked input vectors in the report step; in the cross-check and reconstruction steps, the server communicates with a small set of randomly chosen clients who serve as decryptors. The decryptors are chosen independently from the set $S_t$ that provides inputs in a given round. Every $R$ rounds, the decryptors switch and the old decryptors transfers shares of $SK$ to new decryptors.
  • Figure 3: Ideal functionality for the setup phase.
  • Figure 4: Ideal functionality for Flamingo.
  • Figure 5: Communication complexity and number of steps (client-server round-trips) of Flamingo and BBGLR for $T$ rounds of aggregation. $N$ is the total number of clients and $n_t$ is the number of clients chosen to participate in round $t$. The number of decryptors is $L$, and the dropout rate of clients in $S_t$ is $\delta$. Let $A$ be the upper bound on the number of neighbors of a client, and let $d$ be the dimension of client's input vector.
  • ...and 12 more figures

Theorems & Definitions (13)

  • Theorem 1: Security of setup phase
  • Lemma 1: Graph connectivity
  • Theorem 2: Dropout resilience of $\Phi_T$
  • Theorem 3: Security of $\Phi_T$
  • Definition 1: DDH assumption
  • Definition 2: ElGamal encryption
  • Definition 3: Authenticated encryption
  • Definition 4: Signature scheme
  • Lemma 2
  • Lemma 3
  • ...and 3 more