Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PriRoAgg: Achieving Robust Model Aggregation with Minimum Privacy Leakage for Federated Learning

Sizai Hou, Songze Li, Tayyebeh Jahani-Nezhad, Giuseppe Caire

TL;DR

PriRoAgg formalizes aggregated privacy and integrates it with robust aggregation in single-server federated learning via Lagrange coded computing, secret-sharing, and verifiable SNIP proofs. It defines ${SRA}_{\Omega}$ to ensure that only aggregated statistics $\sum_{i} \phi_{\Omega}(\mathbf{x}_i)$ are revealed, even under Byzantine-server collusion, and instantiates the framework with two protocols: PriRoAgg with RLR for backdoor defense and PriRoAgg with RFA for model poisoning defense. The authors provide security proofs, complexity analyses, and extensive experiments showing robust performance against diverse attacks with improved efficiency over baselines. The work advances practical secure and robust federated learning by enabling powerful aggregation algorithms to run with provable privacy guarantees in the presence of adversaries. The approach is poised to impact real-world FL deployments where privacy and integrity are both critical.

Abstract

Federated learning (FL) has recently gained significant momentum due to its potential to leverage large-scale distributed user data while preserving user privacy. However, the typical paradigm of FL faces challenges of both privacy and robustness: the transmitted model updates can potentially leak sensitive user information, and the lack of central control of the local training process leaves the global model susceptible to malicious manipulations on model updates. Current solutions attempting to address both problems under the one-server FL setting fall short in the following aspects: 1) designed for simple validity checks that are insufficient against advanced attacks (e.g., checking norm of individual update); and 2) partial privacy leakage for more complicated robust aggregation algorithms (e.g., distances between model updates are leaked for multi-Krum). In this work, we formalize a novel security notion of aggregated privacy that characterizes the minimum amount of user information, in the form of some aggregated statistics of users' updates, that is necessary to be revealed to accomplish more advanced robust aggregation. We develop a general framework PriRoAgg, utilizing Lagrange coded computing and distributed zero-knowledge proof, to execute a wide range of robust aggregation algorithms while satisfying aggregated privacy. As concrete instantiations of PriRoAgg, we construct two secure and robust protocols based on state-of-the-art robust algorithms, for which we provide full theoretical analyses on security and complexity. Extensive experiments are conducted for these protocols, demonstrating their robustness against various model integrity attacks, and their efficiency advantages over baselines.

PriRoAgg: Achieving Robust Model Aggregation with Minimum Privacy Leakage for Federated Learning

TL;DR

PriRoAgg formalizes aggregated privacy and integrates it with robust aggregation in single-server federated learning via Lagrange coded computing, secret-sharing, and verifiable SNIP proofs. It defines to ensure that only aggregated statistics are revealed, even under Byzantine-server collusion, and instantiates the framework with two protocols: PriRoAgg with RLR for backdoor defense and PriRoAgg with RFA for model poisoning defense. The authors provide security proofs, complexity analyses, and extensive experiments showing robust performance against diverse attacks with improved efficiency over baselines. The work advances practical secure and robust federated learning by enabling powerful aggregation algorithms to run with provable privacy guarantees in the presence of adversaries. The approach is poised to impact real-world FL deployments where privacy and integrity are both critical.

Abstract

Federated learning (FL) has recently gained significant momentum due to its potential to leverage large-scale distributed user data while preserving user privacy. However, the typical paradigm of FL faces challenges of both privacy and robustness: the transmitted model updates can potentially leak sensitive user information, and the lack of central control of the local training process leaves the global model susceptible to malicious manipulations on model updates. Current solutions attempting to address both problems under the one-server FL setting fall short in the following aspects: 1) designed for simple validity checks that are insufficient against advanced attacks (e.g., checking norm of individual update); and 2) partial privacy leakage for more complicated robust aggregation algorithms (e.g., distances between model updates are leaked for multi-Krum). In this work, we formalize a novel security notion of aggregated privacy that characterizes the minimum amount of user information, in the form of some aggregated statistics of users' updates, that is necessary to be revealed to accomplish more advanced robust aggregation. We develop a general framework PriRoAgg, utilizing Lagrange coded computing and distributed zero-knowledge proof, to execute a wide range of robust aggregation algorithms while satisfying aggregated privacy. As concrete instantiations of PriRoAgg, we construct two secure and robust protocols based on state-of-the-art robust algorithms, for which we provide full theoretical analyses on security and complexity. Extensive experiments are conducted for these protocols, demonstrating their robustness against various model integrity attacks, and their efficiency advantages over baselines.
Paper Structure (25 sections, 2 theorems, 11 equations, 16 figures, 3 tables)

This paper contains 25 sections, 2 theorems, 11 equations, 16 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Vanilla Federated Learning
  4. Privacy-preserving Model Aggregation
  5. Robust Model Aggregation
  6. Robust and Privacy-preserving Model Aggregation
  7. Challenges and Motivation
  8. Problem Formulation
  9. Building Blocks of PriRoAgg
  10. Key Agreement (KA) and Authenticated Encryption (AE)
  11. Secret Sharing with Lagrange Encoding
  12. Verifiable LCC with Constant-Size Commitments
  13. Secret-shared Non-interactive Proof (SNIP)
  14. Detailed Description of PriRoAgg
  15. General PriRoAgg Framework
  16. ...and 10 more sections

Key Result

Theorem 1

(${SRA}_{RLR}$) For a protocol $\Pi_{RLR}$ that securely implements a ${SRA}_{\Omega}$, assuming an adversary $\mathcal{A}$ who controls a subset $U_M$ of malicious users and gains full access to the local state of the server, and denoting $U_H = [N]\backslash U_M$ as the subset of honest users, the where $Real_{\Pi_{RLR}}$ represents the joint view of $U_M\cup \{server\}$ executing $\Pi_{RLR}$; $

Figures (16)

  • Figure 1: Description of framework PriRoAgg
  • Figure 3: The user and the server overhead breakdown of PriRoAgg per iteration. $N,d,K$ are the number of users, dimension of update, and LCC parameter, respectively.
  • Figure 4: The end-to-end overhead of PriRoAgg compared with EIFFeL. $N,d,K$ are the number of users, dimension of update, and LCC parameter, respectively.
  • Figure : (a) Additive with FMNIST
  • Figure : (a) Additive with FMNIST
  • ...and 11 more figures

Theorems & Definitions (6)

  • Definition 1: Aggregated Privacy
  • Definition 2: ${SRA}_{\Omega}$
  • Theorem 1
  • proof
  • Theorem 2
  • proof