Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On Using Secure Aggregation in Differentially Private Federated Learning with Multiple Local Steps

Mikko A. Heikkilä

TL;DR

The paper tackles privacy-preserving federated learning with fine-grained sample-level DP in the presence of secure aggregation. It introduces a simple yet powerful privacy analysis that enables multiple local optimization steps per FL round while maintaining distributed DP guarantees via a trusted aggregator, leveraging sum-dominating mechanisms such as Gaussian and Skellam noise. Empirically, the approach yields notable utility gains under limited communication rounds across Fashion-MNIST, CIFAR-10, and ACS Income datasets, with improvements of up to around 16 percentage points in some settings. This work narrows the gap between theoretical DP-FL guarantees and practical performance, and provides a framework for deploying DP-FL with SecAgg in real-world, communication-constrained environments.

Abstract

Federated learning is a distributed learning setting where the main aim is to train machine learning models without having to share raw data but only what is required for learning. To guarantee training data privacy and high-utility models, differential privacy and secure aggregation techniques are often combined with federated learning. However, with fine-grained protection granularities, e.g., with the common sample-level protection, the currently existing techniques generally require the parties to communicate for each local optimization step, if they want to fully benefit from the secure aggregation in terms of the resulting formal privacy guarantees. In this paper, we show how a simple new analysis allows the parties to perform multiple local optimization steps while still benefiting from using secure aggregation. We show that our analysis enables higher utility models with guaranteed privacy protection under limited number of communication rounds.

On Using Secure Aggregation in Differentially Private Federated Learning with Multiple Local Steps

TL;DR

The paper tackles privacy-preserving federated learning with fine-grained sample-level DP in the presence of secure aggregation. It introduces a simple yet powerful privacy analysis that enables multiple local optimization steps per FL round while maintaining distributed DP guarantees via a trusted aggregator, leveraging sum-dominating mechanisms such as Gaussian and Skellam noise. Empirically, the approach yields notable utility gains under limited communication rounds across Fashion-MNIST, CIFAR-10, and ACS Income datasets, with improvements of up to around 16 percentage points in some settings. This work narrows the gap between theoretical DP-FL guarantees and practical performance, and provides a framework for deploying DP-FL with SecAgg in real-world, communication-constrained environments.

Abstract

Federated learning is a distributed learning setting where the main aim is to train machine learning models without having to share raw data but only what is required for learning. To guarantee training data privacy and high-utility models, differential privacy and secure aggregation techniques are often combined with federated learning. However, with fine-grained protection granularities, e.g., with the common sample-level protection, the currently existing techniques generally require the parties to communicate for each local optimization step, if they want to fully benefit from the secure aggregation in terms of the resulting formal privacy guarantees. In this paper, we show how a simple new analysis allows the parties to perform multiple local optimization steps while still benefiting from using secure aggregation. We show that our analysis enables higher utility models with guaranteed privacy protection under limited number of communication rounds.
Paper Structure (23 sections, 4 theorems, 25 equations, 3 figures, 1 table, 3 algorithms)

This paper contains 23 sections, 4 theorems, 25 equations, 3 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Our Contribution
  3. Related Work
  4. Background
  5. Differential Privacy
  6. Problem with Local Steps in DPFL with SecAgg
  7. Trust Model
  8. Privacy Accounting for Multiple Local Steps Using a Trusted Aggregator
  9. Experiments
  10. Setup and Motivation:
  11. CNN on Fashion-MNIST:
  12. Linear Classifier on Transformed CIFAR-10:
  13. Logistic Model on Income:
  14. Improved Privacy from Model Averaging:
  15. Discussion
  16. ...and 8 more sections

Key Result

Lemma 4.4

Assume ${\mathcal{A}}^{(s)}$ is a sum-dominating mechanism w.r.t. ${\mathcal{A}}_i^{(s)}, i=1,\dots,N$ for every $s=1,\dots,S$. Then the composition of the sum-dominating mechanisms $({\mathcal{A}}^{(1)}, \dots, {\mathcal{A}}^{(S)})$ dominates the composition

Figures (3)

  • Figure 1: CNN on Fashion-MNIST, 10 clients, mean and SEM over 5 seeds. Running more local steps is clearly beneficial.
  • Figure 2: Mean and SEM over 5 seeds of the best performance over training runs for Linear models on CIFAR-10 using pre-trained ResNeXt29 as feature extractor for varying number of FL rounds, 10 clients. Running more local steps is clearly beneficial.
  • Figure 3: LNN on ACS Income, 51 clients, mean and SEM over 5 seeds. Running more local steps is clearly beneficial.

Theorems & Definitions (15)

  • Definition 3.1
  • Definition 3.2
  • Definition 3.3
  • Definition 4.2: Sum-dominating mechanism
  • Example 4.3: Gaussian mechanism
  • Lemma 4.4
  • proof
  • Lemma 4.5
  • proof
  • Theorem 4.6
  • ...and 5 more