Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On Model Protection in Federated Learning against Eavesdropping Attacks

Dipankar Maity, Kushal Chakrabarti

TL;DR

This work addresses protecting the client model in federated learning from eavesdropping on uplink updates. It develops a principled framework to quantify model protection, contrasting update-based FLIP with full-model FLOP communications and drawing comparisons to differential privacy. A key theoretical result provides a lower bound on asymptotic protection that reveals how protection depends on client-sampling probability $p$, eavesdropping probability $p_e$, and misalignment terms, with a stability condition on a proxy matrix $M$. Empirical results on CIFAR-10 with LeNet-5 corroborate the theory, showing substantial protection for FLIP while maintaining utility, and illustrating that DP-FL is less protective for the specific threat model considered. The findings suggest that sharing model increments can significantly strengthen FL against eavesdropping without the accuracy loss typical of DP-based approaches in this context.

Abstract

In this study, we investigate the protection offered by federated learning algorithms against eavesdropping adversaries. In our model, the adversary is capable of intercepting model updates transmitted from clients to the server, enabling it to create its own estimate of the model. Unlike previous research, which predominantly focuses on safeguarding client data, our work shifts attention protecting the client model itself. Through a theoretical analysis, we examine how various factors, such as the probability of client selection, the structure of local objective functions, global aggregation at the server, and the eavesdropper's capabilities, impact the overall level of protection. We further validate our findings through numerical experiments, assessing the protection by evaluating the model accuracy achieved by the adversary. Finally, we compare our results with methods based on differential privacy, underscoring their limitations in this specific context.

On Model Protection in Federated Learning against Eavesdropping Attacks

TL;DR

This work addresses protecting the client model in federated learning from eavesdropping on uplink updates. It develops a principled framework to quantify model protection, contrasting update-based FLIP with full-model FLOP communications and drawing comparisons to differential privacy. A key theoretical result provides a lower bound on asymptotic protection that reveals how protection depends on client-sampling probability , eavesdropping probability , and misalignment terms, with a stability condition on a proxy matrix . Empirical results on CIFAR-10 with LeNet-5 corroborate the theory, showing substantial protection for FLIP while maintaining utility, and illustrating that DP-FL is less protective for the specific threat model considered. The findings suggest that sharing model increments can significantly strengthen FL against eavesdropping without the accuracy loss typical of DP-based approaches in this context.

Abstract

In this study, we investigate the protection offered by federated learning algorithms against eavesdropping adversaries. In our model, the adversary is capable of intercepting model updates transmitted from clients to the server, enabling it to create its own estimate of the model. Unlike previous research, which predominantly focuses on safeguarding client data, our work shifts attention protecting the client model itself. Through a theoretical analysis, we examine how various factors, such as the probability of client selection, the structure of local objective functions, global aggregation at the server, and the eavesdropper's capabilities, impact the overall level of protection. We further validate our findings through numerical experiments, assessing the protection by evaluating the model accuracy achieved by the adversary. Finally, we compare our results with methods based on differential privacy, underscoring their limitations in this specific context.

Paper Structure

This paper contains 10 sections, 4 theorems, 54 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Federated Learning Architecture
  4. Adversary Eavesdropping Model & Estimation Dynamics
  5. Protection Analysis
  6. Perfect Eavesdropping Case: $\mu_t =1$ almost surely
  7. FL with Clients Sending Local Models (FLOP)
  8. Experimental Results
  9. Conclusion
  10. Some Useful Results

Key Result

Lemma 1

A necessary condition for the stability of $\Sigma_t$ is that all the eigenvalues of $M$ must have a magnitude less than $(p(1-\gamma)\max\{\gamma, 1-\gamma\})^{-\frac{1}{2}}$.

Figures (4)

  • Figure 1: CIFAR-10 test set accuracies, of (i) adversary's estimated model ${x}^{\rm a}_t$ with different initialization and (ii) client's true model $x^{\rm c}_t$, when LeNet-5 is trained using FLIP. FL settings: (a) $n=5, K = 3, B = 128$, (b) $n=8, K = 3, B = 128$, (c) $n=5, K = 10, B = 128$, (d) $n=5, K = 3, B = 8$.
  • Figure 2: Accuracies on reconstructed client's training images, of (i) adversary's estimated model ${x}^{\rm a}_t$ in case of FLIP and (ii) adversary's true intercepted model $x^{\rm c}_t$ in case of DP-FL. FL settings are the same as Fig. \ref{['fig:advclient']}.
  • Figure 3: Adversary's reconstructed training samples, in case of learning with (i) FLIP and (ii) DP-FL. FL settings is $n=5, K = 3, B = 8$.
  • Figure 4: CIFAR-10 test set accuracies of client's true model $x^{\rm c}_t$, when LeNet-5 is trained using DP-FL. FL settings are the same as Fig. \ref{['fig:advclient']}.

Theorems & Definitions (10)

  • Remark 1
  • Lemma 1
  • proof
  • Remark 2
  • Theorem 1
  • proof
  • Proposition 1
  • proof
  • Proposition 2
  • proof