Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mitigating Membership Inference Vulnerability in Personalized Federated Learning

Kangsoo Jung, Sayan Biswas, Catuscia Palamidessi

TL;DR

IFCA-MIR addresses the privacy vulnerability of clustering-based Personalized Federated Learning by integrating MIA risk into cluster selection, enabling clients to balance accuracy and privacy. The approach uses a server-side red team to estimate MIA risk for each cluster and delivers privacy scores to clients, who then choose clusters via a privacy-aware objective combining empirical loss and MIA risk. Empirical results on MNIST, FEMNIST, and CIFAR-10 show IFCA-MIR achieves substantially reduced MIA vulnerability with comparable accuracy and preserved fairness, though some minority migration to safer clusters can affect minority performance; convergence guarantees similar to IFCA are maintained. This work provides a practical framework for privacy-conscious PFL and opens avenues for incentive mechanisms and differential privacy integration to further strengthen protections in non-IID settings.

Abstract

Federated Learning (FL) has emerged as a promising paradigm for collaborative model training without the need to share clients' personal data, thereby preserving privacy. However, the non-IID nature of the clients' data introduces major challenges for FL, highlighting the importance of personalized federated learning (PFL) methods. In PFL, models are trained to cater to specific feature distributions present in the population data. A notable method for PFL is the Iterative Federated Clustering Algorithm (IFCA), which mitigates the concerns associated with the non-IID-ness by grouping clients with similar data distributions. While it has been shown that IFCA enhances both accuracy and fairness, its strategy of dividing the population into smaller clusters increases vulnerability to Membership Inference Attacks (MIA), particularly among minorities with limited training samples. In this paper, we introduce IFCA-MIR, an improved version of IFCA that integrates MIA risk assessment into the clustering process. Allowing clients to select clusters based on both model performance and MIA vulnerability, IFCA-MIR achieves an improved performance with respect to accuracy, fairness, and privacy. We demonstrate that IFCA-MIR significantly reduces MIA risk while maintaining comparable model accuracy and fairness as the original IFCA.

Mitigating Membership Inference Vulnerability in Personalized Federated Learning

TL;DR

IFCA-MIR addresses the privacy vulnerability of clustering-based Personalized Federated Learning by integrating MIA risk into cluster selection, enabling clients to balance accuracy and privacy. The approach uses a server-side red team to estimate MIA risk for each cluster and delivers privacy scores to clients, who then choose clusters via a privacy-aware objective combining empirical loss and MIA risk. Empirical results on MNIST, FEMNIST, and CIFAR-10 show IFCA-MIR achieves substantially reduced MIA vulnerability with comparable accuracy and preserved fairness, though some minority migration to safer clusters can affect minority performance; convergence guarantees similar to IFCA are maintained. This work provides a practical framework for privacy-conscious PFL and opens avenues for incentive mechanisms and differential privacy integration to further strengthen protections in non-IID settings.

Abstract

Federated Learning (FL) has emerged as a promising paradigm for collaborative model training without the need to share clients' personal data, thereby preserving privacy. However, the non-IID nature of the clients' data introduces major challenges for FL, highlighting the importance of personalized federated learning (PFL) methods. In PFL, models are trained to cater to specific feature distributions present in the population data. A notable method for PFL is the Iterative Federated Clustering Algorithm (IFCA), which mitigates the concerns associated with the non-IID-ness by grouping clients with similar data distributions. While it has been shown that IFCA enhances both accuracy and fairness, its strategy of dividing the population into smaller clusters increases vulnerability to Membership Inference Attacks (MIA), particularly among minorities with limited training samples. In this paper, we introduce IFCA-MIR, an improved version of IFCA that integrates MIA risk assessment into the clustering process. Allowing clients to select clusters based on both model performance and MIA vulnerability, IFCA-MIR achieves an improved performance with respect to accuracy, fairness, and privacy. We demonstrate that IFCA-MIR significantly reduces MIA risk while maintaining comparable model accuracy and fairness as the original IFCA.

Paper Structure

This paper contains 31 sections, 2 theorems, 15 equations, 10 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Federated learning
  4. Membership inference attack
  5. Preliminaries
  6. Personalized FL with IFCA
  7. Fairness
  8. Merbership inference attack accuracy
  9. Iterative Federated Clustering Algorithm with Membership Inference Robustness
  10. Motivation
  11. Notations and problem formulation
  12. Proposed Method: IFCA-MIR
  13. Convergence analysis
  14. Experiments
  15. Datasets
  16. ...and 16 more sections

Key Result

Theorem 1

If Assumptions assump:strong_convex_L_smooth,assump:bounded_var_loss,assump:bounded_var_grad, and assump:init hold, choosing learning rate $\eta=1/L$, each cluster $j\in [s]$, and any $\delta\in(0,1)$, in every round $t>0$, we have with probability at least $(1-\delta)$: where $\epsilon_0 \leq \frac{\sigma}{\delta L \sqrt{pnB}}+\frac{\eta^2}{\delta \alpha^2 \lambda^2 \Delta^4 B}+\frac{\eta\sigma

Figures (10)

  • Figure 1: MIA accuracy according to training dataset size
  • Figure 2: Model accuracy results for each dataset with varying minority dataset sizes. The white diamond represents the mean, while the red line within the box plot indicates the median
  • Figure 3: Model accuracy results for each dataset with varying image deformation ranges
  • Figure 4: Number of MIA violation for each dataset with varying minority dataset sizes
  • Figure 5: Number of MIA violation for each dataset with varying image deformation ranges
  • ...and 5 more figures

Theorems & Definitions (6)

  • definition 1
  • definition 2
  • definition 3
  • Remark 1
  • Theorem 1
  • Corollary 2