Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedCAPrivacy: Privacy-Preserving Heterogeneous Federated Learning with Anonymous Adaptive Clustering

Yunan Wei, Shengnan Zhao, Chuan Zhao, Zhe Liu, Zhenxiang Chen, Minghao Zhao

TL;DR

FedCAPrivacy tackles privacy-preserving clustering in heterogeneous FL by introducing anonymous adaptive clustering with an oblivious shuffle-based anonymization and an iteration-based decay of clustering frequency. The framework uses a three-party architecture (BS, CSP, devices) and Paillier HE to perform secure cluster-wise aggregation, preventing server inference of participant identities and similarities. It includes a setup phase for key generation, local training with encrypted updates, and secure cluster aggregation with Gaussian similarity and spectral embedding. Experiments on MNIST, Fashion-MNIST, CIFAR-10/100 show the approach yields substantial training efficiency gains (≈$7\times$) while maintaining high privacy across non-IID and heterogeneous resource settings.

Abstract

Federated learning (FL) is a distributed machine learning paradigm enabling multiple clients to train a model collaboratively without exposing their local data. Among FL schemes, clustering is an effective technique addressing the heterogeneity issue (i.e., differences in data distribution and computational ability affect training performance and effectiveness) via grouping participants with similar computational resources or data distribution into clusters. However, intra-cluster data exchange poses privacy risks, while cluster selection and adaptation introduce challenges that may affect overall performance. To address these challenges, this paper introduces anonymous adaptive clustering, a novel approach that simultaneously enhances privacy protection and boosts training efficiency. Specifically, an oblivious shuffle-based anonymization method is designed to safeguard user identities and prevent the aggregation server from inferring similarities through clustering. Additionally, to improve performance, we introduce an iteration-based adaptive frequency decay strategy, which leverages variability in clustering probabilities to optimize training dynamics. With these techniques, we build the FedCAPrivacy; experiments show that FedCAPrivacy achieves ~7X improvement in terms of performance while maintaining high privacy.

FedCAPrivacy: Privacy-Preserving Heterogeneous Federated Learning with Anonymous Adaptive Clustering

TL;DR

FedCAPrivacy tackles privacy-preserving clustering in heterogeneous FL by introducing anonymous adaptive clustering with an oblivious shuffle-based anonymization and an iteration-based decay of clustering frequency. The framework uses a three-party architecture (BS, CSP, devices) and Paillier HE to perform secure cluster-wise aggregation, preventing server inference of participant identities and similarities. It includes a setup phase for key generation, local training with encrypted updates, and secure cluster aggregation with Gaussian similarity and spectral embedding. Experiments on MNIST, Fashion-MNIST, CIFAR-10/100 show the approach yields substantial training efficiency gains (≈) while maintaining high privacy across non-IID and heterogeneous resource settings.

Abstract

Federated learning (FL) is a distributed machine learning paradigm enabling multiple clients to train a model collaboratively without exposing their local data. Among FL schemes, clustering is an effective technique addressing the heterogeneity issue (i.e., differences in data distribution and computational ability affect training performance and effectiveness) via grouping participants with similar computational resources or data distribution into clusters. However, intra-cluster data exchange poses privacy risks, while cluster selection and adaptation introduce challenges that may affect overall performance. To address these challenges, this paper introduces anonymous adaptive clustering, a novel approach that simultaneously enhances privacy protection and boosts training efficiency. Specifically, an oblivious shuffle-based anonymization method is designed to safeguard user identities and prevent the aggregation server from inferring similarities through clustering. Additionally, to improve performance, we introduce an iteration-based adaptive frequency decay strategy, which leverages variability in clustering probabilities to optimize training dynamics. With these techniques, we build the FedCAPrivacy; experiments show that FedCAPrivacy achieves ~7X improvement in terms of performance while maintaining high privacy.

Paper Structure

This paper contains 19 sections, 2 theorems, 16 equations, 5 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Federated Learning
  5. Homomorphic Encryption
  6. FedCAPrivacy Framework
  7. Technical Overview
  8. Construction of FedCAPrivacy
  9. Setup Phase
  10. Training at Local devices
  11. Secure Cluster-wise Aggregation
  12. Security Analysis
  13. Evaluation
  14. Experiment Configuration Details
  15. Ablation Studies
  16. ...and 4 more sections

Key Result

Proposition 5.1

(Honest but curious security, with curious servers): Given a security parameter $\kappa$, any subset of users $U$ and $S=\{BS, CSP\}$, there is a probabilistic polynomial time (PPT) simulator $SIM$ whose output is computationally indistinguishable from $REAL_S^{U,\kappa}$: where "$\approx_c$" represents computationally indistinguishable.

Figures (5)

  • Figure 1: Illustration of FedCAPrivacy workflow.
  • Figure 2: Influence of the frequency of clusters for heterogeneous resources.
  • Figure 3: Influence of the dynamic adjustment of cluster frequency. The results are averaged from three independent experiments.
  • Figure 4: Influence of the heterogeneity on the model optimization in the case of heterogeneous resources.
  • Figure 5: Influence of the number of clustered classes.

Theorems & Definitions (4)

  • Proposition 5.1
  • proof
  • Proposition 5.2
  • proof