Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DFPL: Decentralized Federated Prototype Learning Across Heterogeneous Data Distributions

Hongliang Zhang, Fenghua Xu, Zhongyuan Yu, Shanchen Pang, Chunqiang Hu, Jiguo Yu

TL;DR

This work tackles the challenge of non-IID data in decentralized federated learning by introducing DFPL, a prototype-based framework where each client trains locally and mines blocks while exchanging class prototypes instead of full models. By integrating blockchain-enabled mining on every client, DFPL reduces communication overhead and mitigates heterogeneity through prototype alignment, formalized via an objective combining a classification loss with a prototype-consistency term. The authors provide convergence and time-complexity analyses under resource constraints and demonstrate through experiments on MNIST, FMNIST, CIFAR10, and SVHN that DFPL achieves higher accuracy and improved stability compared with several baselines, particularly in non-IID settings. The results indicate DFPL’s practical potential for privacy-preserving, resource-aware, decentralized FL, with future work proposed on handling model heterogeneity and enhancing privacy guarantees.

Abstract

Federated learning is a distributed machine learning paradigm through centralized model aggregation. However, standard federated learning relies on a centralized server, making it vulnerable to server failures. While existing solutions utilize blockchain technology to implement Decentralized Federated Learning (DFL), the statistical heterogeneity of data distributions among clients severely degrades the performance of DFL. Driven by this issue, this paper proposes a decentralized federated prototype learning framework, named DFPL, which significantly improves the performance of DFL under heterogeneous data distributions. Specifically, DFPL introduces prototype learning into DFL to mitigate the impact of statistical heterogeneity and reduces the amount of parameters exchanged between clients. Additionally, blockchain is embedded into our framework, enabling the training and mining processes to be executed locally on each client. From a theoretical perspective, we analyze the convergence of DFPL by modeling the required computational resources during both training and mining. The experiment results highlight the superiority of DFPL in both model performance and communication efficiency across four benchmark datasets with heterogeneous data distributions.

DFPL: Decentralized Federated Prototype Learning Across Heterogeneous Data Distributions

TL;DR

This work tackles the challenge of non-IID data in decentralized federated learning by introducing DFPL, a prototype-based framework where each client trains locally and mines blocks while exchanging class prototypes instead of full models. By integrating blockchain-enabled mining on every client, DFPL reduces communication overhead and mitigates heterogeneity through prototype alignment, formalized via an objective combining a classification loss with a prototype-consistency term. The authors provide convergence and time-complexity analyses under resource constraints and demonstrate through experiments on MNIST, FMNIST, CIFAR10, and SVHN that DFPL achieves higher accuracy and improved stability compared with several baselines, particularly in non-IID settings. The results indicate DFPL’s practical potential for privacy-preserving, resource-aware, decentralized FL, with future work proposed on handling model heterogeneity and enhancing privacy guarantees.

Abstract

Federated learning is a distributed machine learning paradigm through centralized model aggregation. However, standard federated learning relies on a centralized server, making it vulnerable to server failures. While existing solutions utilize blockchain technology to implement Decentralized Federated Learning (DFL), the statistical heterogeneity of data distributions among clients severely degrades the performance of DFL. Driven by this issue, this paper proposes a decentralized federated prototype learning framework, named DFPL, which significantly improves the performance of DFL under heterogeneous data distributions. Specifically, DFPL introduces prototype learning into DFL to mitigate the impact of statistical heterogeneity and reduces the amount of parameters exchanged between clients. Additionally, blockchain is embedded into our framework, enabling the training and mining processes to be executed locally on each client. From a theoretical perspective, we analyze the convergence of DFPL by modeling the required computational resources during both training and mining. The experiment results highlight the superiority of DFPL in both model performance and communication efficiency across four benchmark datasets with heterogeneous data distributions.
Paper Structure (37 sections, 2 theorems, 45 equations, 11 figures, 5 tables, 1 algorithm)

This paper contains 37 sections, 2 theorems, 45 equations, 11 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. FL with Blockchain
  4. FL with Non-IID Data
  5. Prototype Learning
  6. Design Details of DFPL
  7. Design Goals
  8. Framework of DFPL
  9. Prototype Calculation and Objective of DFPL
  10. Prototype Calculation
  11. Optimization Function
  12. Detailed Implementation in Local Model Training
  13. Iterative Model Training
  14. Prototype Generation
  15. Computation Time Modeling
  16. ...and 22 more sections

Key Result

Corollary 1

(DFPL Convergence). Given fixed$\alpha, \beta, R,$ and $t_{sum}$, for an arbitrary client $k$ in DFPL, the optimization function monotonically decreases in communication round when and where $\eta^{(e^{\prime})}_{k,r}$ denotes the local learning rate of client $k$ at $e$-th local iteration ($e^{\prime} \in \{\frac{1}{2},1, \cdots, E-1 \}$), $\lambda_{k,r}$ denotes the importance weight for the

Figures (11)

  • Figure 1: The performance of DFL using the FedAvg mcmahan2017communication aggregation strategy in IID and Non-IID settings, where the Non-IID data is simulated using a Dirichlet distribution lin2016dirichlet with a concentration parameter of 0.1.
  • Figure 2: The four steps of the DFPL framework during the $r$-th communication round.
  • Figure 3: The heat map about heterogeneous data distributions of CIFAR10 for each client.
  • Figure 4: Convergence of the proposed DFPL on the FMNIST, and CIFAR10.
  • Figure 5: TAL versus $R$ for different $\alpha$ values for MNIST, FMNIST, and CIFAR10 with different data distributions.
  • ...and 6 more figures

Theorems & Definitions (5)

  • proof
  • Corollary 1
  • proof
  • Corollary 2
  • proof