Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Faster Convergence on Heterogeneous Federated Edge Learning: An Adaptive Clustered Data Sharing Approach

Gang Hu, Yinglei Teng, Nan Wang, Zhu Han

TL;DR

This work tackles non-IID data challenges in Federated Edge Learning by introducing a clustered data sharing framework that leverages sidelink multicast to reduce data heterogeneity. It develops a distribution-based adaptive clustering algorithm (DACA) to form privacy-aware clusters and a stochastic successive convex approximation-based joint optimization (JFVO) to allocate compute frequency and data sharing volume. The approach yields faster convergence and higher accuracy under limited communication, validated on MNIST, CIFAR-10, and Shakespeare datasets, with theoretical convergence insights linking heterogeneity to training rounds. The findings offer a practical, scalable mechanism to improve FEL performance in heterogeneous, edge-centric networks while addressing privacy and latency constraints.

Abstract

Federated Edge Learning (FEEL) emerges as a pioneering distributed machine learning paradigm for the 6G Hyper-Connectivity, harnessing data from the Internet of Things (IoT) devices while upholding data privacy. However, current FEEL algorithms struggle with non-independent and non-identically distributed (non-IID) data, leading to elevated communication costs and compromised model accuracy. To address these statistical imbalances within FEEL, we introduce a clustered data sharing framework, mitigating data heterogeneity by selectively sharing partial data from cluster heads to trusted associates through sidelink-aided multicasting. The collective communication pattern is integral to FEEL training, where both cluster formation and the efficiency of communication and computation impact training latency and accuracy simultaneously. To tackle the strictly coupled data sharing and resource optimization, we decompose the overall optimization problem into the clients clustering and effective data sharing subproblems. Specifically, a distribution-based adaptive clustering algorithm (DACA) is devised basing on three deductive cluster forming conditions, which ensures the maximum sharing yield. Meanwhile, we design a stochastic optimization based joint computed frequency and shared data volume optimization (JFVO) algorithm, determining the optimal resource allocation with an uncertain objective function. The experiments show that the proposed framework facilitates FEEL on non-IID datasets with faster convergence rate and higher model accuracy in a limited communication environment.

Faster Convergence on Heterogeneous Federated Edge Learning: An Adaptive Clustered Data Sharing Approach

TL;DR

This work tackles non-IID data challenges in Federated Edge Learning by introducing a clustered data sharing framework that leverages sidelink multicast to reduce data heterogeneity. It develops a distribution-based adaptive clustering algorithm (DACA) to form privacy-aware clusters and a stochastic successive convex approximation-based joint optimization (JFVO) to allocate compute frequency and data sharing volume. The approach yields faster convergence and higher accuracy under limited communication, validated on MNIST, CIFAR-10, and Shakespeare datasets, with theoretical convergence insights linking heterogeneity to training rounds. The findings offer a practical, scalable mechanism to improve FEL performance in heterogeneous, edge-centric networks while addressing privacy and latency constraints.

Abstract

Federated Edge Learning (FEEL) emerges as a pioneering distributed machine learning paradigm for the 6G Hyper-Connectivity, harnessing data from the Internet of Things (IoT) devices while upholding data privacy. However, current FEEL algorithms struggle with non-independent and non-identically distributed (non-IID) data, leading to elevated communication costs and compromised model accuracy. To address these statistical imbalances within FEEL, we introduce a clustered data sharing framework, mitigating data heterogeneity by selectively sharing partial data from cluster heads to trusted associates through sidelink-aided multicasting. The collective communication pattern is integral to FEEL training, where both cluster formation and the efficiency of communication and computation impact training latency and accuracy simultaneously. To tackle the strictly coupled data sharing and resource optimization, we decompose the overall optimization problem into the clients clustering and effective data sharing subproblems. Specifically, a distribution-based adaptive clustering algorithm (DACA) is devised basing on three deductive cluster forming conditions, which ensures the maximum sharing yield. Meanwhile, we design a stochastic optimization based joint computed frequency and shared data volume optimization (JFVO) algorithm, determining the optimal resource allocation with an uncertain objective function. The experiments show that the proposed framework facilitates FEEL on non-IID datasets with faster convergence rate and higher model accuracy in a limited communication environment.
Paper Structure (21 sections, 4 theorems, 53 equations, 9 figures, 2 tables, 2 algorithms)

This paper contains 21 sections, 4 theorems, 53 equations, 9 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Federated learning with non-IID data
  3. Statistical Heterogeneity
  4. Quantify Heterogeneity
  5. Preliminary Experiments and Analysis
  6. System Model
  7. Clustered Data Sharing FEL Framework
  8. Clustered Data Sharing Model
  9. FEL Transmission and Computation Model
  10. Problem Formulation and Solution
  11. Problem Formulation and Decomposition
  12. Distribution-based Adaptive Clustering Algorithm
  13. Joint Computed Frequency and Shared Data Volume Optimization Algorithm
  14. Complexity Analysis
  15. Simulation and Experimental Results
  16. ...and 6 more sections

Key Result

Theorem 1

Putting the constraints (C_2_c) and (C_2_d) aside, if ${{\cal M}^*}$ and ${{\cal C}^*}$ are the optimal solutions to the problem $\mathcal{P}1'$, then condition 2 and condition 3 both hold with ${{\cal M}^*}$ and ${{\cal C}^*}$. The proof is omitted in this paper due to the limited space. The full p

Figures (9)

  • Figure 1: Train loss on clustered data sharing for non-IID FL. (a) For IID setting, each user is randomly assigned a uniform distribution over all classes. For non-IID setting, the proportion of labels of among users are different, leading to varying ${\bar{D}_{{\text{EMD}}}}$. (b) The shared data volume is quantified as $\alpha$, representing the proportion of sharing data volume in its local data volume. (c) The users are randomly divided into clusters, where each cluster consists of cluster heads to transmit data and cluster members to receive data.
  • Figure 2: Clustered data sharing framework for FEL.
  • Figure 3: Flow chard of the proposed algorithms.
  • Figure 4: The convergence and accuracy performance with different data sharing strategies, ${v_{th}} = 3.5 \times 10^5$ bit/s and ${e_{th}} = 0.5$.
  • Figure 5: The illustration of the sample number per class allocated to each user (indicated by dot sizes).
  • ...and 4 more figures

Theorems & Definitions (7)

  • Theorem 1
  • Lemma 1: Results of one round
  • Remark 1
  • Lemma 2: Bounded model drift
  • Remark 2
  • Theorem 2
  • Remark 3