Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Heterogeneity-Aware Cooperative Federated Edge Learning with Adaptive Computation and Communication Compression

Zhenxiao Zhang, Zhidong Gao, Yuanxiong Guo, Yanmin Gong

TL;DR

A heterogeneity-aware CFEL scheme called Heterogeneity-Aware Cooperative Edge-based Federated Averaging (HCEF) that aims to maximize the model accuracy while minimizing the training time and energy consumption via adaptive computation and communication compression in CFEL is proposed.

Abstract

Motivated by the drawbacks of cloud-based federated learning (FL), cooperative federated edge learning (CFEL) has been proposed to improve efficiency for FL over mobile edge networks, where multiple edge servers collaboratively coordinate the distributed model training across a large number of edge devices. However, CFEL faces critical challenges arising from dynamic and heterogeneous device properties, which slow down the convergence and increase resource consumption. This paper proposes a heterogeneity-aware CFEL scheme called \textit{Heterogeneity-Aware Cooperative Edge-based Federated Averaging} (HCEF) that aims to maximize the model accuracy while minimizing the training time and energy consumption via adaptive computation and communication compression in CFEL. By theoretically analyzing how local update frequency and gradient compression affect the convergence error bound in CFEL, we develop an efficient online control algorithm for HCEF to dynamically determine local update frequencies and compression ratios for heterogeneous devices. Experimental results show that compared with prior schemes, the proposed HCEF scheme can maintain higher model accuracy while reducing training latency and improving energy efficiency simultaneously.

Heterogeneity-Aware Cooperative Federated Edge Learning with Adaptive Computation and Communication Compression

TL;DR

A heterogeneity-aware CFEL scheme called Heterogeneity-Aware Cooperative Edge-based Federated Averaging (HCEF) that aims to maximize the model accuracy while minimizing the training time and energy consumption via adaptive computation and communication compression in CFEL is proposed.

Abstract

Motivated by the drawbacks of cloud-based federated learning (FL), cooperative federated edge learning (CFEL) has been proposed to improve efficiency for FL over mobile edge networks, where multiple edge servers collaboratively coordinate the distributed model training across a large number of edge devices. However, CFEL faces critical challenges arising from dynamic and heterogeneous device properties, which slow down the convergence and increase resource consumption. This paper proposes a heterogeneity-aware CFEL scheme called \textit{Heterogeneity-Aware Cooperative Edge-based Federated Averaging} (HCEF) that aims to maximize the model accuracy while minimizing the training time and energy consumption via adaptive computation and communication compression in CFEL. By theoretically analyzing how local update frequency and gradient compression affect the convergence error bound in CFEL, we develop an efficient online control algorithm for HCEF to dynamically determine local update frequencies and compression ratios for heterogeneous devices. Experimental results show that compared with prior schemes, the proposed HCEF scheme can maintain higher model accuracy while reducing training latency and improving energy efficiency simultaneously.
Paper Structure (22 sections, 10 theorems, 72 equations, 8 figures, 2 tables, 3 algorithms)

This paper contains 22 sections, 10 theorems, 72 equations, 8 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related works
  3. Preliminaries and Problem Formulation
  4. CFEL: Cooperative Federated Edge Learning
  5. Heterogeneity-Aware CFEL
  6. Problem Formulation
  7. Convergence Analysis
  8. Algorithm Design
  9. Problem Reformulation
  10. Algorithm Description
  11. Performance Evaluation
  12. Experimental Setup
  13. Experimental Results
  14. Ablation Studies
  15. Conclusions
  16. ...and 7 more sections

Key Result

Lemma 1

Under Assumptions ass:smoothness, ass:gradient, and ass:lowerbounded, if the learning rate $\eta \leq (\rho_n^{l,r})^2-2(\rho_n^{l,r})^2(1-\theta_n^{l,r})(2L(2-\theta_n^{l,r})\rho_{n}^{l,r}), \forall n, r, l,$ the iterates of Algorithm algorithm-1 satisfy

Figures (8)

  • Figure 1: CFEL: Cooperative Federated Edge Learning.
  • Figure 2: Test accuracy versus runtime and energy consumption for CIFAR-10.
  • Figure 3: Test accuracy versus runtime and energy consumption for FEMNIST.
  • Figure 4: Runtime and energy consumption under different non-IID levels for CIFAR-10 with target accuracy $70\%$.
  • Figure 5: (a), (b) Runtime and energy consumption under different backhaul topologies for CIFAR-10 with target accuracy $70\%$; (c), (d) Runtime and energy consumption under different backhaul topologies for FEMNIST with target accuracy $75\%$.
  • ...and 3 more figures

Theorems & Definitions (13)

  • Lemma 1: Convergence Decomposition
  • Lemma 2: Bounded Discrepancy Error
  • Theorem 1: Convergence of HCEF
  • Remark 1: Effect of $\theta_n^{l,r}$ and $\rho_n^{l,r}$
  • Remark 2: Comparison to FedAvg
  • Remark 3: Computational Complexity Analysis:
  • Lemma 3
  • Lemma 4
  • Lemma 5
  • Lemma 6
  • ...and 3 more