Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Energy-Efficient Federated Edge Learning with Streaming Data: A Lyapunov Optimization Approach

Chung-Hsuan Hu, Zheng Chen, Erik G. Larsson

TL;DR

This work forms a stochastic network optimization problem and uses the Lyapunov drift-plus-penalty framework to obtain a dynamic resource management design, which makes adaptive decisions on device scheduling, computational capacity adjustment, and allocation of bandwidth and transmit power in every round.

Abstract

Federated learning (FL) has received significant attention in recent years for its advantages in efficient training of machine learning models across distributed clients without disclosing user-sensitive data. Specifically, in federated edge learning (FEEL) systems, the time-varying nature of wireless channels introduces inevitable system dynamics in the communication process, thereby affecting training latency and energy consumption. In this work, we further consider a streaming data scenario where new training data samples are randomly generated over time at edge devices. Our goal is to develop a dynamic scheduling and resource allocation algorithm to address the inherent randomness in data arrivals and resource availability under long-term energy constraints. To achieve this, we formulate a stochastic network optimization problem and use the Lyapunov drift-plus-penalty framework to obtain a dynamic resource management design. Our proposed algorithm makes adaptive decisions on device scheduling, computational capacity adjustment, and allocation of bandwidth and transmit power in every round. We provide convergence analysis for the considered setting with heterogeneous data and time-varying objective functions, which supports the rationale behind our proposed scheduling design. The effectiveness of our scheme is verified through simulation results, demonstrating improved learning performance and energy efficiency as compared to baseline schemes.

Energy-Efficient Federated Edge Learning with Streaming Data: A Lyapunov Optimization Approach

TL;DR

This work forms a stochastic network optimization problem and uses the Lyapunov drift-plus-penalty framework to obtain a dynamic resource management design, which makes adaptive decisions on device scheduling, computational capacity adjustment, and allocation of bandwidth and transmit power in every round.

Abstract

Federated learning (FL) has received significant attention in recent years for its advantages in efficient training of machine learning models across distributed clients without disclosing user-sensitive data. Specifically, in federated edge learning (FEEL) systems, the time-varying nature of wireless channels introduces inevitable system dynamics in the communication process, thereby affecting training latency and energy consumption. In this work, we further consider a streaming data scenario where new training data samples are randomly generated over time at edge devices. Our goal is to develop a dynamic scheduling and resource allocation algorithm to address the inherent randomness in data arrivals and resource availability under long-term energy constraints. To achieve this, we formulate a stochastic network optimization problem and use the Lyapunov drift-plus-penalty framework to obtain a dynamic resource management design. Our proposed algorithm makes adaptive decisions on device scheduling, computational capacity adjustment, and allocation of bandwidth and transmit power in every round. We provide convergence analysis for the considered setting with heterogeneous data and time-varying objective functions, which supports the rationale behind our proposed scheduling design. The effectiveness of our scheme is verified through simulation results, demonstrating improved learning performance and energy efficiency as compared to baseline schemes.
Paper Structure (30 sections, 4 theorems, 79 equations, 8 figures, 3 tables, 1 algorithm)

This paper contains 30 sections, 4 theorems, 79 equations, 8 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Motivation and Related Work
  3. Novelty and Contributions
  4. Paper Organization
  5. System Model
  6. Energy Consumption Model
  7. Energy Consumption for Computation
  8. Energy Consumption for Transmission
  9. Latency Model
  10. Joint Computation and Communication Resource Management
  11. Dynamic Scheduling and Resource Allocation Algorithm
  12. Scheduling Phase: Determine $\Pi^*(t)$ and $f_k^*(t)$
  13. Aggregation Phase: Determine $\rho_k^*(t)$ and $P_k^*(t)$
  14. Convergence Analysis
  15. Convergence Results
  16. ...and 15 more sections

Key Result

Theorem 1

Under Assumptions asp:smth-asp:bndLocal, with $M(t)=\frac{m}{t+1}$ for some $m>0$, if the learning rate satisfies the following result holds for $t_0>\lfloor m\rfloor$, where for $t_1,t_2>0,\bar{\kappa}_{t_1,t_2}=\frac{1}{|t_2-t_1|+1}\sum_{t=\min(t_1,t_2)}^{\max(t_1,t_2)}\kappa(t),$ $\Omega=2(\varphi_1+\varphi_2)$ and $\mathbb{E}[\cdot]$ is total expectation over $\Pi(j)$, $\{\xi_k(j)|k\in\Pi(j)

Figures (8)

  • Figure 1: The procedure of solving (P0) and the execution timeline in the $t$-th iteration. The server first computes $\Pi^*(t)$ and $f^*_k(t)$ to initiate the local training process. Before the server conducts model aggregation, a feasible subset $\bar{\Pi}(t)\subseteq\Pi^*(t)$ is determined by checking the transmission time budget $T_{\text{rd}}-T_{k}^{\text{cmp}}(t),\forall k$. Then, $\rho_k^*(t)$ and $P_k^*(t)$, $\forall k\in\bar{\Pi}(t)$, are obtained for the update transmission.
  • Figure 2: Test accuracy of the proposed ('prop') and the baseline ('rdm') methods with truncated Gaussian data arrival pattern.
  • Figure 3: Average energy consumption of the proposed ('prop') and the baseline ('rdm') methods with truncated Gaussian data arrival pattern.
  • Figure 4: Test accuracy and average energy consumption of the proposed ('prop') and the baseline ('rdm') methods for CIFAR-10 (non-i.i.d.) with truncated Gaussian data arrival pattern.
  • Figure 5: Testing loss comparison between the proposed ('prop') and the baseline ('rdm') methods with uniformly random ('uni') and truncated Poisson ('ps') data arrival patterns.
  • ...and 3 more figures

Theorems & Definitions (10)

  • Remark 1
  • Remark 2
  • Definition 1
  • Remark 3
  • Theorem 1
  • Lemma 1
  • Remark 4
  • Remark 5
  • Lemma 2
  • Lemma 3