Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Asynchronous Federated Learning Using Outdated Local Updates Over TDMA Channel

Jaeyoung Song, Jun-Pyo Hong

TL;DR

The extensive simulation results demonstrate that asynchronous FL with the intentional delay can rapidly reduce global loss by lowering the staleness of local updates in resource-limited wireless communication networks.

Abstract

In this paper, we consider asynchronous federated learning (FL) over time-division multiple access (TDMA)-based communication networks. Considering TDMA for transmitting local updates can introduce significant delays to conventional synchronous FL, where all devices start local training from a common global model. In the proposed asynchronous FL approach, we partition devices into multiple TDMA groups, enabling simultaneous local computation and communication across different groups. This enhances time efficiency at the expense of staleness of local updates. We derive the relationship between the staleness of local updates and the size of the TDMA group in a training round. Moreover, our convergence analysis shows that although outdated local updates hinder appropriate global model updates, asynchronous FL over the TDMA channel converges even in the presence of data heterogeneity. Notably, the analysis identifies the impact of outdated local updates on convergence rate. Based on observations from our convergence rate, we refine asynchronous FL strategy by introducing an intentional delay in local training. This refinement accelerates the convergence by reducing the staleness of local updates. Our extensive simulation results demonstrate that asynchronous FL with the intentional delay can rapidly reduce global loss by lowering the staleness of local updates in resource-limited wireless communication networks.

Asynchronous Federated Learning Using Outdated Local Updates Over TDMA Channel

TL;DR

The extensive simulation results demonstrate that asynchronous FL with the intentional delay can rapidly reduce global loss by lowering the staleness of local updates in resource-limited wireless communication networks.

Abstract

In this paper, we consider asynchronous federated learning (FL) over time-division multiple access (TDMA)-based communication networks. Considering TDMA for transmitting local updates can introduce significant delays to conventional synchronous FL, where all devices start local training from a common global model. In the proposed asynchronous FL approach, we partition devices into multiple TDMA groups, enabling simultaneous local computation and communication across different groups. This enhances time efficiency at the expense of staleness of local updates. We derive the relationship between the staleness of local updates and the size of the TDMA group in a training round. Moreover, our convergence analysis shows that although outdated local updates hinder appropriate global model updates, asynchronous FL over the TDMA channel converges even in the presence of data heterogeneity. Notably, the analysis identifies the impact of outdated local updates on convergence rate. Based on observations from our convergence rate, we refine asynchronous FL strategy by introducing an intentional delay in local training. This refinement accelerates the convergence by reducing the staleness of local updates. Our extensive simulation results demonstrate that asynchronous FL with the intentional delay can rapidly reduce global loss by lowering the staleness of local updates in resource-limited wireless communication networks.

Paper Structure

This paper contains 14 sections, 6 theorems, 65 equations, 7 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. System Model
  3. Asynchronous Federated Learning Over TDMA Channel
  4. Convergence Analysis
  5. Intentional Delay Federated Learning
  6. Experiments
  7. Impact of Delay
  8. Performance of Intentional Delay Federated Learning
  9. Tradeoff between Computation and Communication Factors
  10. Conclusion
  11. Proof of Lemma \ref{['lem:descent_1']}
  12. Proof of Lemma \ref{['lem:lemma3']}
  13. Proof of Lemma \ref{['lem:sq_norm_dgrad']}
  14. Proof of Theorem \ref{['thm:convergence']}

Key Result

Lemma 1

Under Assumptions assump_smooth, assump_bound, and assump_hetero, we can bound $\mathbb{E} \left[ f(\mathbf{w}_{k+1} ) \right]$ as follows

Figures (7)

  • Figure 1: Federated learning system with $N$ IoT devices and a server
  • Figure 2: Example timeline of asynchronous FL over TDMA channel for $N=6, S = 2, \tau^{\text{comp}} = 1$, and $r=1$
  • Figure 3: Performance of asynchronous FL over TDMA channel
  • Figure 4: Performance of IDFL
  • Figure 5: Comparison of global loss between Asynchronous FL with delayed gradient and IDFL
  • ...and 2 more figures

Theorems & Definitions (6)

  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Theorem 1
  • Proposition 1