Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Batch Allocation for Wireless Federated Learning

Jaeyoung Song, Sang-Woon Jeon

TL;DR

This work analyzes the completion time of wireless federated learning under two access schemes, TDMA and random access (RA). It derives a convergence-based bound that yields the iteration count K(ε) needed to reach a target optimality gap ε and then decomposes per-iteration time into computation and communication components. A novel step-wise batch allocation is proposed, proven optimal for TDMA and optimized for RA in small-device regimes, with numerical optimization for larger N. Experiments on MNIST and CIFAR-10 demonstrate substantial reductions in completion time compared to uniform batching, highlighting the practical benefits of heterogenous batch sizing in wireless FL.

Abstract

Federated learning aims to construct a global model that fits the dataset distributed across local devices without direct access to private data, leveraging communication between a server and the local devices. In the context of a practical communication scheme, we study the completion time required to achieve a target performance. Specifically, we analyze the number of iterations required for federated learning to reach a specific optimality gap from a minimum global loss. Subsequently, we characterize the time required for each iteration under two fundamental multiple access schemes: time-division multiple access (TDMA) and random access (RA). We propose a step-wise batch allocation, demonstrated to be optimal for TDMA-based federated learning systems. Additionally, we show that the non-zero batch gap between devices provided by the proposed step-wise batch allocation significantly reduces the completion time for RA-based learning systems. Numerical evaluations validate these analytical results through real-data experiments, highlighting the remarkable potential for substantial completion time reduction.

Optimal Batch Allocation for Wireless Federated Learning

TL;DR

This work analyzes the completion time of wireless federated learning under two access schemes, TDMA and random access (RA). It derives a convergence-based bound that yields the iteration count K(ε) needed to reach a target optimality gap ε and then decomposes per-iteration time into computation and communication components. A novel step-wise batch allocation is proposed, proven optimal for TDMA and optimized for RA in small-device regimes, with numerical optimization for larger N. Experiments on MNIST and CIFAR-10 demonstrate substantial reductions in completion time compared to uniform batching, highlighting the practical benefits of heterogenous batch sizing in wireless FL.

Abstract

Federated learning aims to construct a global model that fits the dataset distributed across local devices without direct access to private data, leveraging communication between a server and the local devices. In the context of a practical communication scheme, we study the completion time required to achieve a target performance. Specifically, we analyze the number of iterations required for federated learning to reach a specific optimality gap from a minimum global loss. Subsequently, we characterize the time required for each iteration under two fundamental multiple access schemes: time-division multiple access (TDMA) and random access (RA). We propose a step-wise batch allocation, demonstrated to be optimal for TDMA-based federated learning systems. Additionally, we show that the non-zero batch gap between devices provided by the proposed step-wise batch allocation significantly reduces the completion time for RA-based learning systems. Numerical evaluations validate these analytical results through real-data experiments, highlighting the remarkable potential for substantial completion time reduction.
Paper Structure (19 sections, 7 theorems, 76 equations, 10 figures, 1 algorithm)

This paper contains 19 sections, 7 theorems, 76 equations, 10 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Federated Learning System
  4. Minimization of Completion Time
  5. Computing time for local updates
  6. Communication time for model transmission
  7. Characterization of the Number of Required Iterations
  8. Analysis on Iteration Time
  9. Iteration Time for TDMA
  10. Iteration Time for RA
  11. Two-devices case
  12. Three-devices case
  13. N-devices case
  14. Optimal Batch Allocation
  15. Completion Time
  16. ...and 4 more sections

Key Result

Lemma 1

Suppose that $f( \mathbf{w})$ is $M$-strongly convex. Then, for any $\mathbf{w}$,

Figures (10)

  • Figure 1: Federated learning system with a server and $N$ IoT devices.
  • Figure 2: Training loss as a function of the number of iterations.
  • Figure 3: Comparison of global loss for different batch gap using TDMA for $N=20$, $\rho = 10$.
  • Figure 4: Comparison of accuracy for different batch gap using TDMA for $N=20$, $\rho = 10$.
  • Figure 5: Comparison of global loss for different batch gap using RA for $N=20$, $\rho = 10$, $p^{\text{tr}} = 0.2$.
  • ...and 5 more figures

Theorems & Definitions (8)

  • Remark 1
  • Lemma 1
  • Lemma 2
  • Theorem 1
  • Corollary 1
  • Proposition 1
  • Theorem 2
  • Theorem 3