Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uplink Over-the-Air Aggregation for Multi-Model Wireless Federated Learning

Chong Zhang, Min Dong, Ben Liang, Ali Afana, Yahia Ahmed

TL;DR

This work addresses the challenge of efficient wireless federated learning when training multiple models in parallel. It introduces uplink over-the-air aggregation and derives a convergence-based upper bound on the global model update, then casts the design as a joint transmit-receive beamforming problem solved by block coordinate descent. The proposed method yields closed-form updates and significantly outperforms sequential single-model training under realistic wireless conditions. The work demonstrates substantial speedups in training convergence and reduces bandwidth and latency for multi-model FL.

Abstract

We propose an uplink over-the-air aggregation (OAA) method for wireless federated learning (FL) that simultaneously trains multiple models. To maximize the multi-model training convergence rate, we derive an upper bound on the optimality gap of the global model update, and then, formulate an uplink joint transmit-receive beamforming optimization problem to minimize this upper bound. We solve this problem using the block coordinate descent approach, which admits low-complexity closed-form updates. Simulation results show that our proposed multi-model FL with fast OAA substantially outperforms sequentially training multiple models under the conventional single-model approach.

Uplink Over-the-Air Aggregation for Multi-Model Wireless Federated Learning

TL;DR

This work addresses the challenge of efficient wireless federated learning when training multiple models in parallel. It introduces uplink over-the-air aggregation and derives a convergence-based upper bound on the global model update, then casts the design as a joint transmit-receive beamforming problem solved by block coordinate descent. The proposed method yields closed-form updates and significantly outperforms sequential single-model training under realistic wireless conditions. The work demonstrates substantial speedups in training convergence and reduces bandwidth and latency for multi-model FL.

Abstract

We propose an uplink over-the-air aggregation (OAA) method for wireless federated learning (FL) that simultaneously trains multiple models. To maximize the multi-model training convergence rate, we derive an upper bound on the optimality gap of the global model update, and then, formulate an uplink joint transmit-receive beamforming optimization problem to minimize this upper bound. We solve this problem using the block coordinate descent approach, which admits low-complexity closed-form updates. Simulation results show that our proposed multi-model FL with fast OAA substantially outperforms sequentially training multiple models under the conventional single-model approach.
Paper Structure (10 sections, 1 theorem, 19 equations, 2 figures)

This paper contains 10 sections, 1 theorem, 19 equations, 2 figures.

Table of Contents

  1. Introduction
  2. System Model and Problem Statement
  3. Multi-Model FL System
  4. Wireless Communication Model
  5. Uplink OAA for Multi-Model FL
  6. Uplink Aggregation Framework
  7. Multi-Model FL Convergence Analysis under Uplink OAA
  8. Uplink Joint Transmit-Receive Beamforming Design
  9. Simulation Results
  10. Conclusion

Key Result

Proposition 1

Under Assumptions assump_smooth--assump_bound_diff and for $\eta_n<\frac{1}{\lambda}$, $\forall n$, the expected optimality gap after $S$ frames is bounded by where $\Gamma_m \triangleq {\mathbb{E}}[\| \boldsymbol \theta_{m,0} - \boldsymbol \theta_m^\star\|^2]$, $G_n \triangleq 4(1-\eta_n\lambda)^{2JM}$, $\Lambda \triangleq \sum_{n=0}^{S-2}C_{n} (\!\prod_{s=n+1}^{S-1}\!G_s)+ C_{S-1}$ with $C_{n}

Figures (2)

  • Figure 1: Device-model round robin scheduling for $M=3$ models.
  • Figure 2: Top Left: Test accuracy vs. $M$ (Model A). Rest of figures: Test accuracy vs. communication round $t$: Top Right -- Model A; Bottom Left -- Model B; Bottom Right -- Model C ($90\%$ confidence intervals are shown).

Theorems & Definitions (1)

  • Proposition 1