Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beamforming and Device Selection Design in Federated Learning with Over-the-air Aggregation

Faeze Moradi Kalarde, Min Dong, Ben Liang, Yahia A. Eldemerdash Ahmed, Ho Ting Cheng

TL;DR

This work jointly design uplink receiver beamforming and device selection for over-the-air FL over time-varying wireless channels to maximize the training convergence rate and proposes a Greedy Spatial Device Selection approach, which uses a sequential procedure to select devices based on a measure capturing both the channel strength and the channel correlation to the selected devices.

Abstract

Federated learning (FL) with over-the-air computation can efficiently utilize the communication bandwidth but is susceptible to analog aggregation error. Excluding those devices with weak channel conditions can reduce the aggregation error, but it also limits the amount of local training data for FL, which can reduce the training convergence rate. In this work, we jointly design uplink receiver beamforming and device selection for over-the-air FL over time-varying wireless channels to maximize the training convergence rate. We reformulate this stochastic optimization problem into a mixed-integer program using an upper bound on the global training loss over communication rounds. We then propose a Greedy Spatial Device Selection (GSDS) approach, which uses a sequential procedure to select devices based on a measure capturing both the channel strength and the channel correlation to the selected devices. We show that given the selected devices, the receiver beamforming optimization problem is equivalent to downlink single-group multicast beamforming. To reduce the computational complexity, we also propose an Alternating-optimization-based Device Selection and Beamforming (ADSBF) approach, which solves the receiver beamforming and device selection subproblems alternatingly. In particular, despite the device selection being an integer problem, we are able to develop an efficient algorithm to find its optimal solution. Simulation results with real-world image classification demonstrate that our proposed methods achieve faster convergence with significantly lower computational complexity than existing alternatives. Furthermore, although ADSBF shows marginally inferior performance to GSDS, it offers the advantage of lower computational complexity when the number of devices is large.

Beamforming and Device Selection Design in Federated Learning with Over-the-air Aggregation

TL;DR

This work jointly design uplink receiver beamforming and device selection for over-the-air FL over time-varying wireless channels to maximize the training convergence rate and proposes a Greedy Spatial Device Selection approach, which uses a sequential procedure to select devices based on a measure capturing both the channel strength and the channel correlation to the selected devices.

Abstract

Federated learning (FL) with over-the-air computation can efficiently utilize the communication bandwidth but is susceptible to analog aggregation error. Excluding those devices with weak channel conditions can reduce the aggregation error, but it also limits the amount of local training data for FL, which can reduce the training convergence rate. In this work, we jointly design uplink receiver beamforming and device selection for over-the-air FL over time-varying wireless channels to maximize the training convergence rate. We reformulate this stochastic optimization problem into a mixed-integer program using an upper bound on the global training loss over communication rounds. We then propose a Greedy Spatial Device Selection (GSDS) approach, which uses a sequential procedure to select devices based on a measure capturing both the channel strength and the channel correlation to the selected devices. We show that given the selected devices, the receiver beamforming optimization problem is equivalent to downlink single-group multicast beamforming. To reduce the computational complexity, we also propose an Alternating-optimization-based Device Selection and Beamforming (ADSBF) approach, which solves the receiver beamforming and device selection subproblems alternatingly. In particular, despite the device selection being an integer problem, we are able to develop an efficient algorithm to find its optimal solution. Simulation results with real-world image classification demonstrate that our proposed methods achieve faster convergence with significantly lower computational complexity than existing alternatives. Furthermore, although ADSBF shows marginally inferior performance to GSDS, it offers the advantage of lower computational complexity when the number of devices is large.
Paper Structure (22 sections, 1 theorem, 27 equations, 6 figures, 3 tables, 3 algorithms)

This paper contains 22 sections, 1 theorem, 27 equations, 6 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. FL System
  4. FL with Over-the-Air Analog Aggregation
  5. Problem Formulation
  6. Problem Reformulation Based on Global Training Loss Analysis
  7. Upper Bound on the Global Training Loss
  8. Problem Reformulation via Training Loss Upper Bound
  9. Joint Device Selection and Receiver Beamforming with Analog Aggregation
  10. Greedy Spatial Device Selection (GSDS) Approach
  11. Receiver Beamforming Design Given Device Selection
  12. Greedy Selection of Devices
  13. Computational Complexity
  14. Alternating-optimization-based Device Selection and Beamforming (ADSBF) Approach
  15. Receiver Beamforming Design Given Device Selection $\mathbf{s}$
  16. ...and 7 more sections

Key Result

Proposition 1

Algorithm alg2 produces a global optimal point for problem (eq29).

Figures (6)

  • Figure 1: An illustration of federated learning with over-the-air aggregation.
  • Figure 2: An illustration of the sequential operations performed within communication round t.
  • Figure 3: Average test accuracy over epoch (MNIST dataset).
  • Figure 5: Average number of selected devices for different approaches (MNIST dataset).
  • Figure 6: Average test accuracy over epoch (CIFAR-10 dataset).
  • ...and 1 more figures

Theorems & Definitions (1)

  • Proposition 1