Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Massive Digital Over-the-Air Computation for Communication-Efficient Federated Edge Learning

Li Qiao, Zhen Gao, Mahdi Boloursaz Mashhadi, Deniz Gündüz

TL;DR

This paper proposes a massive digital AirComp (MD-AirComp) scheme, that leverages an unsourced massive access protocol, to enhance compatibility with both current and next-generation wireless networks, and applies it to federated edge learning (FEEL), and shows that it significantly accelerates FEEL convergence compared to state-of-the-art while using the same amount of communication resources.

Abstract

Over-the-air computation (AirComp) is a promising technology converging communication and computation over wireless networks, which can be particularly effective in model training, inference, and more emerging edge intelligence applications. AirComp relies on uncoded transmission of individual signals, which are added naturally over the multiple access channel thanks to the superposition property of the wireless medium. Despite significantly improved communication efficiency, how to accommodate AirComp in the existing and future digital communication networks, that are based on discrete modulation schemes, remains a challenge. This paper proposes a massive digital AirComp (MD-AirComp) scheme, that leverages an unsourced massive access protocol, to enhance compatibility with both current and next-generation wireless networks. MD-AirComp utilizes vector quantization to reduce the uplink communication overhead, and employs shared quantization and modulation codebooks. At the receiver, we propose a near-optimal approximate message passing-based algorithm to compute the model aggregation results from the superposed sequences, which relies on estimating the number of devices transmitting each code sequence, rather than trying to decode the messages of individual transmitters. We apply MD-AirComp to the federated edge learning (FEEL), and show that it significantly accelerates FEEL convergence compared to state-of-the-art while using the same amount of communication resources. To support further research and ensure reproducibility, we have made our code available at https://github.com/liqiao19/MD-AirComp.

Massive Digital Over-the-Air Computation for Communication-Efficient Federated Edge Learning

TL;DR

This paper proposes a massive digital AirComp (MD-AirComp) scheme, that leverages an unsourced massive access protocol, to enhance compatibility with both current and next-generation wireless networks, and applies it to federated edge learning (FEEL), and shows that it significantly accelerates FEEL convergence compared to state-of-the-art while using the same amount of communication resources.

Abstract

Over-the-air computation (AirComp) is a promising technology converging communication and computation over wireless networks, which can be particularly effective in model training, inference, and more emerging edge intelligence applications. AirComp relies on uncoded transmission of individual signals, which are added naturally over the multiple access channel thanks to the superposition property of the wireless medium. Despite significantly improved communication efficiency, how to accommodate AirComp in the existing and future digital communication networks, that are based on discrete modulation schemes, remains a challenge. This paper proposes a massive digital AirComp (MD-AirComp) scheme, that leverages an unsourced massive access protocol, to enhance compatibility with both current and next-generation wireless networks. MD-AirComp utilizes vector quantization to reduce the uplink communication overhead, and employs shared quantization and modulation codebooks. At the receiver, we propose a near-optimal approximate message passing-based algorithm to compute the model aggregation results from the superposed sequences, which relies on estimating the number of devices transmitting each code sequence, rather than trying to decode the messages of individual transmitters. We apply MD-AirComp to the federated edge learning (FEEL), and show that it significantly accelerates FEEL convergence compared to state-of-the-art while using the same amount of communication resources. To support further research and ensure reproducibility, we have made our code available at https://github.com/liqiao19/MD-AirComp.
Paper Structure (34 sections, 3 theorems, 53 equations, 9 figures, 2 tables, 2 algorithms)

This paper contains 34 sections, 3 theorems, 53 equations, 9 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contributions
  4. System Model
  5. The Proposed MD-AirComp Scheme
  6. Transmitter Design
  7. Quantization Design
  8. Codebook-Based Modulation
  9. UL Massive Access Model
  10. Receiver Design
  11. CS-Based Detection
  12. Digital Aggregation
  13. Model Update
  14. Proposed CS-Based Digital Aggregation Algorithm
  15. Problem Formulation
  16. ...and 19 more sections

Key Result

Proposition 1

Substituting (eq:BayesDetail1), (eq:BayesDetail2), and (eq:prior2) into (eq:Bayes), the posterior distribution of $x_{n,m}$, $m\in\{2,3,...,M\}$ can be reformulated as where we have

Figures (9)

  • Figure 1: Illustration of the FEEL scenario in cellular networks.
  • Figure 2: The schematic diagram of the proposed MD-AirComp scheme, where the modules with dotted boxes are specific to FEEL.
  • Figure 3: Test accuracy vs training rounds under different VQ parameters.
  • Figure 4: FEEL performance comparison of the proposed MD-AirComp scheme with the benchmarks: (a) Test accuracy vs training rounds; (b) Test accuracy vs the number of transmitted symbols.
  • Figure 5: Wireless transmission performance of the proposed MD-AirComp scheme: (a) Sparsity ratio and device collision probability vs training rounds; (b) Detection NMSE vs training rounds.
  • ...and 4 more figures

Theorems & Definitions (9)

  • Remark 1
  • Remark 2
  • Proposition 1
  • proof
  • Remark 3
  • Theorem 1
  • proof
  • Remark 4
  • Corollary 1