Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Analog-digital Scheduling for Federated Learning: A Communication-Efficient Approach

Muhammad Faraz Ul Abrar, Nicolò Michelusi

TL;DR

This work addresses the trade-off between noise-limited analog over-the-air (OTA) aggregation and latency-limited digital transmissions in Federated Learning over wireless networks. It proposes Analog-Digital Federated Learning (ADFL), which jointly optimizes per-round device transmission modes (OTA vs. digital) and digital bit allocation to minimize the mean squared error (MSE) of the estimated global gradient under a delay constraint, using a per-round upper bound that decomposes into digital and OTA components. A key contribution is a structural result (digital devices should be the K smallest-sm devices by the metric $\text{SM}_m = |h_m| / \|\boldsymbol{g}_m\|_\infty$) that reduces scheduling to a linear search over $\mathcal{O}(N+1)$ configurations, along with a convex-duality solution for the bit-allocation subproblem. Numerical results on MNIST show that ADFL consistently outperforms pure OTA and pure digital schemes in both i.i.d. and non-i.i.d. data regimes, achieving higher accuracy with substantially reduced round time and enabling scalable participation in wireless FL.

Abstract

Over-the-air (OTA) computation has recently emerged as a communication-efficient Federated Learning (FL) paradigm to train machine learning models over wireless networks. However, its performance is limited by the device with the worst SNR, resulting in fast yet noisy updates. On the other hand, allocating orthogonal resource blocks (RB) to individual devices via digital channels mitigates the noise problem, at the cost of increased communication latency. In this paper, we address this discrepancy and present ADFL, a novel Analog-Digital FL scheme: in each round, the parameter server (PS) schedules each device to either upload its gradient via the analog OTA scheme or transmit its quantized gradient over an orthogonal RB using the ``digital" scheme. Focusing on a single FL round, we cast the optimal scheduling problem as the minimization of the mean squared error (MSE) on the estimated global gradient at the PS, subject to a delay constraint, yielding the optimal device scheduling configuration and quantization bits for the digital devices. Our simulation results show that ADFL, by scheduling most of the devices in the OTA scheme while also occasionally employing the digital scheme for a few devices, consistently outperforms OTA-only and digital-only schemes, in both i.i.d. and non-i.i.d. settings.

Analog-digital Scheduling for Federated Learning: A Communication-Efficient Approach

TL;DR

This work addresses the trade-off between noise-limited analog over-the-air (OTA) aggregation and latency-limited digital transmissions in Federated Learning over wireless networks. It proposes Analog-Digital Federated Learning (ADFL), which jointly optimizes per-round device transmission modes (OTA vs. digital) and digital bit allocation to minimize the mean squared error (MSE) of the estimated global gradient under a delay constraint, using a per-round upper bound that decomposes into digital and OTA components. A key contribution is a structural result (digital devices should be the K smallest-sm devices by the metric ) that reduces scheduling to a linear search over configurations, along with a convex-duality solution for the bit-allocation subproblem. Numerical results on MNIST show that ADFL consistently outperforms pure OTA and pure digital schemes in both i.i.d. and non-i.i.d. data regimes, achieving higher accuracy with substantially reduced round time and enabling scalable participation in wireless FL.

Abstract

Over-the-air (OTA) computation has recently emerged as a communication-efficient Federated Learning (FL) paradigm to train machine learning models over wireless networks. However, its performance is limited by the device with the worst SNR, resulting in fast yet noisy updates. On the other hand, allocating orthogonal resource blocks (RB) to individual devices via digital channels mitigates the noise problem, at the cost of increased communication latency. In this paper, we address this discrepancy and present ADFL, a novel Analog-Digital FL scheme: in each round, the parameter server (PS) schedules each device to either upload its gradient via the analog OTA scheme or transmit its quantized gradient over an orthogonal RB using the ``digital" scheme. Focusing on a single FL round, we cast the optimal scheduling problem as the minimization of the mean squared error (MSE) on the estimated global gradient at the PS, subject to a delay constraint, yielding the optimal device scheduling configuration and quantization bits for the digital devices. Our simulation results show that ADFL, by scheduling most of the devices in the OTA scheme while also occasionally employing the digital scheme for a few devices, consistently outperforms OTA-only and digital-only schemes, in both i.i.d. and non-i.i.d. settings.
Paper Structure (6 sections, 1 theorem, 17 equations, 2 figures)

This paper contains 6 sections, 1 theorem, 17 equations, 2 figures.

Table of Contents

  1. Introduction
  2. System Model and analog-digital FL
  3. Transmission over the wireless channel and ADFL
  4. Problem Formulation and Proposed Solution
  5. Numerical Results
  6. Conclusion

Key Result

Theorem 1

Given any scheduling configuration with $K<N$ digital devices, the $K$ devices with the smallest SM should be scheduled as digital devices for optimal MSE performance.

Figures (2)

  • Figure 1: Illustration of ADFL system model
  • Figure 2: Test accuracy comparison of ADFL with existing OTA and digital schemes for i.i.d. and non-i.i.d. settings.

Theorems & Definitions (1)

  • Theorem 1