Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Constrained Over-the-Air Model Updating for Wireless Online Federated Learning with Delayed Information

Juncheng Wang, Yituo Liu, Ben Liang, Min Dong

TL;DR

The paper addresses online over-the-air federated learning under delays in both local loss information and channel state information, coupled with time-varying power constraints. It introduces COMUDO, which uses a novel lower-and-upper-bounded virtual queue and a Lyapunov drift analysis to derive dynamic and static regret bounds as well as hard constraint violations, while yielding closed-form, per-device updates. The proposed method achieves sublinear dynamic and static regret and sublinear hard constraint violation under mild system variation conditions, and simulations on MNIST, Fashion-MNIST, and CIFAR-10 show substantial accuracy gains over state-of-the-art baselines, especially in low-power regimes. These results demonstrate practical improvements for wireless online FL where channel delays and power constraints are unavoidable, enabling more robust edge learning in realistic networks.

Abstract

We study online federated learning over a wireless network, where the central server updates an online global model sequence to minimize the time-varying loss of multiple local devices over time. The server updates the global model through over-the-air model-difference aggregation from the local devices over a noisy multiple-access fading channel. We consider the practical scenario where information on both the local loss functions and the channel states is delayed, and each local device is under a time-varying power constraint. We propose Constrained Over-the-air Model Updating with Delayed infOrmation (COMUDO), where a new lower-and-upper-bounded virtual queue is introduced to counter the delayed information and control the hard constraint violation. We show that its local model updates can be efficiently computed in closed-form expressions. Furthermore, through a new Lyapunov drift analysis, we show that COMUDO provides bounds on the dynamic regret, static regret, and hard constraint violation. Simulation results on image classification tasks under practical wireless network settings show substantial accuracy gain of COMUDO over state-of-the-art approaches, especially in the low-power region.

Constrained Over-the-Air Model Updating for Wireless Online Federated Learning with Delayed Information

TL;DR

The paper addresses online over-the-air federated learning under delays in both local loss information and channel state information, coupled with time-varying power constraints. It introduces COMUDO, which uses a novel lower-and-upper-bounded virtual queue and a Lyapunov drift analysis to derive dynamic and static regret bounds as well as hard constraint violations, while yielding closed-form, per-device updates. The proposed method achieves sublinear dynamic and static regret and sublinear hard constraint violation under mild system variation conditions, and simulations on MNIST, Fashion-MNIST, and CIFAR-10 show substantial accuracy gains over state-of-the-art baselines, especially in low-power regimes. These results demonstrate practical improvements for wireless online FL where channel delays and power constraints are unavoidable, enabling more robust edge learning in realistic networks.

Abstract

We study online federated learning over a wireless network, where the central server updates an online global model sequence to minimize the time-varying loss of multiple local devices over time. The server updates the global model through over-the-air model-difference aggregation from the local devices over a noisy multiple-access fading channel. We consider the practical scenario where information on both the local loss functions and the channel states is delayed, and each local device is under a time-varying power constraint. We propose Constrained Over-the-air Model Updating with Delayed infOrmation (COMUDO), where a new lower-and-upper-bounded virtual queue is introduced to counter the delayed information and control the hard constraint violation. We show that its local model updates can be efficiently computed in closed-form expressions. Furthermore, through a new Lyapunov drift analysis, we show that COMUDO provides bounds on the dynamic regret, static regret, and hard constraint violation. Simulation results on image classification tasks under practical wireless network settings show substantial accuracy gain of COMUDO over state-of-the-art approaches, especially in the low-power region.
Paper Structure (21 sections, 9 theorems, 28 equations, 4 figures, 1 table, 2 algorithms)

This paper contains 21 sections, 9 theorems, 28 equations, 4 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Communication-Efficient Error-Free Distributed Learning
  4. Over-the-Air Federated Learning
  5. Constrained Online Optimization
  6. Online Over-the-Air Federated Learning
  7. OTA FL with Delayed Information
  8. Online Optimization Problem Formulation
  9. Constrained Over-the-air Model Updating with Delayed infOrmation (COMUDO)
  10. Preliminaries
  11. Lower-and-Upper-Bounded Virtual Queue
  12. COMUDO Algorithm
  13. Performance Bounds of COMUDO
  14. Bounding the Regret
  15. Bounding the Hard Constraint Violation
  16. ...and 6 more sections

Key Result

Lemma 1

$\textbf{P}$ has the following properties: where $E=\frac{z_\text{\tiny{UB}}}{N\lambda}$, $R=2\sqrt{d}x_\text{\tiny{UB}}$, and $G=\max_n\{\max\{P^n,\frac{d(R+E)^2\lambda^2}{h_\text{\tiny{LB}}^2}-P^n\}\}$.

Figures (4)

  • Figure 1: An illustration of OTA FL with delayed information. The solid arrows indicate signal transmission, and the dashed arrows indicate information flow. At the beginning of round $t$, when device $n$ updates its local model $\mathbf{x}_t^n$, it only has the delayed local loss function information $f_{t-1}^n(\mathbf{x})$ and the delayed local channel state information $\mathbf{h}_{t-1}^n$. At the end of round $t$, only after updating $\mathbf{x}_t^n$, the information of $f_t^n(\mathbf{x})$ and $\mathbf{h}_t^n$ becomes available to device $n$.
  • Figure 2: Averaged test accuracy and normalized hard power violation for convex logistic regression on MNIST.
  • Figure 3: The impact of transmit power limit $P$ on the averaged test accuracy and training loss.
  • Figure 4: Averaged test accuracy and normalized hard power violation for non-convex neural network training on MNIST.

Theorems & Definitions (13)

  • Remark 1
  • Lemma 1
  • Remark 2
  • Lemma 2
  • Remark 3
  • Lemma 3
  • Theorem 1
  • Theorem 2
  • Lemma 4
  • Theorem 3
  • ...and 3 more