Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spectrum Breathing: Protecting Over-the-Air Federated Learning Against Interference

Zhanwei Wang, Kaibin Huang, Yonina C. Eldar

TL;DR

Spectrum Breathing is proposed, which cascades stochastic-gradient pruning and spread spectrum to suppress interference without bandwidth expansion and shows a performance tradeoff between gradient-pruning and interference-induced error as regulated by the breathing depth.

Abstract

Federated Learning (FL) is a widely embraced paradigm for distilling artificial intelligence from distributed mobile data. However, the deployment of FL in mobile networks can be compromised by exposure to interference from neighboring cells or jammers. Existing interference mitigation techniques require multi-cell cooperation or at least interference channel state information, which is expensive in practice. On the other hand, power control that treats interference as noise may not be effective due to limited power budgets, and also that this mechanism can trigger countermeasures by interference sources. As a practical approach for protecting FL against interference, we propose Spectrum Breathing, which cascades stochastic-gradient pruning and spread spectrum to suppress interference without bandwidth expansion. The cost is higher learning latency by exploiting the graceful degradation of learning speed due to pruning. We synchronize the two operations such that their levels are controlled by the same parameter, Breathing Depth. To optimally control the parameter, we develop a martingale-based approach to convergence analysis of Over-the-Air FL with spectrum breathing, termed AirBreathing FL. We show a performance tradeoff between gradient-pruning and interference-induced error as regulated by the breathing depth. Given receive SIR and model size, the optimization of the tradeoff yields two schemes for controlling the breathing depth that can be either fixed or adaptive to channels and the learning process. As shown by experiments, in scenarios where traditional Over-the-Air FL fails to converge in the presence of strong interference, AirBreahing FL with either fixed or adaptive breathing depth can ensure convergence where the adaptive scheme achieves close-to-ideal performance.

Spectrum Breathing: Protecting Over-the-Air Federated Learning Against Interference

TL;DR

Spectrum Breathing is proposed, which cascades stochastic-gradient pruning and spread spectrum to suppress interference without bandwidth expansion and shows a performance tradeoff between gradient-pruning and interference-induced error as regulated by the breathing depth.

Abstract

Federated Learning (FL) is a widely embraced paradigm for distilling artificial intelligence from distributed mobile data. However, the deployment of FL in mobile networks can be compromised by exposure to interference from neighboring cells or jammers. Existing interference mitigation techniques require multi-cell cooperation or at least interference channel state information, which is expensive in practice. On the other hand, power control that treats interference as noise may not be effective due to limited power budgets, and also that this mechanism can trigger countermeasures by interference sources. As a practical approach for protecting FL against interference, we propose Spectrum Breathing, which cascades stochastic-gradient pruning and spread spectrum to suppress interference without bandwidth expansion. The cost is higher learning latency by exploiting the graceful degradation of learning speed due to pruning. We synchronize the two operations such that their levels are controlled by the same parameter, Breathing Depth. To optimally control the parameter, we develop a martingale-based approach to convergence analysis of Over-the-Air FL with spectrum breathing, termed AirBreathing FL. We show a performance tradeoff between gradient-pruning and interference-induced error as regulated by the breathing depth. Given receive SIR and model size, the optimization of the tradeoff yields two schemes for controlling the breathing depth that can be either fixed or adaptive to channels and the learning process. As shown by experiments, in scenarios where traditional Over-the-Air FL fails to converge in the presence of strong interference, AirBreahing FL with either fixed or adaptive breathing depth can ensure convergence where the adaptive scheme achieves close-to-ideal performance.
Paper Structure (35 sections, 7 theorems, 52 equations, 5 figures, 1 algorithm)

This paper contains 35 sections, 7 theorems, 52 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Models and Metrics
  3. Learning Model
  4. Communication Model
  5. Performance Metric
  6. Distortion from Pruning - Generic Data versus Federated Learning
  7. Generic Data Pruning
  8. Stochastic Gradient Pruning
  9. Why Random Pruning for AirFL?
  10. Overview of Spectrum Breathing
  11. Transmitter Design
  12. Spectrum Contraction
  13. Channel Inversion
  14. Spectrum Spreading
  15. Receiver Design
  16. ...and 20 more sections

Key Result

Lemma 1

Consider an FL system updating as in (gsgd) with a learning rate $\eta<2c\varepsilon \mathbb{G}^2$. If the algorithm has not converged by round $n$, the process defined as is a rate supermartingale with horizon $A=\infty$, where $\mathbb{G}^2 \geq \zeta^2+\sigma_g^2$ is the upper bound of the squared norm of aggregated gradients. Under Assumptions AS1-AS2, $W_n({\mathbf{w}}(n), \ldots, {\mathbf{w

Figures (5)

  • Figure 1: System diagram of AirFL system perturbed by interference.
  • Figure 2: Transceiver of the spectrum breathing system.
  • Figure 3: (a) Performance comparison between AirBreathing FL (in both the cases of fixed and adaptive breathing depth) and benchmarking schemes; (b) Comparison between pruning without spreading ($\gamma$ = 0.5, 0.1) and AirBreathing FL.
  • Figure 4: (a) Performance comparison between COTAF and AirBreathing FL; (b) Comparison between random pruning and importance-aware pruning for AirBreathing FL.
  • Figure 5: The effects of (a) number of devices and (b) receive SIR on the learning performance for given communication time of $7\times 10^6$ chips.

Theorems & Definitions (9)

  • Definition 1: RN97
  • Lemma 1: RN97, Lemma 1
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Lemma 5
  • Definition 2: Breathing Depth
  • Theorem 1
  • Lemma 6