Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks

Shuai Wang, Yanqing Xu, Chaoqun You, Mingjie Shao, Tony Q. S. Quek

TL;DR

This work tackles the communication bottleneck in federated learning over wireless edge networks with device heterogeneity. It introduces FedQVR, a variance-reduced FL algorithm that uses per-device control variates and a server CV to achieve inter-device variance reduction while enabling quantized uplink updates, plus a time-varying heterogeneous local update (HLU) mechanism. The authors prove a sublinear convergence rate for nonconvex objectives under mild assumptions and analyze how quantization and partial participation influence performance, demonstrating substantial communication savings. To address non-ideal wireless channels, FedQVR-E jointly optimizes bandwidth and quantization bits, improving transmission reliability and convergence under delay constraints. Extensive CIFAR-10 and MNIST experiments show FedQVR and FedQVR-E outperform baselines in both convergence speed and communication efficiency, highlighting practical impact for edge deployments.

Abstract

Federated learning (FL) has been recognized as a viable solution for local-privacy-aware collaborative model training in wireless edge networks, but its practical deployment is hindered by the high communication overhead caused by frequent and costly server-device synchronization. Notably, most existing communication-efficient FL algorithms fail to reduce the significant inter-device variance resulting from the prevalent issue of device heterogeneity. This variance severely decelerates algorithm convergence, increasing communication overhead and making it more challenging to achieve a well-performed model. In this paper, we propose a novel communication-efficient FL algorithm, named FedQVR, which relies on a sophisticated variance-reduced scheme to achieve heterogeneity-robustness in the presence of quantized transmission and heterogeneous local updates among active edge devices. Comprehensive theoretical analysis justifies that FedQVR is inherently resilient to device heterogeneity and has a comparable convergence rate even with a small number of quantization bits, yielding significant communication savings. Besides, considering non-ideal wireless channels, we propose FedQVR-E which enhances the convergence of FedQVR by performing joint allocation of bandwidth and quantization bits across devices under constrained transmission delays. Extensive experimental results are also presented to demonstrate the superior performance of the proposed algorithms over their counterparts in terms of both communication efficiency and application performance.

Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks

TL;DR

This work tackles the communication bottleneck in federated learning over wireless edge networks with device heterogeneity. It introduces FedQVR, a variance-reduced FL algorithm that uses per-device control variates and a server CV to achieve inter-device variance reduction while enabling quantized uplink updates, plus a time-varying heterogeneous local update (HLU) mechanism. The authors prove a sublinear convergence rate for nonconvex objectives under mild assumptions and analyze how quantization and partial participation influence performance, demonstrating substantial communication savings. To address non-ideal wireless channels, FedQVR-E jointly optimizes bandwidth and quantization bits, improving transmission reliability and convergence under delay constraints. Extensive CIFAR-10 and MNIST experiments show FedQVR and FedQVR-E outperform baselines in both convergence speed and communication efficiency, highlighting practical impact for edge deployments.

Abstract

Federated learning (FL) has been recognized as a viable solution for local-privacy-aware collaborative model training in wireless edge networks, but its practical deployment is hindered by the high communication overhead caused by frequent and costly server-device synchronization. Notably, most existing communication-efficient FL algorithms fail to reduce the significant inter-device variance resulting from the prevalent issue of device heterogeneity. This variance severely decelerates algorithm convergence, increasing communication overhead and making it more challenging to achieve a well-performed model. In this paper, we propose a novel communication-efficient FL algorithm, named FedQVR, which relies on a sophisticated variance-reduced scheme to achieve heterogeneity-robustness in the presence of quantized transmission and heterogeneous local updates among active edge devices. Comprehensive theoretical analysis justifies that FedQVR is inherently resilient to device heterogeneity and has a comparable convergence rate even with a small number of quantization bits, yielding significant communication savings. Besides, considering non-ideal wireless channels, we propose FedQVR-E which enhances the convergence of FedQVR by performing joint allocation of bandwidth and quantization bits across devices under constrained transmission delays. Extensive experimental results are also presented to demonstrate the superior performance of the proposed algorithms over their counterparts in terms of both communication efficiency and application performance.
Paper Structure (33 sections, 12 theorems, 85 equations, 5 figures, 1 table, 2 algorithms)

This paper contains 33 sections, 12 theorems, 85 equations, 5 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related works
  4. System Model and Problem formulation
  5. Federated learning
  6. Review of FedAvg and SCAFFOLD
  7. Quantized transmission
  8. Proposed FedQVR Algorithm and Its Theoretical Analysis
  9. Algorithm development
  10. Robust inter-device variance reduction
  11. Convergence analysis
  12. An Enhanced FedQVR Algorithm
  13. Wireless Model Description and Problem Formulation
  14. Solving the Radio Resource Allocation Problem
  15. Experiment Results
  16. ...and 18 more sections

Key Result

Lemma 1

For any round $r \geq 0$ and device $i \in {\mathcal{A}}^r$, it holds that $\forall t = 0, \ldots, E_i^r - 1$, where ${\bf b}_i^r \triangleq [b_i^{r, 0}, b_i^{r, 1}, \ldots, b_i^{r, E_i^r-1}]^\top \in {\mathbb{R}}^{E_i^r}$, $b_i^{r, t} =(\frac{1}{1 + \gamma \eta})^{E_i^r - t}$, $G_i^r = \sum_{t= 0}^{E_i^r - 1}\frac{b_i^{r, t} }{\|{\bf b}_i^r\|_1} g_i({\bm \theta}_i^{r,t})$.

Figures (5)

  • Figure 1: Convergence performance of the proposed FedQVR algorithm with various choices of $a$ and $B_i^r$
  • Figure 2: Performance comparison between the proposed FedQVR algorithm and baseline algorithms without quantization
  • Figure 3: Performance comparison between the proposed FedQVR algorithm and baseline algorithms with quantization
  • Figure 4: Performance comparison between the proposed FedQVR algorithm and baseline algorithms with different $m$ or under the setting of HLU
  • Figure 5: Convergence performance of the proposed FedQVR-E algorithm and some baseline algorithms under different settings.

Theorems & Definitions (17)

  • Lemma 1
  • Lemma 2
  • Theorem 1
  • Remark 1
  • Remark 2
  • Corollary 1
  • Remark 3
  • Corollary 2
  • Remark 4
  • Remark 5
  • ...and 7 more