FedSparQ: Adaptive Sparse Quantization with Error Feedback for Robust & Efficient Federated Learning

TL;DR

FedSparQ tackles the communication bottleneck in federated learning by combining adaptive per‑round threshold sparsification, float16 quantization of retained updates, and error‑feedback to preserve convergence under data heterogeneity. The framework also integrates FedProx to mitigate client drift and applies server‑side momentum to stabilize global updates, achieving large bandwidth savings with minimal accuracy loss. The paper provides theoretical convergence guarantees for non‑convex objectives and shows empirically that FedSparQ reduces uploaded data by up to 90% while matching or exceeding FedAvg accuracy across IID and non‑IID benchmarks, notably on CIFAR‑10. This work offers a practical, hyperparameter‑light solution for bandwidth‑constrained FL and lays groundwork for adaptive precision and privacy‑preserving extensions in real‑world deployments.

Abstract

Federated Learning (FL) enables collaborative model training across decentralized clients while preserving data privacy by keeping raw data local. However, FL suffers from significant communication overhead due to the frequent exchange of high-dimensional model updates over constrained networks. In this paper, we present FedSparQ, a lightweight compression framework that dynamically sparsifies the gradient of each client through an adaptive threshold, applies half-precision quantization to retained entries and integrates residuals from error feedback to prevent loss of information. FedSparQ requires no manual tuning of sparsity rates or quantization schedules, adapts seamlessly to both homogeneous and heterogeneous data distributions, and is agnostic to model architecture. Through extensive empirical evaluation on vision benchmarks under independent and identically distributed (IID) and non-IID data, we show that FedSparQ substantially reduces communication overhead (reducing by 90% of bytes sent compared to FedAvg) while preserving or improving model accuracy (improving by 6% compared to FedAvg non-compressed solution or to state-of-the-art compression models) and enhancing convergence robustness (by 50%, compared to the other baselines). Our approach provides a practical, easy-to-deploy solution for bandwidth-constrained federated deployments and lays the groundwork for future extensions in adaptive precision and privacy-preserving protocols.

Figures (6)

• Figure 1: Taxonomy of communication-efficient federated learning (FL). The top node states the goal; the middle layer distinguishes three complementary families: (i) Quantization; (ii) Sparsification; (iii) Compression (scheduling/physical-layer). Arrows to Hybrid Techniques indicate methods that combine multiple techniques to obtain multiplicative savings while preserving convergence.
• Figure 2: FedSparQ system model.
• Figure 3: Communication Cost (Bytes sent) on CIFAR-10, FMNIST and MNIST.
• Figure 4: Test accuracy vs. rounds on different dataset.
• Figure 5: Test loss vs. rounds on different dataset.