Efficient Asynchronous Federated Learning with Sparsification and Quantization

TL;DR

TEASQ-Fed tackles the inefficiencies of traditional FL by enabling asynchronous participation from idle edge devices, guided by a concurrency cap to balance speed and stability. It integrates a caching mechanism and a staleness-aware weighted averaging strategy to mitigate non IID data effects and update delays, while dynamically adjusting sparsification and quantization to reduce communication overhead. The approach yields up to 16.67% accuracy improvements and up to 2x faster convergence on Fashion-MNIST with non IID data, outperforming several baselines. This work meaningfully enhances edge utilization and robustness in practical FL deployments, offering a scalable path toward faster, more reliable decentralized learning.

Abstract

While data is distributed in multiple edge devices, Federated Learning (FL) is attracting more and more attention to collaboratively train a machine learning model without transferring raw data. FL generally exploits a parameter server and a large number of edge devices during the whole process of the model training, while several devices are selected in each round. However, straggler devices may slow down the training process or even make the system crash during training. Meanwhile, other idle edge devices remain unused. As the bandwidth between the devices and the server is relatively low, the communication of intermediate data becomes a bottleneck. In this paper, we propose Time-Efficient Asynchronous federated learning with Sparsification and Quantization, i.e., TEASQ-Fed. TEASQ-Fed can fully exploit edge devices to asynchronously participate in the training process by actively applying for tasks. We utilize control parameters to choose an appropriate number of parallel edge devices, which simultaneously execute the training tasks. In addition, we introduce a caching mechanism and weighted averaging with respect to model staleness to further improve the accuracy. Furthermore, we propose a sparsification and quantitation approach to compress the intermediate data to accelerate the training. The experimental results reveal that TEASQ-Fed improves the accuracy (up to 16.67% higher) while accelerating the convergence of model training (up to twice faster).

Figures (9)

• Figure 1: The overview of TEASQ-Fed protocol.
• Figure 2: Impacts of $\mu$ in terms of accuracy vs. training time with non-IID dataset.
• Figure 3: Impacts of $C$ in terms of accuracy vs. training time with non-IID and IID dataset.
• Figure 4: Impacts of $C$ and time required to reach the target test accuracy with non-IID and IID dataset. In (b), FedAvg fails to reach the target accuracy.
• Figure 5: Impacts of $C$ in terms of accuracy vs. training rounds with non-IID and IID dataset.