Asyn2F: An Asynchronous Federated Learning Framework with Bidirectional Model Aggregation

TL;DR

The paper addresses inefficiencies in asynchronous federated learning caused by heterogeneous worker resources and stale information. It introduces Asyn2F, a bidirectional framework where the server aggregates local updates asynchronously while workers continuously integrate new global versions into their ongoing training. Two novel aggregation strategies are proposed: a server-side delay-aware, data-quality-weighted scheme and a worker-side local-aggregation rule that blends global and local updates to mitigate obsolescence. Experiments on CIFAR10 and EMBER show superior accuracy and faster convergence than baselines, and the framework demonstrates practical deployment through cloud storage, RabbitMQ-based messaging, and real-time monitoring with privacy considerations.

Abstract

In federated learning, the models can be trained synchronously or asynchronously. Many research works have focused on developing an aggregation method for the server to aggregate multiple local models into the global model with improved performance. They ignore the heterogeneity of the training workers, which causes the delay in the training of the local models, leading to the obsolete information issue. In this paper, we design and develop Asyn2F, an Asynchronous Federated learning Framework with bidirectional model aggregation. By bidirectional model aggregation, Asyn2F, on one hand, allows the server to asynchronously aggregate multiple local models and results in a new global model. On the other hand, it allows the training workers to aggregate the new version of the global model into the local model, which is being trained even in the middle of a training epoch. We develop Asyn2F considering the practical implementation requirements such as using cloud services for model storage and message queuing protocols for communications. Extensive experiments with different datasets show that the models trained by Asyn2F achieve higher performance compared to the state-of-the-art techniques. The experiments also demonstrate the effectiveness, practicality, and scalability of Asyn2F, making it ready for deployment in real scenarios.

Figures (9)

• Figure 1: General Architecture of Asyn2F.
• Figure 2: Illustration of asynchronous updates with Asyn2F.
• Figure 3: Sequence Diagram between Server and Workers.
• Figure 4: Data size and label distribution of 10 sub-datasets (non-overlapping and non-iid).
• Figure 5: Performance of the global model trained with 10 workers by different techniques (overlapping sub-datasets).