Communication-Efficient Federated Learning with Adaptive Compression under Dynamic Bandwidth

TL;DR

The paper addresses high communication costs in federated learning arising from dynamic and heterogeneous client bandwidth. It introduces AdapComFL, which uses bandwidth awareness and LSTM-based bandwidth prediction to adaptively compress local updates via an enhanced sketch mechanism, enabling aggregation of sketches of varying sizes. Key techniques include fixing sketch columns, elastically adjusting rows with $D_i = T b_i \log_2(1+SNR)$ and $a_i = \text{floor}(D_i / b)$, applying coefficient of variation filtering, size alignment to $a_{max}$, and row-wise averaging followed by median-based decompression. Experiments on real bandwidth traces and standard FL benchmarks show reduced uplink data and time with competitive accuracy compared to FedAvg and SketchFL, highlighting practical impact for scalable FL over heterogeneous networks.

Abstract

Federated learning can train models without directly providing local data to the server. However, the frequent updating of the local model brings the problem of large communication overhead. Recently, scholars have achieved the communication efficiency of federated learning mainly by model compression. But they ignore two problems: 1) network state of each client changes dynamically; 2) network state among clients is not the same. The clients with poor bandwidth update local model slowly, which leads to low efficiency. To address this challenge, we propose a communication-efficient federated learning algorithm with adaptive compression under dynamic bandwidth (called AdapComFL). Concretely, each client performs bandwidth awareness and bandwidth prediction. Then, each client adaptively compresses its local model via the improved sketch mechanism based on his predicted bandwidth. Further, the server aggregates sketched models with different sizes received. To verify the effectiveness of the proposed method, the experiments are based on real bandwidth data which are collected from the network topology we build, and benchmark datasets which are obtained from open repositories. We show the performance of AdapComFL algorithm, and compare it with existing algorithms. The experimental results show that our AdapComFL achieves more efficient communication as well as competitive accuracy compared to existing algorithms.

Figures (10)

• Figure 1: The federated Learning under dynamic bandwidth. The thickness of the line represents the condition of the bandwidth.
• Figure 2: AdapComFL. Firstly, each client carries out bandwidth awareness while training the model, and predicts bandwidth based on aware data to obtain upload data volume, as in Step 1. Thus, the gradient is adaptively compressed and uploaded to the server, i.e., Steps 2-3. Then, the server aggregates all sketch models of different sizes into one and sends it, i.e., Steps 4-5. Finally, the client decompresses the updated sketch model as presented in Step 6.
• Figure 3: Compress and decompress in the sketch mechanism. The left node is the client and the right node is the server. The left node compresses the model gradient $\pmb{g}$ to sketch model $S$ and transmits it to the right node. The right node decompresses $S$ to obtain $\overline{\pmb{g}}$.
• Figure 4: Network topology of bandwidth collection.
• Figure 5: Accuracy results of bandwidth prediction.