Adaptive Federated Learning in Heterogeneous Wireless Networks with Independent Sampling

TL;DR

This work addresses the wall-clock inefficiency of federated learning in heterogeneous wireless networks by introducing independent client sampling that accounts for both data and system heterogeneity. It develops a non-convex convergence bound under independent participation, an adaptive bandwidth allocation scheme, and a practical optimization framework to select sampling probabilities that minimize total training time while meeting a convergence target. The authors propose an efficient algorithm to solve the non-convex sampling problem by bounding per-round time and transforming it into a convex subproblem, then verify the approach on a real hardware prototype, achieving substantial wall-clock time reductions compared with existing sampling methods. The results demonstrate significant practical gains across multiple tasks and datasets, highlighting the method’s applicability to real-world wireless FL ecosystems.

Abstract

Federated Learning (FL) algorithms commonly sample a random subset of clients to address the straggler issue and improve communication efficiency. While recent works have proposed various client sampling methods, they have limitations in joint system and data heterogeneity design, which may not align with practical heterogeneous wireless networks. In this work, we advocate a new independent client sampling strategy to minimize the wall-clock training time of FL, while considering data heterogeneity and system heterogeneity in both communication and computation. We first derive a new convergence bound for non-convex loss functions with independent client sampling and then propose an adaptive bandwidth allocation scheme. Furthermore, we propose an efficient independent client sampling algorithm based on the upper bounds on the convergence rounds and the expected per-round training time, to minimize the wall-clock time of FL, while considering both the data and system heterogeneity. Experimental results under practical wireless network settings with real-world prototype demonstrate that the proposed independent sampling scheme substantially outperforms the current best sampling schemes under various training models and datasets.

Figures (5)

• Figure 1: A heterogeneous federated learning over wireless networks, where, clients with heterogeneous computation and communication capacities, as well as data distribution.
• Figure 2: CNN@MNIST
• Figure 3: MobileNet@CIFAR10
• Figure 4: LSTM@Shakespeare
• Figure 5: Speedup with different $f_{tot}$