Federated Learning Model Aggregation in Heterogenous Aerial and Space Networks

TL;DR

The paper addresses data heterogeneity in federated learning for aerial and space networks under privacy constraints. It introduces WeiAvgCS, a weighted averaging and client-selection framework that emphasizes updates from high-diversity clients using a projection-based proxy for diversity to avoid sharing private label distributions. Theoretical analysis and experiments on FashionMNIST and CIFAR10 demonstrate accelerated convergence and compatibility with other FL methods such as FedProx, MOON, and Scaffold. This approach enhances FL performance in heterogeneous ASN environments while preserving privacy and incurring minimal additional communication or computation costs, making it practically impactful for edge-enabled ASN deployments.

Abstract

Federated learning offers a promising approach under the constraints of networking and data privacy constraints in aerial and space networks (ASNs), utilizing large-scale private edge data from drones, balloons, and satellites. Existing research has extensively studied the optimization of the learning process, computing efficiency, and communication overhead. An important yet often overlooked aspect is that participants contribute predictive knowledge with varying diversity of knowledge, affecting the quality of the learned federated models. In this paper, we propose a novel approach to address this issue by introducing a Weighted Averaging and Client Selection (WeiAvgCS) framework that emphasizes updates from high-diversity clients and diminishes the influence of those from low-diversity clients. Direct sharing of the data distribution may be prohibitive due to the additional private information that is sent from the clients. As such, we introduce an estimation for the diversity using a projection-based method. Extensive experiments have been performed to show WeiAvgCS's effectiveness. WeiAvgCS could converge 46% faster on FashionMNIST and 38% faster on CIFAR10 than its benchmarks on average in our experiments.

Figures (7)

• Figure 1: (a) Different devices would possess varying diversity levels of data concerning their locations, including orbiting heights and cruising directions. When performing the FL aggregation, we propose to assign higher weights to clients with more diverse data. (b) The projection of client $k$'s local update ($\textbf{w}^i$ to $\textbf{w}_k^i$) at the $i$th round onto the global update ($\textbf{w}^i$ to $\textbf{w}^{i+1}$).
• Figure 2: Correlation between projection and variance of WeiAvgCS with different $\lambda$ on FashionMNIST and CIFAR10.
• Figure 3: WeiAvgCS($\lambda$) with using (a) variance and (b) projection.
• Figure 4: Test accuracy of FedAvg, WeiAvgCS with 3 benchmarks (a) FedProx, (b) MOON, (c) Scaffold and their corresponding combination on FashionMNIST(p=1).
• Figure 5: Test accuracy of FedAvg, WeiAvgCS with 3 benchmarks (a) FedProx, (b) MOON, (c) Scaffold and their corresponding combination on CIFAR10(p=1).