Fair Resource Allocation For Hierarchical Federated Edge Learning in Space-Air-Ground Integrated Networks via Deep Reinforcement Learning with Hybrid Control

TL;DR

The paper addresses efficient, fair, multi-task learning over a space-air-ground integrated network by embedding hierarchical federated learning with UAV-edge and LEO-cloud servers. It introduces a DSAC-based, hybrid-action DRL framework (H-DSAC) to jointly optimize UAV trajectories, user-UAV pairings, UAV-satellite pairings, aggregation weights, and final aggregation under tight satellite service windows. A dynamic fairness reward is proposed to balance convergence across multiple tasks, and a decoupling-coupling strategy enables effective handling of discrete-continuous action spaces. Simulation results demonstrate that the proposed approach achieves higher convergence accuracy and fairness than several baselines, underscoring its potential for real-time, fair federated learning in SAGINs. The work advances practical federated learning in globally distributed, highly dynamic networks and highlights important design choices like trajectory planning and ISL-enabled aggregation for scalable edge intelligence.

Abstract

The space-air-ground integrated network (SAGIN) has become a crucial research direction in future wireless communications due to its ubiquitous coverage, rapid and flexible deployment, and multi-layer cooperation capabilities. However, integrating hierarchical federated learning (HFL) with edge computing and SAGINs remains a complex open issue to be resolved. This paper proposes a novel framework for applying HFL in SAGINs, utilizing aerial platforms and low Earth orbit (LEO) satellites as edge servers and cloud servers, respectively, to provide multi-layer aggregation capabilities for HFL. The proposed system also considers the presence of inter-satellite links (ISLs), enabling satellites to exchange federated learning models with each other. Furthermore, we consider multiple different computational tasks that need to be completed within a limited satellite service time. To maximize the convergence performance of all tasks while ensuring fairness, we propose the use of the distributional soft-actor-critic (DSAC) algorithm to optimize resource allocation in the SAGIN and aggregation weights in HFL. Moreover, we address the efficiency issue of hybrid action spaces in deep reinforcement learning (DRL) through a decoupling and recoupling approach, and design a new dynamic adjusting reward function to ensure fairness among multiple tasks in federated learning. Simulation results demonstrate the superiority of our proposed algorithm, consistently outperforming baseline approaches and offering a promising solution for addressing highly complex optimization problems in SAGINs.

Figures (12)

• Figure 1: System model of a SAGIN.
• Figure 2: The coverage area of a LEO satellite in SAGINs.
• Figure 3: The structure of the hierarchical federated edge learning in SAGINs.
• Figure 4: The structure of DSAC.
• Figure 5: The structure of hybrid action space in DSAC.