Cluster-Based Multi-Agent Task Scheduling for Space-Air-Ground Integrated Networks

TL;DR

This paper tackles scalable task scheduling in Space-Air-Ground Integrated Networks by introducing CMADDPG, a cluster-based, cooperative multi-agent reinforcement learning approach. It combines a two-stage UAV clustering method (KMDUC) with a centralized-training, distributed-execution MADDPG framework to enable efficient, low-overhead offloading decisions within clusters. The approach jointly optimizes resource allocation, load balancing, and delay-sensitive performance, achieving at least 25% higher system profit than strong baselines and robust performance across diverse scenarios. By leveraging satellites for global coordination and UAVs for flexible execution, CMADDPG demonstrates substantial potential for scalable, intelligent resource management in SAGINs.

Abstract

The Space-Air-Ground Integrated Network (SAGIN) framework is a crucial foundation for future networks, where satellites and aerial nodes assist in computational task offloading. The low-altitude economy, leveraging the flexibility and multifunctionality of Unmanned Aerial Vehicles (UAVs) in SAGIN, holds significant potential for development in areas such as communication and sensing. However, effective coordination is needed to streamline information exchange and enable efficient system resource allocation. In this paper, we propose a Clustering-based Multi-agent Deep Deterministic Policy Gradient (CMADDPG) algorithm to address the multi-UAV cooperative task scheduling challenges in SAGIN. The CMADDPG algorithm leverages dynamic UAV clustering to partition UAVs into clusters, each managed by a Cluster Head (CH) UAV, facilitating a distributed-centralized control approach. Within each cluster, UAVs delegate offloading decisions to the CH UAV, reducing intra-cluster communication costs and decision conflicts, thereby enhancing task scheduling efficiency. Additionally, by employing a multi-agent reinforcement learning framework, the algorithm leverages the extensive coverage of satellites to achieve centralized training and distributed execution of multi-agent tasks, while maximizing overall system profit through optimized task offloading decision-making. Simulation results reveal that the CMADDPG algorithm effectively optimizes resource allocation, minimizes queue delays, maintains balanced load distribution, and surpasses existing methods by achieving at least a 25\% improvement in system profit, showcasing its robustness and adaptability across diverse scenarios.

Figures (9)

• Figure 1: Task scheduling cooperation under SAGIN.
• Figure 2: Schematic of the cluster maintenance algorithm.
• Figure 3: Centralized training and distributed execution cooperative framework.
• Figure 4: Reward during training cycles.
• Figure 5: Episodes for convergence as UAV count increases.