Spatiotemporal Continual Learning for Mobile Edge UAV Networks: Mitigating Catastrophic Forgetting

TL;DR

This paper addresses the critical challenge of coordinating mobile edge UAV networks to maintain robust service in highly dynamic spatiotemporal environments with a computationally efficient Spatiotemporal Continual Learning (STCL) framework realized through a Group-Decoupled Multi-Agent Proximal Policy Optimization (G-MAPPO) algorithm.

Abstract

This paper addresses the critical challenge of coordinating mobile edge UAV networks to maintain robust service in highly dynamic spatiotemporal environments. Conventional Deep Reinforcement Learning (DRL) approaches often suffer from catastrophic forgetting when transitioning between distinct task scenarios, such as moving from dense urban clusters to sparse rural areas. These transitions typically necessitate computationally expensive retraining or model resets to adapt to new user distributions, leading to service interruptions. To overcome these limitations, we propose a computationally efficient Spatiotemporal Continual Learning (STCL) framework realized through a Group-Decoupled Multi-Agent Proximal Policy Optimization (G-MAPPO) algorithm. Our approach integrates a novel Group-Decoupled Policy Optimization (GDPO) mechanism that utilizes dynamic $z$-score normalization to autonomously balance heterogeneous objectives, including energy efficiency, user fairness, and coverage. This mechanism effectively mitigates gradient conflicts induced by concept drifts without requiring offline retraining. Furthermore, the framework leverages the 3D mobility of UAVs as a spatial compensation layer, enabling the swarm to autonomously adjust altitudes to accommodate extreme density fluctuations. Extensive simulations demonstrate that the proposed STCL framework achieves superior resilience, characterized by an elastic recovery of service reliability to approximately 0.95 during phase transitions. Compared to the MADDPG baseline, G-MAPPO not only prevents knowledge forgetting but also delivers an effective capacity gain of 20\% under extreme traffic loads, validating its potential as a scalable solution for edge-enabled aerial swarms.

Figures (10)

• Figure 1: Illustration of the 3D aerial-ground integrated network deployed across a heterogeneous environment. The service area features a spatial transition from a dense Urban Area (left), through a Suburban Area (center), to a sparse Rural Area (right). A central Macro Ground Base Station (GBS) provides ubiquitous omnidirectional coverage (hemispherical dome), while multiple UAVs function as mobile small cells. The dashed arrows illustrate the spatiotemporal migration of the UAV swarm, emphasizing its continuous adaptation to shifting user densities across the sequential task chain.
• Figure 2: Schematic overview of the proposed Spatiotemporal Continual Learning (STCL) framework. The architecture leverages a Group-Decoupled Multi-Agent Proximal Policy Optimization (G-MAPPO) approach within a Centralized Training with Decentralized Execution (CTDE) paradigm. A key innovation is the Group-Decoupled Policy Optimization (GDPO) mechanism (highlighted in green), which dynamically normalizes heterogeneous reward signals from the non-stationary 3D environment. This ensures stable scalar feedback for the centralized critic, mitigating catastrophic forgetting across varying spatiotemporal tasks.
• Figure 3: G-MAPPO: Spatiotemporal Continual Learning with GDPO
• Figure 4: Convergence analysis of the Total System Reward (Weighted Sum) across the spatiotemporal task chain (Urban $\rightarrow$ Suburban $\rightarrow$ Rural). The Static K-Means (green dashed line) serves as an idealized upper bound utilizing perfect global information. The proposed G-MAPPO (blue line) demonstrates superior learning stability and resilience, maintaining a higher reward plateau compared to the MADDPG baseline (orange line), which suffers from significant policy degradation and high variance (wide shaded regions) after the first environmental transition at Step 700.
• Figure 5: Evolution of the Spatial Service Reliability ($P_{\text{cov}}$) across spatiotemporal phases. The Static K-Means (green dashed line) represents the theoretical upper bound. In the dense Urban phase, both RL algorithms degrade due to physical capacity constraints ($M=140$); however, G-MAPPO (blue solid line) exhibits active exploration compared to the rapid policy stagnation of the MADDPG baseline (orange solid line). Notably, at the transition to the Suburban phase (Step 700), G-MAPPO demonstrates a superior "elastic rebound," recovering to near-optimal reliability, thereby verifying its resilience against catastrophic forgetting.