Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Green Emergency Communications in RIS- and MA-Assisted Multi-UAV SAGINs: A Partially Observable Reinforcement Learning Approach

Liangshun Wu, Wen Chen, Shunqing Zhang, Yajun Wang, Kunlun Wang

TL;DR

Experimental results show the spatiotemporal A2C policy outperforms IA2C, ConseNet, FPrint, DIAL, and CommNet--achieving faster convergence, higher asymptotic reward, reduced Temporal-Difference(TD)/advantage errors, and a better communication throughput-energy trade-off.

Abstract

In post-disaster space-air-ground integrated networks (SAGINs), terrestrial infrastructure is often impaired, and unmanned aerial vehicles (UAVs) must rapidly restore connectivity for mission-critical ground terminals in cluttered non-line-of-sight (NLoS) urban environments. To enhance coverage, UAVs employ movable antennas (MAs), while reconfigurable intelligent surfaces (RISs) on surviving high-rises redirect signals. The key challenge is communication-limited partial observability, leaving each UAV with a narrow, fast-changing neighborhood view that destabilizes value estimation. Existing multi-agent reinforcement learning (MARL) approaches are inadequate--non-communication methods rely on unavailable global critics, heuristic sharing is brittle and redundant, and learnable protocols (e.g., CommNet, DIAL) lose per-neighbor structure and aggravate non-stationarity under tight bandwidth. To address partial observability, we propose a spatiotemporal A2C where each UAV transmits prior-decision messages with local state, a compact policy fingerprint, and a recurrent belief, encoded per neighbor and concatenated. A spatial discount shapes value targets to emphasize local interactions, while analysis under one-hop-per-slot latency explains stable training with delayed views. Experimental results show our policy outperforms IA2C, ConseNet, FPrint, DIAL, and CommNet--achieving faster convergence, higher asymptotic reward, reduced Temporal-Difference(TD)/advantage errors, and a better communication throughput-energy trade-off.

Green Emergency Communications in RIS- and MA-Assisted Multi-UAV SAGINs: A Partially Observable Reinforcement Learning Approach

TL;DR

Experimental results show the spatiotemporal A2C policy outperforms IA2C, ConseNet, FPrint, DIAL, and CommNet--achieving faster convergence, higher asymptotic reward, reduced Temporal-Difference(TD)/advantage errors, and a better communication throughput-energy trade-off.

Abstract

In post-disaster space-air-ground integrated networks (SAGINs), terrestrial infrastructure is often impaired, and unmanned aerial vehicles (UAVs) must rapidly restore connectivity for mission-critical ground terminals in cluttered non-line-of-sight (NLoS) urban environments. To enhance coverage, UAVs employ movable antennas (MAs), while reconfigurable intelligent surfaces (RISs) on surviving high-rises redirect signals. The key challenge is communication-limited partial observability, leaving each UAV with a narrow, fast-changing neighborhood view that destabilizes value estimation. Existing multi-agent reinforcement learning (MARL) approaches are inadequate--non-communication methods rely on unavailable global critics, heuristic sharing is brittle and redundant, and learnable protocols (e.g., CommNet, DIAL) lose per-neighbor structure and aggravate non-stationarity under tight bandwidth. To address partial observability, we propose a spatiotemporal A2C where each UAV transmits prior-decision messages with local state, a compact policy fingerprint, and a recurrent belief, encoded per neighbor and concatenated. A spatial discount shapes value targets to emphasize local interactions, while analysis under one-hop-per-slot latency explains stable training with delayed views. Experimental results show our policy outperforms IA2C, ConseNet, FPrint, DIAL, and CommNet--achieving faster convergence, higher asymptotic reward, reduced Temporal-Difference(TD)/advantage errors, and a better communication throughput-energy trade-off.

Paper Structure

This paper contains 27 sections, 3 theorems, 53 equations, 10 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Partial Observability
  3. MARL Addressing Partial Observability
  4. Contributions
  5. System Model and Problem Formulation
  6. System Model
  7. UAV Trajectory
  8. UAV Propulsion Energy Consumption
  9. RIS Phase Configuration
  10. MA on Each UAV
  11. UAV-RIS Link Channel Model
  12. RIS-GT Link Channel Model
  13. UAV–RIS–GT Cascaded Channel Model
  14. Problem Formulation
  15. Solved by MARL
  16. ...and 12 more sections

Key Result

Proposition 1

Each augmented observation $\tilde{\mathcal{S}}_j[n]$ contains UAV $j$’s local state and aggregated multi-hop information from its own neighbors, recursively propagated up to $h$ hops, while the spatial discount $\alpha^{d_{ji}}$ reduces the influence of distant UAVs, ensuring the learning target is

Figures (10)

  • Figure 1: SAGIN emergency communication scenario.
  • Figure 2: System model.
  • Figure 3: MA positioning with a limited set of discrete locations.
  • Figure 4: UAV–RIS LoS geometry.
  • Figure 5: Framework of our method.
  • ...and 5 more figures

Theorems & Definitions (8)

  • Definition 1: Networked MDP
  • Definition 2: Spatiotemporal Discounted Reward
  • Proposition 1
  • Proposition 2: Spatial information propagation
  • Proposition 3: Spatial gradient propagation
  • proof
  • proof
  • proof