Reinforcement Learning for Opportunistic Routing in Software-Defined LEO-Terrestrial Systems

TL;DR

This work addresses low-latency data delivery in dynamic LEO-terrestrial networks by introducing opportunistic routing managed via a GEO-resident SDN controller. It combines a constrained stochastic optimization with a residual reinforcement-learning framework that augments a queue-stabilizing backpressure baseline to adapt to time-varying topology and gateway availability, optimizing the objective $\mathbb{E}\left[\max_k Q_k (t{+}1)\right]$. The authors propose a link-to-ground aware residual policy (LG-BP) built on backpressure, enhanced by a DDQN-based learner that exploits quasi-periodic orbital patterns, and demonstrate substantial queue-length reductions across multiple constellations and gateway scenarios. The approach advances integrated LEO-terrestrial networks by enabling low-latency routing with scalable central control and data-plane flexibility, informing design for future mega-constellations and resilient backhaul management.

Abstract

The proliferation of large-scale low Earth orbit (LEO) satellite constellations is driving the need for intelligent routing strategies that can effectively deliver data to terrestrial networks under rapidly time-varying topologies and intermittent gateway visibility. Leveraging the global control capabilities of a geostationary (GEO)-resident software-defined networking (SDN) controller, we introduce opportunistic routing, which aims to minimize delivery delay by forwarding packets to any currently available ground gateways rather than fixed destinations. This makes it a promising approach for achieving low-latency and robust data delivery in highly dynamic LEO networks. Specifically, we formulate a constrained stochastic optimization problem and employ a residual reinforcement learning framework to optimize opportunistic routing for reducing transmission delay. Simulation results over multiple days of orbital data demonstrate that our method achieves significant improvements in queue length reduction compared to classical backpressure and other well-known queueing algorithms.

Figures (6)

• Figure 1: Aggregated activity rate over the globe with the ground-track of three Starlink satellites (Orbit A, Orbit B, Orbit C), together with gateway locations and their access periods. Aggregated activity denotes the total user traffic demand (MB per time step) summed over the satellite footprint.
• Figure 2: Residual reinforcement learning for opportunistic routing in the LEO-terrestrial system.
• Figure 3: Training reward versus episode. The proposed residual-learning policy (blue) consistently outperforms the vanilla DDQN baseline (green).
• Figure 4: Queue length across varying maximum number of neighbors per satellite (left) and varying constellations (right) for different policies.
• Figure 5: Performance evaluation: (a) Varying number of satellites, and (b) empirical CDF when $K = 10$ satellites.