Learning Decentralized Routing Policies via Graph Attention-based Multi-Agent Reinforcement Learning in Lunar Delay-Tolerant Networks

TL;DR

This work addresses data routing for autonomous lunar rovers operating over Lunar Delay-Tolerant Networks with intermittent connectivity. It introduces a Graph Attention-based Multi-Agent Reinforcement Learning policy with Centralized Training, Decentralized Execution to learn routing decisions from local observations, without global topology updates or packet replication. The approach is formalized as a partially observable optimization problem and solved via CTDE with a packet-centric Q-learning framework, leveraging local subgraph features and attention to guide forwarding. Experiments in a packet-level simulator show superior delivery rates, no duplicates, and strong generalization to larger rover teams, highlighting the method's potential for scalable, beyond-line-of-sight space missions.

Abstract

We present a fully decentralized routing framework for multi-robot exploration missions operating under the constraints of a Lunar Delay-Tolerant Network (LDTN). In this setting, autonomous rovers must relay collected data to a lander under intermittent connectivity and unknown mobility patterns. We formulate the problem as a Partially Observable Markov Decision Problem (POMDP) and propose a Graph Attention-based Multi-Agent Reinforcement Learning (GAT-MARL) policy that performs Centralized Training, Decentralized Execution (CTDE). Our method relies only on local observations and does not require global topology updates or packet replication, unlike classical approaches such as shortest path and controlled flooding-based algorithms. Through Monte Carlo simulations in randomized exploration environments, GAT-MARL provides higher delivery rates, no duplications, and fewer packet losses, and is able to leverage short-term mobility forecasts; offering a scalable solution for future space robotic systems for planetary exploration, as demonstrated by successful generalization to larger rover teams.

Figures (3)

• Figure 1: Snapshot of a Lunar Delay Tolerant Network (LDTN) with three rovers $\mathfrak{r}\in\mathfrak{R}$ and a lander $\mathcal{L}$, which serves as the base station. Rovers $\mathfrak{r}_i$ and $\mathfrak{r}_k$ have direct connection with $\mathcal{L}$ and with one another, whereas $\mathfrak{r}_j$ can only communicate with $\mathfrak{r}_k$. Every rover explores a different sub-area of the map $\mathcal{M}$ by defining objectives $\mathbf{o}$. Each $\mathfrak{r}$ collects a local observation $\mathbf{X}_r = [\mathbf{x}_\mathfrak{n}]_{\mathfrak{n} \in \mathcal{N}_r}$ from its neighborhood (e.g., $\mathcal{N}_i = \{\mathfrak{r}_i, \mathfrak{r}_k, \mathcal{L}\}$ for $\mathfrak{r}_i$), which serves as input to a pre-trained policy $\pi$ that determines the routing decisions.
• Figure 2: The exploration region $\mathcal{M}$ is partitioned into sub-regions, each assigned to a different rover. Rovers depart from the lander and autonomously explore their assigned areas, selecting frontier points as navigation objectives while avoiding obstacles.
• Figure 3: Monte Carlo comparison of Spray and Wait, Greedy forwarding, and under different objective horizons $\mathbf{O}$. \ref{['fig:10_rovers']} shows generalization to exploration episodes $\tau$ with 10 rovers when the model was only trained on 3–5 rovers.