Earth Observation Satellite Scheduling with Graph Neural Networks and Monte Carlo Tree Search

TL;DR

The paper tackles Earth Observation Satellite Planning (EOSP), an NP-hard scheduling problem with time-dependent maneuver delays and oversubscription. It introduces a Graph Neural Network (GNN) driven reinforcement learning framework, operating on a continuous-time graph representation, augmented by a discrete-time simulator and a post-training Monte Carlo Tree Search (MCTS) to harvest further gains. Key contributions include a continuous-time graph representation for EOSP, a graph rewiring strategy for effective information flow, and a PPO-based learning pipeline with action masking integrated with a GNN encoder, plus inference-time MCTS that improves solution quality. Experiments on real-world-scale instances show competitive performance against Greedy and RAMP baselines and indicate strong generalization to larger problems, with MCTS providing additional improvements under constrained budgets.

Abstract

Earth Observation Satellite Planning (EOSP) is a difficult optimization problem with considerable practical interest. A set of requested observations must be scheduled on an agile Earth observation satellite while respecting constraints on their visibility window, as well as maneuver constraints that impose varying delays between successive observations. In addition, the problem is largely oversubscribed: there are much more candidate observations than can possibly be achieved. Therefore, one must select the set of observations that will be performed while maximizing their cumulative benefit and propose a feasible schedule for these observations. As previous work mostly focused on heuristic and iterative search algorithms, this paper presents a new technique for selecting and scheduling observations based on Graph Neural Networks (GNNs) and Deep Reinforcement Learning (DRL). GNNs are used to extract relevant information from the graphs representing instances of the EOSP, and DRL drives the search for optimal schedules. A post-learning search step based on Monte Carlo Tree Search (MCTS) is added that is able to find even better solutions. Experiments show that it is able to learn on small problem instances and generalize to larger real-world instances, with very competitive performance compared to traditional approaches.

Figures (11)

• Figure 1: Discrete graph for 4 candidate acquisitions. Corresponding continuous graph is a 4-node clique.
• Figure 2: Wheatley general architecture
• Figure 3: Unitary scores: single problem of 106 acquisitions.
• Figure 4: Unitary scores: performance on 31 unseen problems (training on 128 different problems)
• Figure 5: Unitary scores: Wheatley vs. Greedy