Monte Carlo Tree Search Satellite Scheduling Under Cloud Cover Uncertainty

TL;DR

Addresses the multi-satellite collection scheduling problem under cloud cover uncertainty by applying Monte Carlo Tree Search (MCTS) to generate full multi-orbit schedules on a graph representation. It investigates two MCTS variants—average-value and best-value—alongside hyperparameter tuning, and formalizes the problem with sets $O$ (orbits) and $T$ (tasks) under a stochastic objective; the objective can be expressed as $ \max \sum_{i \in T} \varpi_i \bigl(1 - \prod_{k \in O} (1 - p_i^k \sum_{j \in T, j \neq i} x_{ij}^k)\bigr)$, with resource and sequencing constraints. The authors compare MCTS against dynamic programming, deep Q-learning, and genetic algorithms on Wang and Genetic test sets, demonstrating competitive solution quality and efficiency, with the average-value variant excelling on larger, denser problems. The work highlights MCTS as a viable scheduling tool for dynamic satellite operations, offering a tunable balance between exploration, computation, and solution quality, while acknowledging stochasticity as both a strengths and a limitation for real-world deployment.

Abstract

Efficient utilization of satellite resources in dynamic environments remains a challenging problem in satellite scheduling. This paper addresses the multi-satellite collection scheduling problem (m-SatCSP), aiming to optimize task scheduling over a constellation of satellites under uncertain conditions such as cloud cover. Leveraging Monte Carlo Tree Search (MCTS), a stochastic search algorithm, two versions of MCTS are explored to schedule satellites effectively. Hyperparameter tuning is conducted to optimize the algorithm's performance. Experimental results demonstrate the effectiveness of the MCTS approach, outperforming existing methods in both solution quality and efficiency. Comparative analysis against other scheduling algorithms showcases competitive performance, positioning MCTS as a promising solution for satellite task scheduling in dynamic environments.

Figures (4)

• Figure 1: An example problem graph for Wang Wang 9 Orbit, 10 Task, Problem 4. In this example, all nodes are connected, but this is not always the case. Not all paths through the graph are possible, for example, in orbit 5 the path 5-4-0 is not possible due to energy constraints. The best path for this problem is highlighted in blue.
• Figure 2: A visual representation of the phases of Monte Carlo Tree Search
• Figure 3: A graphical representation of the performance of each algorithm on all test sets from Wang.
• Figure 4: A graphical representation of the performance of each algorithm on all test sets from Genetic.