Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing Mission Planning for Multi-Debris Rendezvous Using Reinforcement Learning with Refueling and Adaptive Collision Avoidance

Agni Bandyopadhyay, Gunther Waxenegger-Wilfing

TL;DR

This paper tackles autonomous multi-debris ADR in congested LEO by formulating it as an adaptive, multi-objective planning problem and solving it with a masked PPO reinforcement learning framework. The agent jointly learns debris sequencing, refueling strategy, and collision-avoidance maneuvers, using probabilistic collision zones and ellipsoidal detours to maintain safety. Results show the RL approach, especially when both sequencing and avoidance are learned (RL-RL), achieves higher debris coverage and safer operations than heuristic baselines, demonstrating robustness across randomized scenarios. The work suggests wide applicability to other autonomous space missions and highlights end-to-end learning as a viable path for safe, efficient autonomous space operations.

Abstract

As the orbital environment around Earth becomes increasingly crowded with debris, active debris removal (ADR) missions face significant challenges in ensuring safe operations while minimizing the risk of in-orbit collisions. This study presents a reinforcement learning (RL) based framework to enhance adaptive collision avoidance in ADR missions, specifically for multi-debris removal using small satellites. Small satellites are increasingly adopted due to their flexibility, cost effectiveness, and maneuverability, making them well suited for dynamic missions such as ADR. Building on existing work in multi-debris rendezvous, the framework integrates refueling strategies, efficient mission planning, and adaptive collision avoidance to optimize spacecraft rendezvous operations. The proposed approach employs a masked Proximal Policy Optimization (PPO) algorithm, enabling the RL agent to dynamically adjust maneuvers in response to real-time orbital conditions. Key considerations include fuel efficiency, avoidance of active collision zones, and optimization of dynamic orbital parameters. The RL agent learns to determine efficient sequences for rendezvousing with multiple debris targets, optimizing fuel usage and mission time while incorporating necessary refueling stops. Simulated ADR scenarios derived from the Iridium 33 debris dataset are used for evaluation, covering diverse orbital configurations and debris distributions to demonstrate robustness and adaptability. Results show that the proposed RL framework reduces collision risk while improving mission efficiency compared to traditional heuristic approaches. This work provides a scalable solution for planning complex multi-debris ADR missions and is applicable to other multi-target rendezvous problems in autonomous space mission planning.

Optimizing Mission Planning for Multi-Debris Rendezvous Using Reinforcement Learning with Refueling and Adaptive Collision Avoidance

TL;DR

This paper tackles autonomous multi-debris ADR in congested LEO by formulating it as an adaptive, multi-objective planning problem and solving it with a masked PPO reinforcement learning framework. The agent jointly learns debris sequencing, refueling strategy, and collision-avoidance maneuvers, using probabilistic collision zones and ellipsoidal detours to maintain safety. Results show the RL approach, especially when both sequencing and avoidance are learned (RL-RL), achieves higher debris coverage and safer operations than heuristic baselines, demonstrating robustness across randomized scenarios. The work suggests wide applicability to other autonomous space missions and highlights end-to-end learning as a viable path for safe, efficient autonomous space operations.

Abstract

As the orbital environment around Earth becomes increasingly crowded with debris, active debris removal (ADR) missions face significant challenges in ensuring safe operations while minimizing the risk of in-orbit collisions. This study presents a reinforcement learning (RL) based framework to enhance adaptive collision avoidance in ADR missions, specifically for multi-debris removal using small satellites. Small satellites are increasingly adopted due to their flexibility, cost effectiveness, and maneuverability, making them well suited for dynamic missions such as ADR. Building on existing work in multi-debris rendezvous, the framework integrates refueling strategies, efficient mission planning, and adaptive collision avoidance to optimize spacecraft rendezvous operations. The proposed approach employs a masked Proximal Policy Optimization (PPO) algorithm, enabling the RL agent to dynamically adjust maneuvers in response to real-time orbital conditions. Key considerations include fuel efficiency, avoidance of active collision zones, and optimization of dynamic orbital parameters. The RL agent learns to determine efficient sequences for rendezvousing with multiple debris targets, optimizing fuel usage and mission time while incorporating necessary refueling stops. Simulated ADR scenarios derived from the Iridium 33 debris dataset are used for evaluation, covering diverse orbital configurations and debris distributions to demonstrate robustness and adaptability. Results show that the proposed RL framework reduces collision risk while improving mission efficiency compared to traditional heuristic approaches. This work provides a scalable solution for planning complex multi-debris ADR missions and is applicable to other multi-target rendezvous problems in autonomous space mission planning.
Paper Structure (21 sections, 6 equations, 4 figures, 1 table)

This paper contains 21 sections, 6 equations, 4 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. BACKGROUND
  3. Problem Statement
  4. REINFORCEMENT LEARNING FRAMEWORK
  5. State and Action Representation
  6. Reward Function Design
  7. Masked PPO Algorithm
  8. METHODOLOGY AND EVALUATION STRATEGY
  9. Training Methodology
  10. Collision Avoidance Maneuvers
  11. Evaluation Modes
  12. Metrics and Logging
  13. Simulation Tools
  14. RESULTS AND DISCUSSION
  15. Training Performance Over Time
  16. ...and 6 more sections

Figures (4)

  • Figure 1: Illustration of a Hohmann transfer maneuver from an initial circular orbit to a higher target orbit. This is the core maneuver model used in our rendezvous planning.
  • Figure 2: Collision Avoidance Maneuvers: The agent performs CA_Above or CA_Below detours around a $5 \times 5 \times 5$ km collision zone to maintain safety margins.
  • Figure 7: Masked PPO Training Reward over Time ($\alpha = 0.99$). Reward trends upward and stabilizes, indicating learning convergence.
  • Figure 8: Debris Visited per Strategy (AVG) across 100 randomized test cases. RL-RL performs best across almost all scenarios.