Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Close-Proximity Satellite Operations through Deep Reinforcement Learning and Terrestrial Testing Environments

Henry Lei, Joshua Aurand, Zachary S. Lippay, Sean Phillips

TL;DR

The paper addresses safe autonomous satellite operations in crowded space by deploying a DRL-based low-level controller within a hierarchical framework and evaluating safety via a runtime assurance (RTA) module. It uses the LINCS Lab to compare training, simulation, and hardware-in-the-loop performance, highlighting sim-to-real gaps and the impact of RTA on mission efficiency. Across three experimental sets, results show strong baseline performance in simulation but notable degradation in hardware environments, especially under high-risk, close-proximity scenarios. The work emphasizes the importance of robust training, realistic dynamics, and safety-first control architectures for practical autonomous satellite operations, guiding future improvements in sim2real robustness and safety integration.

Abstract

With the increasingly congested and contested space environment, safe and effective satellite operation has become increasingly challenging. As a result, there is growing interest in autonomous satellite capabilities, with common machine learning techniques gaining attention for their potential to address complex decision-making in the space domain. However, the "black-box" nature of many of these methods results in difficulty understanding the model's input/output relationship and more specifically its sensitivity to environmental disturbances, sensor noise, and control intervention. This paper explores the use of Deep Reinforcement Learning (DRL) for satellite control in multi-agent inspection tasks. The Local Intelligent Network of Collaborative Satellites (LINCS) Lab is used to test the performance of these control algorithms across different environments, from simulations to real-world quadrotor UAV hardware, with a particular focus on understanding their behavior and potential degradation in performance when deployed beyond the training environment.

Close-Proximity Satellite Operations through Deep Reinforcement Learning and Terrestrial Testing Environments

TL;DR

The paper addresses safe autonomous satellite operations in crowded space by deploying a DRL-based low-level controller within a hierarchical framework and evaluating safety via a runtime assurance (RTA) module. It uses the LINCS Lab to compare training, simulation, and hardware-in-the-loop performance, highlighting sim-to-real gaps and the impact of RTA on mission efficiency. Across three experimental sets, results show strong baseline performance in simulation but notable degradation in hardware environments, especially under high-risk, close-proximity scenarios. The work emphasizes the importance of robust training, realistic dynamics, and safety-first control architectures for practical autonomous satellite operations, guiding future improvements in sim2real robustness and safety integration.

Abstract

With the increasingly congested and contested space environment, safe and effective satellite operation has become increasingly challenging. As a result, there is growing interest in autonomous satellite capabilities, with common machine learning techniques gaining attention for their potential to address complex decision-making in the space domain. However, the "black-box" nature of many of these methods results in difficulty understanding the model's input/output relationship and more specifically its sensitivity to environmental disturbances, sensor noise, and control intervention. This paper explores the use of Deep Reinforcement Learning (DRL) for satellite control in multi-agent inspection tasks. The Local Intelligent Network of Collaborative Satellites (LINCS) Lab is used to test the performance of these control algorithms across different environments, from simulations to real-world quadrotor UAV hardware, with a particular focus on understanding their behavior and potential degradation in performance when deployed beyond the training environment.

Paper Structure

This paper contains 18 sections, 17 equations, 8 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Commonly Used Notation
  4. LINCS Lab
  5. Reference Problem
  6. Hierarchical Control
  7. Low-level Control
  8. Runtime Assurance
  9. Relaxed Quadratic Program
  10. Experimental Setup
  11. Methodology
  12. Factors for Evaluation
  13. Parameter Specification
  14. Results
  15. Experiment 1: Single-Agent Motion Control
  16. ...and 3 more sections

Figures (8)

  • Figure 1: Local Intelligent Network of Collaborative Satellites (LINCS) Laboratory overview
  • Figure 2: The left diagram shows simulation workflow with RTA enforcement in the LINCS Lab. The right diagram shows an analogous version for LINCS Cyber-Physical emulation using quadrotor UAVs.
  • Figure 3: The left diagram shows sample trajectories generated using the DRL hierarchical controller proposed in Lei22. The right diagram details task assignment by the HL planner. The solution presented is formulated to provide visually diverse viewing conditions to collaboratively inspect an unknown RSO, also called "Chief".
  • Figure 4: The left diagram shows a single agent satellite trajectory driven by the DRL LL controller from Lei22. The right diagram shows distance to goal over time; an episode is considered successful once the satellite reaches a fixed threshold around the goal waypoint.
  • Figure 5: Experiment 1 test trajectories. The left hand panel shows LINCS SIM results, the right hand panel show LINCS CPS results.
  • ...and 3 more figures