Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Demonstrating Reinforcement Learning and Run Time Assurance for Spacecraft Inspection Using Unmanned Aerial Vehicles

Kyle Dunlap, Nathaniel Hamilton, Zachary Lippay, Matthew Shubert, Sean Phillips, Kerianne L. Hobbs

TL;DR

This work tackles autonomous, safe spacecraft inspection by combining PPO-trained Neural Network Controllers (NNCs) with Run Time Assurance (RTA) implemented via multiple Control Barrier Functions (CBFs) in an Active Set Invariance Filter (ASIF). The approach is validated on the LINCS lab, where quadrotors emulate Hill's-frame spacecraft dynamics (Clohessy-Wiltshire with perturbations) and operate under safety constraints derived from CBFs. Open-loop and closed-loop experiments demonstrate that the NNCs can accomplish inspection tasks while the ASIF-RTA enforces safety, with sim2real experiments highlighting robustness to disturbances but also indicating needs for stronger CBF design under noisy measurements. Overall, the results support the practical viability of deploying RL-based autonomous spacecraft inspection with real-time safety guarantees in a laboratory setting, paving the way for safer on-orbit servicing and manufacturing missions.

Abstract

On-orbit spacecraft inspection is an important capability for enabling servicing and manufacturing missions and extending the life of spacecraft. However, as space operations become increasingly more common and complex, autonomous control methods are needed to reduce the burden on operators to individually monitor each mission. In order for autonomous control methods to be used in space, they must exhibit safe behavior that demonstrates robustness to real world disturbances and uncertainty. In this paper, neural network controllers (NNCs) trained with reinforcement learning are used to solve an inspection task, which is a foundational capability for servicing missions. Run time assurance (RTA) is used to assure safety of the NNC in real time, enforcing several different constraints on position and velocity. The NNC and RTA are tested in the real world using unmanned aerial vehicles designed to emulate spacecraft dynamics. The results show this emulation is a useful demonstration of the capability of the NNC and RTA, and the algorithms demonstrate robustness to real world disturbances.

Demonstrating Reinforcement Learning and Run Time Assurance for Spacecraft Inspection Using Unmanned Aerial Vehicles

TL;DR

This work tackles autonomous, safe spacecraft inspection by combining PPO-trained Neural Network Controllers (NNCs) with Run Time Assurance (RTA) implemented via multiple Control Barrier Functions (CBFs) in an Active Set Invariance Filter (ASIF). The approach is validated on the LINCS lab, where quadrotors emulate Hill's-frame spacecraft dynamics (Clohessy-Wiltshire with perturbations) and operate under safety constraints derived from CBFs. Open-loop and closed-loop experiments demonstrate that the NNCs can accomplish inspection tasks while the ASIF-RTA enforces safety, with sim2real experiments highlighting robustness to disturbances but also indicating needs for stronger CBF design under noisy measurements. Overall, the results support the practical viability of deploying RL-based autonomous spacecraft inspection with real-time safety guarantees in a laboratory setting, paving the way for safer on-orbit servicing and manufacturing missions.

Abstract

On-orbit spacecraft inspection is an important capability for enabling servicing and manufacturing missions and extending the life of spacecraft. However, as space operations become increasingly more common and complex, autonomous control methods are needed to reduce the burden on operators to individually monitor each mission. In order for autonomous control methods to be used in space, they must exhibit safe behavior that demonstrates robustness to real world disturbances and uncertainty. In this paper, neural network controllers (NNCs) trained with reinforcement learning are used to solve an inspection task, which is a foundational capability for servicing missions. Run time assurance (RTA) is used to assure safety of the NNC in real time, enforcing several different constraints on position and velocity. The NNC and RTA are tested in the real world using unmanned aerial vehicles designed to emulate spacecraft dynamics. The results show this emulation is a useful demonstration of the capability of the NNC and RTA, and the algorithms demonstrate robustness to real world disturbances.
Paper Structure (16 sections, 14 equations, 14 figures, 3 tables)

This paper contains 16 sections, 14 equations, 14 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Reinforcement Learning
  4. Run Time Assurance
  5. LINCS
  6. Experimental Setup
  7. Inspection Task
  8. Dynamics
  9. RL Environment Setup
  10. Safety Constraints
  11. Training the Neural Network Controllers
  12. Lab Experiments
  13. Results
  14. Open Loop
  15. Closed Loop
  16. ...and 1 more sections

Figures (14)

  • Figure 1: Basic formulation of Deep Reinforcement Learning for a control problem.
  • Figure 2: Feedback control system with RTA filter. Components with low safety confidence are outlined in red, and components with high safety confidence are outlined in blue.
  • Figure 3: Local Intelligent Network of Collaborative Satellites (LINCS) Laboratory overview
  • Figure 4: Deputy spacecraft in relation to a chief spacecraft in Hill's Frame.
  • Figure 5: The NNC architecture for the "all sensors" variation of the inspection task.
  • ...and 9 more figures