Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transformer-Based Reinforcement Learning for Autonomous Orbital Collision Avoidance in Partially Observable Environments

Thomas Georges, Adam Abdin

TL;DR

This work tackles autonomous collision avoidance in low-Earth orbit under partial observability by formulating the problem as a $POMDP$ and integrating distance-dependent sensing, UKF-based state estimation, and a Mahalanobis-distance risk surrogate. A Transformer-XL policy is employed to aggregate intermittent observations over long horizons, enabling more fuel-efficient and robust maneuvers without compromising safety. Key contributions include a differentiable distance-based observability model via Lagrange interpolation, a risk-aware reward structure grounded in UKF covariance, and systematic evaluation across observability regimes showing consistent gains over memoryless baselines, particularly in intermediate degradation. The findings have practical implications for on-orbit autonomy, offering memory-augmented decision-making that preserves safety while reducing propellant consumption in imperfect sensing environments.

Abstract

We introduce a Transformer-based Reinforcement Learning framework for autonomous orbital collision avoidance that explicitly models the effects of partial observability and imperfect monitoring in space operations. The framework combines a configurable encounter simulator, a distance-dependent observation model, and a sequential state estimator to represent uncertainty in relative motion. A central contribution of this work is the use of transformer-based Partially Observable Markov Decision Process (POMDP) architecture, which leverage long-range temporal attention to interpret noisy and intermittent observations more effectively than traditional architectures. This integration provides a foundation for training collision avoidance agents that can operate more reliably under imperfect monitoring environments.

Transformer-Based Reinforcement Learning for Autonomous Orbital Collision Avoidance in Partially Observable Environments

TL;DR

This work tackles autonomous collision avoidance in low-Earth orbit under partial observability by formulating the problem as a and integrating distance-dependent sensing, UKF-based state estimation, and a Mahalanobis-distance risk surrogate. A Transformer-XL policy is employed to aggregate intermittent observations over long horizons, enabling more fuel-efficient and robust maneuvers without compromising safety. Key contributions include a differentiable distance-based observability model via Lagrange interpolation, a risk-aware reward structure grounded in UKF covariance, and systematic evaluation across observability regimes showing consistent gains over memoryless baselines, particularly in intermediate degradation. The findings have practical implications for on-orbit autonomy, offering memory-augmented decision-making that preserves safety while reducing propellant consumption in imperfect sensing environments.

Abstract

We introduce a Transformer-based Reinforcement Learning framework for autonomous orbital collision avoidance that explicitly models the effects of partial observability and imperfect monitoring in space operations. The framework combines a configurable encounter simulator, a distance-dependent observation model, and a sequential state estimator to represent uncertainty in relative motion. A central contribution of this work is the use of transformer-based Partially Observable Markov Decision Process (POMDP) architecture, which leverage long-range temporal attention to interpret noisy and intermittent observations more effectively than traditional architectures. This integration provides a foundation for training collision avoidance agents that can operate more reliably under imperfect monitoring environments.
Paper Structure (20 sections, 16 equations, 9 figures, 4 tables)

This paper contains 20 sections, 16 equations, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. The Collision Avoidance Decision Problem
  4. State Space and Dynamics
  5. Partial Observability
  6. POMDP Formulation
  7. Objectives and Constraints
  8. Methodology
  9. Orbital Dynamics and Trajectory Modelling
  10. Distance-Dependent Partial Observability via Lagrange Interpolation
  11. Policy Optimization
  12. State Estimation and Collision Risk
  13. Reward Design
  14. Transformer-Based Policy Architecture
  15. Results
  16. ...and 5 more sections

Figures (9)

  • Figure 1: Schematic of the inverse-encounter trajectory generation procedure.
  • Figure 2: MLP training under medium observability: comparison of collision-risk reward formulations.
  • Figure 3: MLP training dynamics across five partial-observability regimes.
  • Figure 4: MLP safety metrics across five partial-observability regimes.
  • Figure 5: Transformer-XL training dynamics across five partial-observability regimes.
  • ...and 4 more figures