Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bridging the Domain Gap for Flight-Ready Spaceborne Vision

Tae Ha Park, Simone D'Amico

TL;DR

SPNv3 introduces a ViT-based, heatmap-driven monocular pose estimator for known non-cooperative spacecraft that is optimized for spaceborne edge devices. Through extensive ablations on SPEED+ data and HIL-like conditions, the authors show that data augmentation, transfer learning, and higher input resolution can yield state-of-the-art robustness without unlabeled HIL data during offline training. The model balances accuracy, latency, and memory to meet flight-ready constraints, achieving sub-40 ms inference on Jetson Nano and strong performance at 0.5 Hz GN&C update rates. The work demonstrates a practical pathway to bridge sim2real gaps in spaceborne vision and informs design choices for future onboard perception systems.

Abstract

This work presents Spacecraft Pose Network v3 (SPNv3), a Neural Network (NN) for monocular pose estimation of a known, non-cooperative target spacecraft. SPNv3 is designed and trained to be computationally efficient while providing robustness to spaceborne images that have not been observed during offline training and validation on the ground. These characteristics are essential to deploying NNs on space-grade edge devices. They are achieved through careful NN design choices, and an extensive trade-off analysis reveals features such as data augmentation, transfer learning and vision transformer architecture as a few of those that contribute to simultaneously maximizing robustness and minimizing computational overhead. Experiments demonstrate that the final SPNv3 can achieve state-of-the-art pose accuracy on hardware-in-the-loop images from a robotic testbed while having trained exclusively on computer-generated synthetic images, effectively bridging the domain gap between synthetic and real imagery. At the same time, SPNv3 runs well above the update frequency of modern satellite navigation filters when tested on a representative graphical processing unit system with flight heritage. Overall, SPNv3 is an efficient, flight-ready NN model readily applicable to close-range rendezvous and proximity operations with target resident space objects.

Bridging the Domain Gap for Flight-Ready Spaceborne Vision

TL;DR

SPNv3 introduces a ViT-based, heatmap-driven monocular pose estimator for known non-cooperative spacecraft that is optimized for spaceborne edge devices. Through extensive ablations on SPEED+ data and HIL-like conditions, the authors show that data augmentation, transfer learning, and higher input resolution can yield state-of-the-art robustness without unlabeled HIL data during offline training. The model balances accuracy, latency, and memory to meet flight-ready constraints, achieving sub-40 ms inference on Jetson Nano and strong performance at 0.5 Hz GN&C update rates. The work demonstrates a practical pathway to bridge sim2real gaps in spaceborne vision and informs design choices for future onboard perception systems.

Abstract

This work presents Spacecraft Pose Network v3 (SPNv3), a Neural Network (NN) for monocular pose estimation of a known, non-cooperative target spacecraft. SPNv3 is designed and trained to be computationally efficient while providing robustness to spaceborne images that have not been observed during offline training and validation on the ground. These characteristics are essential to deploying NNs on space-grade edge devices. They are achieved through careful NN design choices, and an extensive trade-off analysis reveals features such as data augmentation, transfer learning and vision transformer architecture as a few of those that contribute to simultaneously maximizing robustness and minimizing computational overhead. Experiments demonstrate that the final SPNv3 can achieve state-of-the-art pose accuracy on hardware-in-the-loop images from a robotic testbed while having trained exclusively on computer-generated synthetic images, effectively bridging the domain gap between synthetic and real imagery. At the same time, SPNv3 runs well above the update frequency of modern satellite navigation filters when tested on a representative graphical processing unit system with flight heritage. Overall, SPNv3 is an efficient, flight-ready NN model readily applicable to close-range rendezvous and proximity operations with target resident space objects.
Paper Structure (38 sections, 5 equations, 8 figures, 8 tables)

This paper contains 38 sections, 5 equations, 8 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. ML Approaches to Spacecraft Pose Estimation
  4. Algorithms to Bridge Domain Gap
  5. Source-Free Domain Adaptation
  6. Spaceborne Applications
  7. Requirements of Spaceborne ML Models
  8. Computationally Efficiency
  9. Remark on GPU Flight Heritage
  10. Number of Parameters
  11. Batch-Agnostic Architecture
  12. Summary
  13. Pose Estimation Pipeline
  14. Searching for Next Spacecraft Pose Network (SPNv3)
  15. Pose Estimation Architectures
  16. ...and 23 more sections

Figures (8)

  • Figure 1: (left) TRON simulation room and its components. Figure from park_2021_aas_tron. (right) Example HIL images from the $\texttt{lightbox}$ (top) and $\texttt{sunlamp}$ domains (bottom) of the SPEED+ dataset park_2022_aero_speedplus.
  • Figure 2: Visualization of an image cropped around the far-away target spacecraft.
  • Figure 3: Visualization of far-range AON kruger_2021_acta_aon (left) and close-range pose estimation (right).
  • Figure 4: Visualization of the pose estimation pipeline adopted in this work.
  • Figure 5: Visualization of different pose estimation architectures: EfficientDet tan_2020_cvpr_efficientdet, HRNet wang_2021_tpami_hrnet and ViTPose xu_2022_nips_vitpose. The architectures commonly consist of a backbone/encoder network and a heatmap prediction head.
  • ...and 3 more figures