Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Supervised Training of Spaceborne Vision during Proximity Operations using Adaptive Kalman Filtering

Tae Ha Park, Simone D'Amico

TL;DR

This work addresses the sim2real gap in spaceborne monocular pose estimation by introducing Online Supervised Training (OST), which refines a lightweight NN online during Rendezvous and Proximity Operations using pseudo-labels produced by an Adaptive Unscented Kalman Filter (AUKF). The NN is integrated as a measurement module in the filter and updated via a single backpropagation on each flight image, with pseudo-labels derived from AUKF state-estimates and projected keypoints. Offline robust training uses aggressive data augmentations to bias the model toward geometry, while HIL and flight experiments show OST improves both AUKF steady-state errors and NN pose predictions across diverse viewing directions, demonstrating effective sim2real gap closure under onboard constraints. The approach relies on diverse viewing geometry and assumes a converged AUKF state, with future work targeting longer trajectories and more sophisticated OST scheduling and uncertainty modeling.

Abstract

This work presents an Online Supervised Training (OST) method to enable robust vision-based navigation about a non-cooperative spacecraft. Spaceborne Neural Networks (NN) are susceptible to domain gap as they are primarily trained with synthetic images due to the inaccessibility of space. OST aims to close this gap by training a pose estimation NN online using incoming flight images during Rendezvous and Proximity Operations (RPO). The pseudo-labels are provided by adaptive unscented Kalman filter where the NN is used in the loop as a measurement module. Specifically, the filter tracks the target's relative orbital and attitude motion, and its accuracy is ensured by robust on-ground training of the NN using only synthetic data. The experiments on real hardware-in-the-loop trajectory images show that OST can improve the NN performance on the target image domain given that OST is performed on images of the target viewed from a diverse set of directions during RPO.

Online Supervised Training of Spaceborne Vision during Proximity Operations using Adaptive Kalman Filtering

TL;DR

This work addresses the sim2real gap in spaceborne monocular pose estimation by introducing Online Supervised Training (OST), which refines a lightweight NN online during Rendezvous and Proximity Operations using pseudo-labels produced by an Adaptive Unscented Kalman Filter (AUKF). The NN is integrated as a measurement module in the filter and updated via a single backpropagation on each flight image, with pseudo-labels derived from AUKF state-estimates and projected keypoints. Offline robust training uses aggressive data augmentations to bias the model toward geometry, while HIL and flight experiments show OST improves both AUKF steady-state errors and NN pose predictions across diverse viewing directions, demonstrating effective sim2real gap closure under onboard constraints. The approach relies on diverse viewing geometry and assumes a converged AUKF state, with future work targeting longer trajectories and more sophisticated OST scheduling and uncertainty modeling.

Abstract

This work presents an Online Supervised Training (OST) method to enable robust vision-based navigation about a non-cooperative spacecraft. Spaceborne Neural Networks (NN) are susceptible to domain gap as they are primarily trained with synthetic images due to the inaccessibility of space. OST aims to close this gap by training a pose estimation NN online using incoming flight images during Rendezvous and Proximity Operations (RPO). The pseudo-labels are provided by adaptive unscented Kalman filter where the NN is used in the loop as a measurement module. Specifically, the filter tracks the target's relative orbital and attitude motion, and its accuracy is ensured by robust on-ground training of the NN using only synthetic data. The experiments on real hardware-in-the-loop trajectory images show that OST can improve the NN performance on the target image domain given that OST is performed on images of the target viewed from a diverse set of directions during RPO.
Paper Structure (6 sections, 4 equations, 8 figures, 1 table)

This paper contains 6 sections, 4 equations, 8 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Experiments
  5. Results
  6. Conclusion

Figures (8)

  • Figure 1: Various image domains of the Tango spacecraft from the SPEED+ dataset park_2022_speedplus (a, b, c) and actual space flight (d).
  • Figure 2: Illustration of OST using an Adaptive Unscented Kalman Filter (AUKF) during space RPO.
  • Figure 3: Visualization of (a) far-range AON kruger_2021_acta_multitrack and (b) close-range pose estimation on a cropped image. The keypoint locations and uncertainties are extracted from heatmaps pasqualettocassinis_2021_acta_coupled. Note that marked distances are specific to the presented imagery and may vary for camera FoV and target size.
  • Figure 4: Image augmentations. From left to right: SPEED+ $\texttt{synthetic}$ images; StyleAugment jackson_2019_cvpr_styleaug; DeepAugment hendrycks_2021_iccv_imagenet_r; RandConv xu_2021_iclr_randconv.
  • Figure 5: Visualization of $\texttt{lightbox}$ images from the SHIRT ROE1 (top) and ROE2 (bottom) trajectories with the target wireframe model projected with the ground-truth pose labels. Time ($t$) unit is in orbits.
  • ...and 3 more figures