Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Bridging the Space Domain Gap for Satellite Pose Estimation using Event Sensing

Mohsi Jawaid, Ethan Elms, Yasir Latif, Tat-Jun Chin

TL;DR

It is demonstrated that event sensing offers a promising solution to generalise from the simulation to the target domain under stark illumination differences, and the main contribution is an event-based satellite pose estimation technique, trained purely on synthetic event data with basic data augmentation to improve robustness against practical (noisy) event sensors.

Abstract

Deep models trained using synthetic data require domain adaptation to bridge the gap between the simulation and target environments. State-of-the-art domain adaptation methods often demand sufficient amounts of (unlabelled) data from the target domain. However, this need is difficult to fulfil when the target domain is an extreme environment, such as space. In this paper, our target problem is close proximity satellite pose estimation, where it is costly to obtain images of satellites from actual rendezvous missions. We demonstrate that event sensing offers a promising solution to generalise from the simulation to the target domain under stark illumination differences. Our main contribution is an event-based satellite pose estimation technique, trained purely on synthetic event data with basic data augmentation to improve robustness against practical (noisy) event sensors. Underpinning our method is a novel dataset with carefully calibrated ground truth, comprising of real event data obtained by emulating satellite rendezvous scenarios in the lab under drastic lighting conditions. Results on the dataset showed that our event-based satellite pose estimation method, trained only on synthetic data without adaptation, could generalise to the target domain effectively.

Towards Bridging the Space Domain Gap for Satellite Pose Estimation using Event Sensing

TL;DR

It is demonstrated that event sensing offers a promising solution to generalise from the simulation to the target domain under stark illumination differences, and the main contribution is an event-based satellite pose estimation technique, trained purely on synthetic event data with basic data augmentation to improve robustness against practical (noisy) event sensors.

Abstract

Deep models trained using synthetic data require domain adaptation to bridge the gap between the simulation and target environments. State-of-the-art domain adaptation methods often demand sufficient amounts of (unlabelled) data from the target domain. However, this need is difficult to fulfil when the target domain is an extreme environment, such as space. In this paper, our target problem is close proximity satellite pose estimation, where it is costly to obtain images of satellites from actual rendezvous missions. We demonstrate that event sensing offers a promising solution to generalise from the simulation to the target domain under stark illumination differences. Our main contribution is an event-based satellite pose estimation technique, trained purely on synthetic event data with basic data augmentation to improve robustness against practical (noisy) event sensors. Underpinning our method is a novel dataset with carefully calibrated ground truth, comprising of real event data obtained by emulating satellite rendezvous scenarios in the lab under drastic lighting conditions. Results on the dataset showed that our event-based satellite pose estimation method, trained only on synthetic data without adaptation, could generalise to the target domain effectively.
Paper Structure (36 sections, 4 equations, 9 figures, 2 tables)

This paper contains 36 sections, 4 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. Satellite pose estimation
  4. Visual domain adaptation
  5. Event-based vision
  6. Event datasets
  7. Satellite pose with event sensing
  8. From event streams to event frames
  9. Pose estimation from event frames
  10. Proposed method
  11. Landmark selection
  12. Object detection
  13. Landmark regression
  14. Pose estimation
  15. Training data
  16. ...and 21 more sections

Figures (9)

  • Figure 1: (a) Rendered frame of a satellite under normal lighting, with ground truth bounding box and set of $24$ landmarks. (b) Real image of a 3D model of the satellite under extreme lighting; observe the lens flare and uneven contrast. (c) Event frame corresponding to (a), generated using V2E hu2021v2e. (d) Real event frame corresponding to (b). Our premise is that the domain gap between (c) and (d) is lower than that between (a) and (b).
  • Figure 2: Proposed pipeline for satellite pose estimation from event frames.
  • Figure 3: Our augmentations for event frames. (Left) Event frame generated using V2E. (Middle) With RandomEventNoise augmentation. (Right) With RandomEventNoise and RandomEventLines augmentations.
  • Figure 4: Blender viewport with the HST 3D model and three point light sources (orange) for illumination. (Left) Top view and (Right) side view.
  • Figure 5: (Left) Our setup for capturing real event data. Shown here is the ringbelow lighting configuration. (Right) Distribution of ground truth event camera poses for the approach (red) and orbit (green) trajectories with approximate light source positions and directions (not to scale).
  • ...and 4 more figures