Stereo Event-based, 6-DOF Pose Tracking for Uncooperative Spacecraft

TL;DR

The paper tackles the challenge of tracking the pose of uncooperative spacecraft using a stereo event camera to overcome motion blur and illumination issues. It introduces a complete pipeline: (i) initialize a wireframe model from a first stereo event cluster via line triangulation and Plücker-line geometry, (ii) perform pose tracking by aligning observed events to projected wireframe lines through event-line matching, and (iii) refine the $6$-DOF pose with robust bundle adjustment that minimizes event-line distances. A new stereo event-based dataset, including both simulated and real sequences, supports quantitative evaluation, where the proposed method outperforms baselines such as SGM+ICP, Line-based, and ESVO on RPE and ATE metrics. The approach eliminates reliance on intermediate image reconstructions, offering a streamlined, illumination-robust solution with potential impact on on-orbit servicing and debris removal missions. The work also provides open-source code to foster further research in stereo event-based spacecraft pose estimation.

Abstract

Pose tracking of uncooperative spacecraft is an essential technology for space exploration and on-orbit servicing, which remains an open problem. Event cameras possess numerous advantages, such as high dynamic range, high temporal resolution, and low power consumption. These attributes hold the promise of overcoming challenges encountered by conventional cameras, including motion blur and extreme illumination, among others. To address the standard on-orbit observation missions, we propose a line-based pose tracking method for uncooperative spacecraft utilizing a stereo event camera. To begin with, we estimate the wireframe model of uncooperative spacecraft, leveraging the spatio-temporal consistency of stereo event streams for line-based reconstruction. Then, we develop an effective strategy to establish correspondences between events and projected lines of uncooperative spacecraft. Using these correspondences, we formulate the pose tracking as a continuous optimization process over 6-DOF motion parameters, achieved by minimizing event-line distances. Moreover, we construct a stereo event-based uncooperative spacecraft motion dataset, encompassing both simulated and real events. The proposed method is quantitatively evaluated through experiments conducted on our self-collected dataset, demonstrating an improvement in terms of effectiveness and accuracy over competing methods. The code will be open-sourced at https://github.com/Zibin6/SE6PT.

Figures (9)

• Figure 1: Block diagram of our method. The core of our method consists of model initialization and pose tracking. Model initialization is a one-time operation, followed by pose tracking. Our method takes stereo event streams and stereo camera parameters as input and provides the pose and trajectory of uncooperative spacecraft as output.
• Figure 2: The schematic diagram of stereo event clustering. The red dots represent individual events, while the blue brackets denote clusters of events. $\left\{ \mathbf{E}_{t-1}^{l}, \mathbf{E}_{t-1}^{r} \right\}, \left\{ \mathbf{E}_{t}^{l}, \mathbf{E}_{t}^{r} \right\}, \left\{ \mathbf{E}_{t+1}^{l}, \mathbf{E}_{t+1}^{r} \right\}$ represent event clusters for the left and right cameras at time $t-1$, $t$, and $t+1$, respectively.
• Figure 3: The geometric interpretation of model initialization in a stereo camera rig. Initially, perform stereo event clustering, followed by the extraction of lines from the event cluster. Subsequently, proceed with the sequential execution of (a) line triangulation, (b) line optimization, and (c) endpoint determination.
• Figure 4: Illustrative diagram of event-line matching. The matching of events and projection lines is based on distance constraints, where $d_1$ represents the distance from an event to the nearest line segment, $d_2$ represents the distance from an event to the midpoint of the nearest line segment, and $d_3$ represents the distance from an event to the second-nearest line segment.
• Figure 5: Experimental scenarios of our self-collected dataset. (a) Rendered experimental scenarios of Sequence 01. (b) Experimental scenarios of real events. The tested satellite models include Model I and Model II.