ChiNet: Deep Recurrent Convolutional Learning for Multimodal Spacecraft Pose Estimation

Abstract

This paper presents an innovative deep learning pipeline which estimates the relative pose of a spacecraft by incorporating the temporal information from a rendezvous sequence. It leverages the performance of long short-term memory (LSTM) units in modelling sequences of data for the processing of features extracted by a convolutional neural network (CNN) backbone. Three distinct training strategies, which follow a coarse-to-fine funnelled approach, are combined to facilitate feature learning and improve end-to-end pose estimation by regression. The capability of CNNs to autonomously ascertain feature representations from images is exploited to fuse thermal infrared data with red-green-blue (RGB) inputs, thus mitigating the effects of artefacts from imaging space objects in the visible wavelength. Each contribution of the proposed framework, dubbed ChiNet, is demonstrated on a synthetic dataset, and the complete pipeline is validated on experimental data.

Figures (12)

• Figure 1: Qualitative results of the proposed method on two simulated ncrv sequences with Envisat from the Astos dataset. ChiNet provides continuous and robust pose estimation throughout the whole trajectories, explicitly taking into account information from previous frames.
• Figure 2: ChiNet system overview. The proposed drcnn architecture performs end-to-end spacecraft pose estimation from a sequence of multimodal rgbt image inputs of arbitrary size.
• Figure 3: Block diagram of a *lstm recurrent memory unit. $\mathrm{sigm}$ and $\mathrm{tanh}$ denote the sigmoid and hyperbolic tangent activation functions, respectively; $\odot$ and $+$ denote element-wise product and addition, respectively.
• Figure 4: Characteristics of the synthetic Astos dataset.
• Figure 5: Comparison of estimated position and attitude errors over time in terms of training stages used for ASTOS/06. All models are trained on a cnn taking rgb inputs. (S2-100) Stage 2.0 trained for 100.0 epochs. (S2-200) Stage 2.0 trained for 200.0 epochs. (S1,S2) Stage 1.0 and Stage 2.0. (S1,S2,S3) Stage 1.0, Stage 2.0, and Stage 3.0.