SPIN: Spacecraft Imagery for Navigation

TL;DR

SPIN introduces an open-source, Unity-based simulator capable of generating realistic two-spacecraft imagery with extensive ground-truth (depth, dense pose, segmentation, keypoints) and support for custom 3D models and varying camera/illumination settings. It enables in-scene data augmentation to preserve image structure while changing scene appearance, improving model generalization beyond traditional synthetic datasets. Validation on spacecraft pose estimation shows substantial improvements when trained on SPIN data (up to 47% average error reduction) and even larger gains (up to 60%) when applying SPIN-based augmentation. The work highlights SPIN's potential to close the gap between synthetic and real imagery, expand ground-truth richness, and support broader testing scenarios for autonomous relative navigation.

Abstract

The scarcity of data acquired under actual space operational conditions poses a significant challenge for developing learning-based visual navigation algorithms crucial for autonomous spacecraft navigation. This data shortage is primarily due to the prohibitive costs and inherent complexities of space operations. While existing datasets, predominantly relying on computer-simulated data, have partially addressed this gap, they present notable limitations. Firstly, these datasets often utilize proprietary image generation tools, restricting the evaluation of navigation methods in novel, unseen scenarios. Secondly, they provide limited ground-truth data, typically focusing solely on the spacecraft's translation and rotation relative to the camera. To address these limitations, we present SPIN (SPacecraft Imagery for Navigation), an open-source spacecraft image generation tool designed to support a wide range of visual navigation scenarios in space, with a particular focus on relative navigation tasks. SPIN provides multiple modalities of ground-truth data and allows researchers to employ custom 3D models of satellites, define specific camera-relative poses, and adjust settings such as camera parameters or environmental illumination conditions. We also propose a method for exploiting our tool as a data augmentation module. We validate our tool on the spacecraft pose estimation task by training with a SPIN-generated replica of SPEED+, reaching a 47% average error reduction on SPEED+ testbed data (that simulates realistic space conditions), further reducing it to a 60% error reduction when using SPIN as a data augmentation method. Both the SPIN tool (and source code) and our SPIN-generated version of SPEED+ will be publicly released upon paper acceptance on GitHub. https://github.com/vpulab/SPIN

Figures (4)

• Figure 1: A comparative between SPIN and other satellite datasets. On the left, the matrix compares their features (light blue cells indicate available features). Images show SPIN data examples: a rendering and a depth ground-truth (top images); our keypoint tool and a dense-pose ground-truth (bottom images).
• Figure 2: Representative images from the SHIRT park2023adaptive and SPEED+ park2022speed+ datasets. Top row are from SHIRT dataset: Left image - synthetic domain; Right image - testbed domain. Bottom row are from SPEED+ dataset: Left to right - simulated, testbed Lightbox, testbed Sunlamp settings.
• Figure 3: Pipeline of SPIN. The input to the tool is a 3D model of the spacecraft (the Tango one is the default) and a set of poses. Realism can be configured acting on three scene elements: 1) Camera, that allows to modify the intrinsics and non-idealities such as camera glare, sensor noise, and color adjustment; 2) Environment, that allows to define illumination, background, and shadows rendering; and 3) Materials, that allow enabling high-quality reflective materials. SPIN outputs the intensity images (RGB or grayscale) and ground-truth data including the dense pose, the depth, and the segmentation mask; additionally, a keypoint labeling and heatmap generation tool is provided.
• Figure 4: In-Scene Data Augmentation example. Different renderings for the same pose generated with our tool with different combinations of background, illumination, and color settings.