Improved 3D Gaussian Splatting of Unknown Spacecraft Structure Using Space Environment Illumination Knowledge

TL;DR

This work addresses the problem of reconstructing unknown spacecraft geometry from image sequences captured during Rendezvous and Proximity Operations under dynamic space illumination. It introduces a Sun-vector–driven online training pipeline for 3D Gaussian Splatting (3DGS) that replaces per-image appearance embeddings with per-Gaussian features and directional encodings, and augments the rendering with shadow splatting and a lightweight shadow refinement network. Key contributions include Sun-direction conditioning, shadow-aware visibility, isotropic regularization to sharpen shadows, and a simple keyframe management strategy, all evaluated on high-fidelity synthetic RPO data. The results demonstrate improved photometric fidelity and the ability to capture global illumination and self-occlusion, enabling more reliable online pose estimation for space missions, with clear paths to relax current assumptions via SLAM integrations and additional sensing modalities.

Abstract

This work presents a novel pipeline to recover the 3D structure of an unknown target spacecraft from a sequence of images captured during Rendezvous and Proximity Operations (RPO) in space. The target's geometry and appearance are represented as a 3D Gaussian Splatting (3DGS) model. However, learning 3DGS requires static scenes, an assumption in contrast to dynamic lighting conditions encountered in spaceborne imagery. The trained 3DGS model can also be used for camera pose estimation through photometric optimization. Therefore, in addition to recovering a geometrically accurate 3DGS model, the photometric accuracy of the rendered images is imperative to downstream pose estimation tasks during the RPO process. This work proposes to incorporate the prior knowledge of the Sun's position, estimated and maintained by the servicer spacecraft, into the training pipeline for improved photometric quality of 3DGS rasterization. Experimental studies demonstrate the effectiveness of the proposed solution, as 3DGS models trained on a sequence of images learn to adapt to rapidly changing illumination conditions in space and reflect global shadowing and self-occlusion.

Figures (8)

• Figure 1: The rendered images of a previously unknown satellite generated by a 3DGS model trained on a sequence of images captured during a simulated RPO scenario. Compared with vanilla 3DGS, the proposed training pipeline explicitly incorporates knowledge of the Sun vector maintained by the onboard AOCS module through shadow splatting, significantly improving the photometric accuracy of the rendered images.
• Figure 2: Proposed 3DGS pipeline with shadow splatting using the known Sun vector.
• Figure 3: (Left) Visualization of the adopted satellite models. (Right) The servicer's orbit relative to the target in the RTN frame.
• Figure 4: Qualitative results of different configurations trained on sequential datasets.
• Figure 5: Visualization of raw Gaussians after training without (left) and with (right) the $\mathcal{L}_\text{iso}$ term.