A Photorealistic Dataset and Vision-Based Algorithm for Anomaly Detection During Proximity Operations in Lunar Orbit

TL;DR

This work tackles anomaly detection during proximity operations in lunar orbit by introducing ALLO, a photorealistic synthetic dataset, and MRAD, a Model Reference Anomaly Detection algorithm. MRAD uses a known camera pose and a CAD model to render a reference scene and computes per-pixel deviations via a cross-image RXD-based score, followed by noise filtering and region-growing to localize anomalies. On ALLO, MRAD achieves state-of-the-art pixel-level AP and competitive image-level AUROC, outperforming multiple learning-based baselines and highlighting the value of reference-aware, learning-free methods in space domains. The dataset and method address the domain gap between Earth-based datasets and space imagery, enabling robust autonomous perception for space missions and guiding future research toward hybrid learning-reference approaches.

Abstract

NASA's forthcoming Lunar Gateway space station, which will be uncrewed most of the time, will need to operate with an unprecedented level of autonomy. One key challenge is enabling the Canadarm3, the Gateway's external robotic system, to detect hazards in its environment using its onboard inspection cameras. This task is complicated by the extreme and variable lighting conditions in space. In this paper, we introduce the visual anomaly detection and localization task for the space domain and establish a benchmark based on a synthetic dataset called ALLO (Anomaly Localization in Lunar Orbit). We show that state-of-the-art visual anomaly detection methods often fail in the space domain, motivating the need for new approaches. To address this, we propose MRAD (Model Reference Anomaly Detection), a statistical algorithm that leverages the known pose of the Canadarm3 and a CAD model of the Gateway to generate reference images of the expected scene appearance. Anomalies are then identified as deviations from this model-generated reference. On the ALLO dataset, MRAD surpasses state-of-the-art anomaly detection algorithms, achieving an AP score of 62.9% at the pixel level and an AUROC score of 75.0% at the image level. Given the low tolerance for risk in space operations and the lack of domain-specific data, we emphasize the need for novel, robust, and accurate anomaly detection methods to handle the challenging visual conditions found in lunar orbit and beyond.

Figures (7)

• Figure 1: Visualization of the rendering process used to generate an anomalous image in the ALLO dataset. (a) Positions of the Sun, Moon, and space station are determined and scene lighting is configured accordingly; (b) An anomaly is inserted into the scene; (c) The Blender Cycles engine renders the image; and (d) the corresponding three-class segmentation mask is generated.
• Figure 2: Examples of models of anomalous objects used in the ALLO dataset.
• Figure 3: Sample images from the ALLO dataset. Top Row: Normal (anomaly-free) images captured from five distinct camera views of the ISS. Bottom Row: Images containing anomalies, identified by red ellipses, that include free-floating equipment such as cables, thermal blankets, and a hand drill.
• Figure 4: Real images of the ISS (left) and their synthetic recreations (right).
• Figure 5: Left to right: reference image from the ALLO test set with one grid cell highlighted; pixel intensity histogram of the highlighted cell; anomalous query image; predicted anomaly map.