Fast Vision in the Dark: A Case for Single-Photon Imaging in Planetary Navigation

TL;DR

This paper tackles the challenge of vision-based planetary navigation under extreme illumination by proposing single-photon imaging with SPAD cameras as a passive, high-dynamic-range sensing modality. It presents the first comprehensive evaluation of SPAD cameras against a conventional monochrome camera in lunar-analog lighting, across imaging, segmentation, lander detection, and visual odometry tasks. Key contributions include detailing the SPAD operating principle and imaging model, benchmarking performance under dawn/dusk/noon/night conditions, and highlighting both advantages (robust low-light imaging and HDR, high temporal resolution) and practical limitations (need for algorithms that operate on raw binary streams and bandwidth considerations) along with roadmap for technology readiness. The study suggests SPADs can significantly enhance autonomous planetary robotics by enabling reliable perception in perceptually challenging environments while reducing dependence on external illumination.

Abstract

Improving robotic navigation is critical for extending exploration range and enhancing operational efficiency. Vision-based navigation relying on traditional CCD or CMOS cameras faces major challenges when complex illumination conditions are paired with motion, limiting the range and accessibility of mobile planetary robots. In this study, we propose a novel approach to planetary navigation that leverages the unique imaging capabilities of Single-Photon Avalanche Diode (SPAD) cameras. We present the first comprehensive evaluation of single-photon imaging as an alternative passive sensing technology for robotic exploration missions targeting perceptually challenging locations, with a special emphasis on high-latitude lunar regions. We detail the operating principles and performance characteristics of SPAD cameras, assess their advantages and limitations in addressing key perception challenges of upcoming exploration missions to the Moon, and benchmark their performance under representative illumination conditions.

Figures (6)

• Figure 1: Unlike conventional cameras, which accumulate electric charge in each pixel over a given amount of time and subsequently convert it to a digital value (often of 8 bits per pixel), single-photon cameras provide a direct and synchronized digital output in the form of binary frames (1 bit per pixel). Each pixel value (0 or 1) indicates the absence or presence of an incident photon. This photon-level sensitivity enables the generation of images with arbitrary bit-depth by integrating 2$^n$ number of binary frames, where $n$ is the desired bit-depth of the final image. SPADs can also capture these binary frames at very high speeds, up to 100 kfps (10 $\mu$ per frame), or even combined binary frames taken at different exposures to create high dynamic range images (> 100 dB).
• Figure 2: Experimental setup for our tests at the LunaLab.
• Figure 3: Selected frames taken by a conventional camera and a single-photon camera under different illumination conditions and at 8-bit equivalent exposure times. These frames are extracted from different sections of the SPICE-HL3 dataset trajectory A rodriguez2025spice. All images were captured with the rover headlights switched on.
• Figure 4: Segmented images resulted from running SAM kirillov2023segment over conventional and single-photon images captured under different illumination scenarios. All masks are semantic-free and randomly colorized. Segmentations from 4-bit single-photon frames are also illustrated, to demonstrate the SPAD’s potential ability to reproduce comparable results even at reduced bit depths, despite models like SAM not being originally designed for low-dimensional data. Note that the equivalent exposure time at 4 bit-per-pixel would be 80$\mu$s.
• Figure 5: Results from extracting a SAM-based mask of the background lander structure on both conventional and single-photon camera images.