Accurate Simulation Pipeline for Passive Single-Photon Imaging

TL;DR

This work tackles the scarcity of SPAD data by introducing a physics-based passive SPAD imaging simulator that models per-pixel noise sources and supports asynchronous TR-SPAD, synchronous STR-SPAD, and SPAD-QIS modalities. It maps grayscale images to photon flux, generates photon-detection streams with Poisson statistics, and produces modality-specific outputs, enabling realistic SPAD data generation. The authors validate the simulator against two commercial sensors and release SPAD-MNIST, a large three-modal SPAD-dataset derived from MNIST for very low-light conditions. They further demonstrate that CNN classifiers trained on synthetic SPAD data can perform well on real SPAD data, highlighting the practical value of synthetic SPAD data for low-light vision and model pretraining. Overall, the paper provides a valuable tool and benchmark to accelerate SPAD imaging research and development of SPAD-specific processing pipelines.

Abstract

Single-Photon Avalanche Diodes (SPADs) are new and promising imaging sensors. These sensors are sensitive enough to detect individual photons hitting each pixel, with extreme temporal resolution and without readout noise. Thus, SPADs stand out as an optimal choice for low-light imaging. Due to the high price and limited availability of SPAD sensors, the demand for an accurate data simulation pipeline is substantial. Indeed, the scarcity of SPAD datasets hinders the development of SPAD-specific processing algorithms and impedes the training of learning-based solutions. In this paper, we present a comprehensive SPAD simulation pipeline and validate it with multiple experiments using two recent commercial SPAD sensors. Our simulator is used to generate the SPAD-MNIST, a single-photon version of the seminal MNIST dataset, to investigate the effectiveness of convolutional neural network (CNN) classifiers on reconstructed fluxes, even at extremely low light conditions, e.g., 5 mlux. We also assess the performance of classifiers exclusively trained on simulated data on real images acquired from SPAD sensors at different light conditions. The synthetic dataset encompasses different SPAD imaging modalities and is made available for download. Project page: https://boracchi.faculty.polimi.it/Projects/SPAD-MNIST.html.

Figures (10)

• Figure 1: Illustration of the SPAD simulation model. The SPAD model is applied pixel-wise in the simulations. Notice that the QIS frame exposure time is typically much longer than the dead time of the SPAD (i.e. not plotted in scale).
• Figure 2: Experiment setup consisting of a tunable narrowband light source, integrating sphere and the SPAD sensor.
• Figure 3: SPAD23 simulator and experiments comparison at intermediate flux. The mean photon flux is 17.0 kcps in both experiments and simulations for 500 nm, and 2.28 kcps and 2.27 kcps for 700 nm, respectively.
• Figure 4: SPAD512$^2$ simulator and experiments count histogram comparison at low photon counts. At 500 nm, the mean photon counts are 470 and 457 for the experiments (EXPT) and simulations (SIM), respectively, and for 700 nm the mean photon counts are 62 and 60. The counts are over 10000 binary frames with 1 µ s integration time. The histograms also show the measured dark counts (DC) with a mean photon count of 2.6, with 265 (20) counts per second mean (median) DCR.
• Figure 5: SPAD512$^2$ simulator and experiments count histogram comparison at high photon counts. At 500 nm, the mean photon counts are 9715 and 9740 for the experiments (EXPT) and simulations (SIM), respectively, and for 700 nm the mean photon counts are 4011 and 4184. The counts are over 10000 binary frames with 10 µ s integration time. Note that compared to Fig. \ref{['fig:spad512val']}, the overall incoming flux is also higher. The simulations currently consider all pixels to have a uniform PDP, and the effect of "lazy pixels" can be seen in the experiments.