Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Time-Resolved MNIST Dataset for Single-Photon Recognition

Aleksi Suonsivu, Lauri Salmela, Edoardo Peretti, Leevi Uosukainen, Radu Ciprian Bilcu, Giacomo Boracchi

TL;DR

A realistic simulation process for SPAD imaging is described, which takes into account both the stochastic nature of photon arrivals and all the noise sources involved in the acquisition process of time-resolved SPAD arrays.

Abstract

Time-resolved single photon imaging is a promising imaging modality characterized by the unique capability of timestamping the arrivals of single photons. Single-Photon Avalanche Diodes (SPADs) are the leading technology for implementing modern time-resolved pixels, suitable for passive imaging with asynchronous readout. However, they are currently limited to small sized arrays, thus there is a lack of datasets for passive time-resolved SPAD imaging, which in turn hinders research on this peculiar imaging data. In this paper we describe a realistic simulation process for SPAD imaging, which takes into account both the stochastic nature of photon arrivals and all the noise sources involved in the acquisition process of time-resolved SPAD arrays. We have implemented this simulator in a software prototype able to generate arbitrary-sized time-resolved SPAD arrays operating in passive mode. Starting from a reference image, our simulator generates a realistic stream of timestamped photon detections. We use our simulator to generate a time-resolved version of MNIST, which we make publicly available. Our dataset has the purpose of encouraging novel research directions in time-resolved SPAD imaging, as well as investigating the performance of CNN classifiers in extremely low-light conditions.

Time-Resolved MNIST Dataset for Single-Photon Recognition

TL;DR

A realistic simulation process for SPAD imaging is described, which takes into account both the stochastic nature of photon arrivals and all the noise sources involved in the acquisition process of time-resolved SPAD arrays.

Abstract

Time-resolved single photon imaging is a promising imaging modality characterized by the unique capability of timestamping the arrivals of single photons. Single-Photon Avalanche Diodes (SPADs) are the leading technology for implementing modern time-resolved pixels, suitable for passive imaging with asynchronous readout. However, they are currently limited to small sized arrays, thus there is a lack of datasets for passive time-resolved SPAD imaging, which in turn hinders research on this peculiar imaging data. In this paper we describe a realistic simulation process for SPAD imaging, which takes into account both the stochastic nature of photon arrivals and all the noise sources involved in the acquisition process of time-resolved SPAD arrays. We have implemented this simulator in a software prototype able to generate arbitrary-sized time-resolved SPAD arrays operating in passive mode. Starting from a reference image, our simulator generates a realistic stream of timestamped photon detections. We use our simulator to generate a time-resolved version of MNIST, which we make publicly available. Our dataset has the purpose of encouraging novel research directions in time-resolved SPAD imaging, as well as investigating the performance of CNN classifiers in extremely low-light conditions.

Paper Structure

This paper contains 15 sections, 7 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Passive Time-Resolved Single-Photon Imaging
  3. Alternative paradigms for SPAD sensors
  4. Related Work
  5. Synthetic datasets for TR-SPAD applications
  6. Relation with event cameras
  7. Algorithms for stream of events
  8. Low-light datasets
  9. Simulator
  10. SPAD Characteristics
  11. From scene to photons
  12. Dataset
  13. Image classification and photons statistics at low-light
  14. Baseline and Experiments
  15. Conclusion

Figures (4)

  • Figure 1: Illustration of pixel-wise SPAD model used by our simulator. An input image is transformed to a photon flux, via a reference lux level, which is then used to simulate photon detections. The pixel-wise SPAD model is applied to all pixels in the image.
  • Figure 2: Samples from TR-MNIST-rec of reconstructions at different lux levels with 1 ms integration time. All other simulation parameters have been fixed in Section \ref{['sec:simulator']}.
  • Figure 3: Histograms of photon counts at different lux levels with 1 ms integration time.
  • Figure 4: Classification accuracy of LeNet on TR-SPAD-rec, with different reconstruction methods and at different lux levels. The reference accuracy is obtained from the original MNIST dataset. Note that the accuracy of $\widehat{\Phi}_c$ and $\widehat{\Phi}_{PF}$ are almost coincident.