Single Pixel Image Classification using an Ultrafast Digital Light Projector

Abstract

Pattern recognition and image classification are essential tasks in machine vision. Autonomous vehicles, for example, require being able to collect the complex information contained in a changing environment and classify it in real time. Here, we experimentally demonstrate image classification at multi-kHz frame rates combining the technique of single pixel imaging (SPI) with a low complexity machine learning model. The use of a microLED-on-CMOS digital light projector for SPI enables ultrafast pattern generation for sub-ms image encoding. We investigate the classification accuracy of our experimental system against the broadly accepted benchmarking task of the MNIST digits classification. We compare the classification performance of two machine learning models: An extreme learning machine (ELM) and a backpropagation trained deep neural network. The complexity of both models is kept low so the overhead added to the inference time is comparable to the image generation time. Crucially, our single pixel image classification approach is based on a spatiotemporal transformation of the information, entirely bypassing the need for image reconstruction. By exploring the performance of our SPI based ELM as binary classifier we demonstrate its potential for efficient anomaly detection in ultrafast imaging scenarios.

Figures (10)

• Figure 1: Schematic diagram of our single pixel image classification (SPIC) framework. Each Hadamard pattern in the imaging base is projected onto the object. The superposition of both images is converted into a scalar photocurrent and recorded by a real time oscilloscope. The sequence of all patterns is processed by an artificial neural network whose output is already the object's class.
• Figure 2: Experimental setup for single pixel image classification. The microLED projector generates the sequence of Hadamard patterns that illuminate the image presented on the Digital Micromirror Device (DMD). The Single Pixel detector collects the superposition of each image pair and transforms it into a scalar photocurrent registered by the real time oscilloscope. The full time series corresponding to each image is post-processed by a software encoded ML model.
• Figure 3: Single pixel measurement and reconstruction process. (a) Original and binarized MNIST digit used as the target object for structured illumination. (b) Illustration of the SPI process: The upper panels show an optically encoded binarized MNIST digit and its corresponding photodetected time series. The panels below show a magnification of this time series, superposing the corresponding Hadamard patterns and the reconstructed MNIST digit.
• Figure 4: Basic architectures and training of the two machine learning models considered in this work. Diagrams (a) and (b) depict the extreme learning machine model used for multiclass and one-vs-all binary classification with fixed random input weights and output training only using ridge regression. Diagrams (c) and (d) depict a feed-forward deep neural network for multiclass classification with trainable weights using gradient descent.
• Figure 5: (a) Classification performance of the multiclass ELM model versus hidden neurons at $\alpha = 1$. (b) Normalized confusion matrix (%) for the multiclass ELM model using $1000$ hidden neurons at $\alpha = 1$.