Digital-analog hybrid matrix multiplication processor for optical neural networks

TL;DR

A digital-analog hybrid matrix multiplication processor achieving 16-bit numerical precision and offering a potential solution to practical optical neural networks is demonstrated.

Abstract

The computational demands of modern AI have spurred interest in optical neural networks (ONNs) which offer the potential benefits of increased speed and lower power consumption. However, current ONNs face various challenges,most significantly a limited calculation precision (typically around 4 bits) and the requirement for high-resolution signal format converters (digital-to-analogue conversions (DACs) and analogue-to-digital conversions (ADCs)). These challenges are inherent to their analog computing nature and pose significant obstacles in practical implementation. Here, we propose a digital-analog hybrid optical computing architecture for ONNs, which utilizes digital optical inputs in the form of binary words. By introducing the logic levels and decisions based on thresholding, the calculation precision can be significantly enhanced. The DACs for input data can be removed and the resolution of the ADCs can be greatly reduced. This can increase the operating speed at a high calculation precision and facilitate the compatibility with microelectronics. To validate our approach, we have fabricated a proof-of-concept photonic chip and built up a hybrid optical processor (HOP) system for neural network applications. We have demonstrated an unprecedented 16-bit calculation precision for high-definition image processing, with a pixel error rate (PER) as low as $1.8\times10^{-3}$ at an signal-to-noise ratio (SNR) of 18.2 dB. We have also implemented a convolutional neural network for handwritten digit recognition that shows the same accuracy as the one achieved by a desktop computer. The concept of the digital-analog hybrid optical computing architecture offers a methodology that could potentially be applied to various ONN implementations and may intrigue new research into efficient and accurate domain-specific optical computing architectures for neural networks.

Figures (5)

• Figure 1: Concept of the digital-analog hybrid photonic *MVM core.a, Digital signal is more robust to noise and crosstalk in a signal processing system. *ONN can be seen as a signal processing system where digital signals can potentially be applied for better calculation repeatability, precision, scalability, and compatibility with microelectronics compared to analog signals. b, The abstracted analog photonic *MVM core utilizing analog signals for both input $d$ and weight $w$. c, The proposed digital-analog hybrid photonic *MVM core utilizing digital signals for input $d$, with relaxed constraints for signal format converters.
• Figure 2: A *MRM based implementation of the hybrid digital-analog photonic *MVM core.a, The implementation of a single hybrid digital-analog photonic multiplication core using an *MRM, where the input $d$ in the form of a digital optical signal is loaded through the high-speed port while the weight $w$ is loaded using microheater based modulation bias. b, The measured relationship between the normalized weight and the required heater voltage for the modulation bias, can be used as a lookup table to load the weight. c, The implementation and optical signal temporal evolution of the digital-analog hybrid photonic *MVM core, including a multi-wavelength light source, an array of microring modulators, and a photodiode. d, Post-processing of the multilevel signal includes an *ADC and a shift-add operation. Here the multilevel signal is converted to a binary signal and the final result can be recovered via shifts and adds.
• Figure 3: Simulation setup and results.a, Simulation setup. An image "Chelsea" from the scikit-image datasetvan2014scikit is convolved with a Prewitt operator (vertical edge detection). We explore the noise tolerance of both the analog and the hybrid optical computing systems by adding additive white Gaussian noise to the weights and examine the performance of the system by investigating the noise distribution of the outputs. b,c, Distribution of expected pixel values against the processed pixel values (both normalized) at an SNR of 25 dB, for the analog and hybrid computing systems, respectively. Insets show the corresponding processed images. Noisy pixels can be clearly observed in the image processed using analog computing. d,e, Noise distribution of the analog and hybrid computing systems, respectively, at an SNR of 25 dB. Analog computing reveals a Gaussian noise distribution with a standard deviation of 0.027, corresponding to a calculation precision of 3.6 bits. The HOP shows a greatly improved noise distribution thanks to the introduction of logic levels and decisions based on thresholding. f, performance of the analog and hybrid computing schemes in terms of RMSE with different SNR.
• Figure 4: Experimental setup.a, Data loading scheme. The inputs of the HOP are the pixel values of the "Chelsea" image, used to perform the convolution operation. The inputs are loaded to the high-speed modulation ports of a set of 9 MRM in binary words. The convolution operator is reshaped into a $1\times9$ vector and applied to the same set of MRM using microheater-based modulation biases. b, Measurement setup, including DSP and signal postprocessing. The HOP consists of a packaged PIC chip containing 20 cascaded MRM, and an external PD. Insets give the picture of the packaged chip and the microscopic image of the cascaded MRM. c, The optical spectrum of the optical frequency comb source. 9 flattened comb lines are generated and fed into the MRM chip. d, Detailed operation condition and signal flow for the *HDIP task. e, Detailed operation condition and signal flow for the *HWDR task.
• Figure 5: Experiment results.a, The original 16-bit image and the processed image channels using the Prewitt vertical, Sobel vertical and Laplacian operators. b, A section of the processed sequences of pixel values. Up: processed by the HOP; Down: processed by a desktop computer. c, Distribution of expected pixel values against the processed pixel values (both normalized). d, Noise distribution and calculation accuracy. e, Pixel error rate against light source OSNR. The measurements are performed on an 8-bit image processed with the Prewitt vertical operator. f, Layer structure of the CNN to perform the HWDR task using the MNIST database. The convolutional layer of the CNN is implemented using the HOP and the rest of the network is performed offline by a desktop computer. g, Confusion matrices for the prediction results, calculated by a desktop computer and the HOP.