RF-Photonic Deep Learning Processor with Shannon-Limited Data Movement

TL;DR

An artificial intelligence hardware accelerator that experimentally computes fully analog deep learning on raw radio frequency (RF) signals, performing modulation classification that quickly converges to 95% accuracy and exhibits scalability with nearly 4 million fully analog operations for MNIST digit classification.

Abstract

Edholm's Law predicts exponential growth in data rate and spectrum bandwidth for communications and is forecasted to remain true for the upcoming deployment of 6G. Compounding this issue is the exponentially increasing demand for deep neural network (DNN) compute, including DNNs for signal processing. However, the slowing of Moore's Law due to the limitations of transistor-based electronics means that completely new paradigms for computing will be required to meet these increasing demands for advanced communications. Optical neural networks (ONNs) are promising DNN accelerators with ultra-low latency and energy consumption. Yet state-of-the-art ONNs struggle with scalability and implementing linear with in-line nonlinear operations. Here we introduce our multiplicative analog frequency transform ONN (MAFT-ONN) that encodes the data in the frequency domain, achieves matrix-vector products in a single shot using photoelectric multiplication, and uses a single electro-optic modulator for the nonlinear activation of all neurons in each layer. We experimentally demonstrate the first hardware accelerator that computes fully-analog deep learning on raw RF signals, performing single-shot modulation classification with 85% accuracy, where a 'majority vote' multi-measurement scheme can boost the accuracy to 95% within 5 consecutive measurements. In addition, we demonstrate frequency-domain finite impulse response (FIR) linear-time-invariant (LTI) operations, enabling a powerful combination of traditional and AI signal processing. We also demonstrate the scalability of our architecture by computing nearly 4 million fully-analog multiplies-and-accumulates for MNIST digit classification. Our latency estimation model shows that due to the Shannon capacity-limited analog data movement, MAFT-ONN is hundreds of times faster than traditional RF receivers operating at their theoretical peak performance.

Figures (4)

• Figure 1: An overview of the MAFT-ONN architecture. (a) The MAFT-ONN processor accelerates both traditional signal processing operations and AI inference for waveforms like radio waves. The analog received waveform is fed into MAFT-ONN for fully-analog processing, after which the output may be read out digitally using an analog-to-digital converter (ADC) or fed into another analog system. (b) An outline of the MAFT-ONN architecture. Each photoelectric multiplication physically computes either a fully connected (FC) or a 1D convolution (CONV) layer. The nonlinear activation (NL) for each layer physically corresponds to the nonlinear region of the following modulator. The units can be cascaded to implement several DNN layers fully in analog with no digital overhead. (c) A close-up of a single FC or CONV layer. For a FC layer, the weight signal is programmed such that the photoelectric multiplication yields a matrix-vector product in the frequency domain, where the green region in the frequency domain is isolated with a filter. For a CONV layer, all frequencies of the output signal are used.
• Figure 2: The experimental demonstration of the MAFT-ONN. (a) An outline of the 3-layer MAFT-ONN. (b) The experimental DNN consists of two CONV layers with a nonlinear activation in the hidden layer. (c) A breakdown of the experimental inference of an $14\times14$ MNIST image as the input signal $V_{X}^{(1)}(t)$. The two convolutions are achieved with the weight signals $V_{W}^{(1)}(t)$ and $V_{W}^{(2)}(t)$, respectively. The nonlinear activation is achieved using an amplifier so that the signal $V_{\text{out}}^{(1)}(t)$ reaches the nonlinear region of the DPMZM. As shown in the zoom plot of $V_{\text{out}}^{(2)}(t)$, the image is correctly classified. (d) In the upper half, 2D histograms compare the experimental output values $\hat{Y}$ to the expected curve fitted value $Y$. In the lower half, 1D histograms plot the error $Y - \hat{Y}$. The scalar-scalar plot contains 10,000 randomized $1 \times 1$ matrix products, yielding 9-bit precision compared to the curve fit. The matrix-vector plot contains 10,000 randomized $10\times10$ matrix products (thus 100,000 values), yielding 8-bit precision. (e) An experimental characterization of the nonlinear activation function of an MZM. We programmed $V^{(1)}_X(t)$ as a $10\times1$ input vector, and gradually increased its amplitude until it reached the nonlinear regime of the MZM. We then curve fitted an analytical model to the experimental data. (f) A confusion matrix of the experimental 3-layer DNN over 200 $14\times14$ MNIST images, yielding an experimental accuracy of 90.5%.
• Figure 3: Experimental demonstrations of the RF signal processing capabilities of MAFT-ONN. The signal processing operations in (a)-(c) used the experimental setup in Figure \ref{['fig:A']}(c). (a) The frequency-domain LTI framework was used to implement a Wiener filter that recovers a signal that suffered from frequency-crosstalk and AWGN. (b) The matrix-vector product method from Figure \ref{['fig:A']}(c) was used to implement an LLSE estimator to recover a signal that suffered jointly Gaussian noise in various frequency bands. (c) The frequency-domain LTI framework was used to scan the spectrum of a signal for a specific frequency signature. (d) The 3-layer DNN hardware in Figure \ref{['fig:exp']}(c) was used to experimentally implement modulation classification on raw RF signals using MAFT-ONN. For single-shot inference, the modulation classification experimentally achieved 85.0% accuracy compared to the digital accuracy of 89.2%. (e) With the experimental single-shot accuracy as the baseline, we show that a 'majority vote' multi-measurement scheme will asymptotically improve the accuracy to 100%. With only 5 measurements, the accuracy improves to 95%.
• Figure 4: System-level latency analysis comparing the MAFT-ONN architecture to state-of-the-art digital architectures. (a) Modeling various high-performance computing architectures in the context of receiving an RF signal $x(t)$, implementing a filter on it, and then sending it to a 'decision system' that performs some operation based on the inferred qualities of the processed $x(t)$. The green highlights show that the analog information 'flows' into and through MAFT-ONN, and thus the amount of information moving through the MAFT-ONN processor is limited by the Shannon capacity of the analog channels feeding it information (in the absence of non-ideal factors like multi-path fading and interference). (b) An example of Edholm's Law applied to mobile communications. This shows that wireless mobile bandwidth and data rates have been and will likely continue to exponentially grow. (c) Using the models in (a), the latency for processing $x(t)$ with increasing bandwidth $B$ is estimated for various high-performance processors. We assume that the digital processors are performing at peak specifications with 100% processor and memory utilization in ideal conditions. For simplicity we assume that all FPGA components are running at 500 MHz. Because MAFT-ONN avoids the bottleneck of digital data movement, its overall latency improves by two orders of magnitude compared to traditional receivers.