Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Photonic convolutional neural network with pre-trained in-situ training

Saurabh Ranjan, Sonika Thakral, Amit Sehgal

Abstract

Photonic computing is a computing paradigm which have great potential to overcome the energy bottlenecks of electronic von Neumann architecture. Throughput and power consumption are fundamental limitations of Complementary-metal-oxide-semiconductor (CMOS) chips, therefore convolutional neural network (CNN) is revolutionising machine learning, computer vision and other image based applications. In this work, we propose and validate a fully photonic convolutional neural network (PCNN) that performs MNIST image classification entirely in the optical domain, achieving 94 percent test accuracy. Unlike existing architectures that rely on frequent in-between conversions from optical to electrical and back to optical (O/E/O), our system maintains coherent processing utilizing Mach-Zehnder interferometer (MZI) meshes, wavelength-division multiplexed (WDM) pooling, and microring resonator-based nonlinearities. The max pooling unit is fully implemented on silicon photonics, which does not require opto-electrical or electrical conversions. To overcome the challenges of training physical phase shifter parameters, we introduce a hybrid training methodology deploying a mathematically exact differentiable digital twin for ex-situ backpropagation, followed by in-situ fine-tuning via Simultaneous Perturbation Stochastic Approximation (SPSA) algorithm. Our evaluation demonstrates significant robustness to thermal crosstalk (only 0.43 percent accuracy degradation at severe coupling) and achieves 100 to 242 times better energy efficiency than state-of-the-art electronic GPUs for single-image inference.

Photonic convolutional neural network with pre-trained in-situ training

Abstract

Photonic computing is a computing paradigm which have great potential to overcome the energy bottlenecks of electronic von Neumann architecture. Throughput and power consumption are fundamental limitations of Complementary-metal-oxide-semiconductor (CMOS) chips, therefore convolutional neural network (CNN) is revolutionising machine learning, computer vision and other image based applications. In this work, we propose and validate a fully photonic convolutional neural network (PCNN) that performs MNIST image classification entirely in the optical domain, achieving 94 percent test accuracy. Unlike existing architectures that rely on frequent in-between conversions from optical to electrical and back to optical (O/E/O), our system maintains coherent processing utilizing Mach-Zehnder interferometer (MZI) meshes, wavelength-division multiplexed (WDM) pooling, and microring resonator-based nonlinearities. The max pooling unit is fully implemented on silicon photonics, which does not require opto-electrical or electrical conversions. To overcome the challenges of training physical phase shifter parameters, we introduce a hybrid training methodology deploying a mathematically exact differentiable digital twin for ex-situ backpropagation, followed by in-situ fine-tuning via Simultaneous Perturbation Stochastic Approximation (SPSA) algorithm. Our evaluation demonstrates significant robustness to thermal crosstalk (only 0.43 percent accuracy degradation at severe coupling) and achieves 100 to 242 times better energy efficiency than state-of-the-art electronic GPUs for single-image inference.

Paper Structure

This paper contains 13 sections, 4 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Architecture overview
  3. Training approach
  4. Differentiable Digital Twin
  5. In-Situ Fine-Tuning
  6. Results
  7. Energy Efficiency and Latency
  8. Discussion
  9. Inference Latency Analysis
  10. Energy Per Operation
  11. Thermal Crosstalk Robustness
  12. Calibration and Digital Twin Fidelity
  13. Conclusion

Figures (2)

  • Figure 1: Digital twin pre-training dynamics. (Left) Cross-entropy loss curve over 20 training epochs. (Right) Training and test accuracy curves, converging to 96.92% test accuracy in 20 epochs.
  • Figure 2: Hardware PCNN confusion matrix on the full MNIST test set (10,000 images). Rows represent true digit classes and columns represent predicted classes. The strong diagonal indicates high per-class accuracy across all 10 digits.