Annealing-inspired training of an optical neural network with ternary weights

TL;DR

This work tackles in-situ learning for photonic neural networks under hardware constraints by introducing a ternary-weight readout for a LA-VCSEL reservoir and an annealing-inspired adaptive learning algorithm compatible with Boolean or ternary weights. The approach achieves substantial performance gains, including a notable improvement when moving from Boolean to ternary weights on MNIST, while maintaining stability over extended operation. Experimental results on binary header recognition and MNIST with ~450 neurons show the ternary scheme closely approaches a digital linear baseline, and long-term inference remains highly stable (consistency > 99% over 10 hours). The study highlights weight resolution as the main bottleneck and demonstrates the viability of energy-efficient, in-situ learning for photonic hardware.

Abstract

Artificial neural networks (ANNs) represent a fundamentally connectionnist and distributed approach to computing, and as such they differ from classical computers that utilize the von Neumann architecture. This has revived research interest in new unconventional hardware to enable more efficient implementations of ANNs rather than emulating them on traditional machines. In order to fully leverage the capabilities of this new generation of ANNs, optimization algorithms that take into account hardware limitations and imperfections are necessary. Photonics represents a particularly promising platform, offering scalability, high speed, energy efficiency, and the capability for parallel information processing. Yet, fully fledged implementations of autonomous optical neural networks (ONNs) with in-situ learning remain scarce. In this work, we propose a ternary weight architecture high-dimensional semiconductor laser-based ONN. We introduce a simple method for achieving ternary weights with Boolean hardware, significantly increasing the ONN's information processing capabilities. Furthermore, we design a novel in-situ optimization algorithm that is compatible with, both, Boolean and ternary weights, and provide a detailed hyperparameter study of said algorithm for two different tasks. Our novel algorithm results in benefits, both in terms of convergence speed and performance. Finally, we experimentally characterize the long-term inference stability of our ONN and find that it is extremely stable with a consistency above 99\% over a period of more than 10 hours, addressing one of the main concerns in the field. Our work is of particular relevance in the context of in-situ learning under restricted hardware resources, especially since minimizing the power consumption of auxiliary hardware is crucial to preserving efficiency gains achieved by non-von Neumann ANN implementations.

Figures (7)

• Figure 1: Experimental setup and concept scheme.a Schematic of the full experimental setup. Details about the individual components are given in table \ref{['tab:components_ref']}. b Simplified diagram explaining our experimental scheme for clarity.
• Figure 2: Experimental charaterization of the LA-VCSEL.a L-I curve of the LA-VCSEL. b Spectra of the free running (blue) and injection locked (red) LA-VCSEL. c Mode profiles of the free running (blue) and injection locked (red) LA-VCSEL.
• Figure 3: Ternary weights implementation. Diagram showing how we use a single DMD to achieve ternary weights.
• Figure 4: Impact of hyperparameter $\alpha$ on the learning performance for a 4-bit header recognition task.
• Figure 5: Impact of tuning hyperparameter $\alpha$ on the learning performance for classification of digit 8 in the MNIST dataset.