Overcoming Quadratic Hardware Scaling for a Fully Connected Digital Oscillatory Neural Network

TL;DR

The paper tackles the quadratic hardware growth in fully connected digital Oscillatory Neural Networks (ONNs) by introducing a hybrid architecture that serializes the weighted-sum computation, achieving near-linear hardware scaling. It demonstrates a 10.5× increase in implementable oscillators on a single FPGA (up to 506 nodes) while preserving ONN dynamics and pattern retrieval performance comparable to a baseline recurrent design. Hardware-scaling analysis shows the recurrent architecture scales quadratically in resource usage, whereas the hybrid approach scales roughly linearly with a slope near 1.2, allowing much larger networks to be realized. Pattern retrieval benchmarks across multiple pattern sizes confirm similar accuracy and settling times between architectures, validating the hybrid design for large-scale ONN applications and edge computing. This work thus enables large-scale, fully connected digital ONNs suitable for combinatorial optimization and related tasks, by trading some oscillator frequency for substantial gains in hardware efficiency.

Abstract

Computing with coupled oscillators or oscillatory neural networks (ONNs) has recently attracted a lot of interest due to their potential for massive parallelism and energy-efficient computing. However, to date, ONNs have primarily been explored either analytically or through analog circuit implementations. This paper shifts the focus to the digital implementation of ONNs, examining various design architectures. We first report on an existing digital ONN design based on a recurrent architecture. The major challenge for scaling such recurrent architectures is the quadratic increase in coupling hardware with the network size. To overcome this challenge, we introduce a novel hybrid architecture that balances serialization and parallelism in the coupling elements that shows near-linear hardware scaling, on the order of about 1.2 with the network size. Furthermore, we evaluate the benefits and costs of these different digital ONN architectures in terms time to solution and resource usage on FPGA emulation. The proposed hybrid architecture allows for a 10.5$\times$ increase in the number of oscillators while using 5-bits to represent the coupling weights and 4-bits to represent the oscillator phase on a Zynq-7020 FPGA board. The near-linear scaling is a major step towards implementing large scale ONN architectures. To the best of our knowledge, this work presents the largest fully connected digital ONN architecture implemented thus far with a total of 506 fully connected oscillators.

Figures (12)

• Figure 1: Illustration of oscillatory neural network for pattern retrieval. Pattern retrieval process where each pixel in the pattern is represented with an oscillator. The coupling weights in the network are determined using a learning rule. A corrupted pattern is injected into a network as initial condition. The oscillator phases naturally evolve in parallel to the memorized pattern.
• Figure 2: Global architecture overview for the digital ONN. The network is divided into two parts. One part for the coupling elements and one part for the oscillators. The coupling elements contain a $N\times N$ weight matrix for the coupling weights and $N$ arithmetic circuits, where $N$ is the number of oscillators in the architecture.
• Figure 3: Phase controlled oscillator architecture. A circular shift registered is multiplexed to create a phase controlled oscillator. The first half of the registers are initialized with 1's and the second half with 0's.
• Figure 4: Parallel arithmetic circuit implementation for recurrent architecture. Oscillator amplitudes are multiplied by their respective coupling weight. These values are then accumulated to a total sum. The computation circuitry is implemented completely in combinatorial logic, leading to a deep logic structure with a long path delay.
• Figure 5: Serial arithmetic circuit implementation for hybrid architecture. Oscillator amplitudes are multiplied by their respective coupling weight. These values are then accumulated to a total sum. The computation circuitry is implemented with one adder to compute the sum serially over multiple cycles.