Scalable Superconductor Neuron with Ternary Synaptic Connections for Ultra-Fast SNN Hardware

TL;DR

The paper tackles the challenge of building ultra-fast, energy-efficient neuromorphic hardware by introducing a high-fan-in differential superconductor neuron built from multiple loops with two Josephson Junctions per branch and ternary synapses, driven by Single Flux Quantum pulses. The approach enables scalable SNN inference, demonstrated with a fully connected MNIST network achieving 97.07% accuracy pre-pruning and 96.1% post-pruning, at throughput up to 8.92 GHz and energy around 1.5 nJ per inference (including cooling). Key contributions include a practical neuron design with tunable thresholds, compatibility with various synaptic devices, 1-bit input quantization, and a pruning strategy that dramatically reduces synapses and static power with minimal accuracy loss. The results underscore the potential of superconducting SFQ-based hardware to deliver high-throughput, low-energy SNN accelerators for real-time neuromorphic applications.

Abstract

A novel high-fan-in differential superconductor neuron structure designed for ultra-high-performance Spiking Neural Network (SNN) accelerators is presented. Utilizing a high-fan-in neuron structure allows us to design SNN accelerators with more synaptic connections, enhancing the overall network capabilities. The proposed neuron design is based on superconductor electronics fabric, incorporating multiple superconducting loops, each with two Josephson Junctions. This arrangement enables each input data branch to have positive and negative inductive coupling, supporting excitatory and inhibitory synaptic data. Compatibility with synaptic devices and thresholding operation is achieved using a single flux quantum (SFQ) pulse-based logic style. The neuron design, along with ternary synaptic connections, forms the foundation for a superconductor-based SNN inference. To demonstrate the capabilities of our design, we train the SNN using snnTorch, augmenting the PyTorch framework. After pruning, the demonstrated SNN inference achieves an impressive 96.1% accuracy on MNIST images. Notably, the network exhibits a remarkable throughput of 8.92 GHz while consuming only 1.5 nJ per inference, including the energy consumption associated with cooling to 4K. These results underscore the potential of superconductor electronics in developing high-performance and ultra-energy-efficient neural network accelerator architectures.

Figures (5)

• Figure 1: I-V curve of a JJ and SFQ pulse generation.
• Figure 2: Network architecture comprising a standard D Flip-Flop (DFF) and Leaky Integrate and Fire (LIF) neuron. DFFs provide input pulses to the network and perform an initial synchronization for the data flow. On the right side, a neuron circuit schematic includes multiple branches to accommodate the high-fan-in feature. In this configuration, the neuron performs the leaky accumulation within the resistive loops and activation function with JJs.
• Figure 3: Two different pre- and post-quantization images for the number 9. Each image is 28$\times$28 pixels.
• Figure 4: The simulation of the proposed neuron with 32 synaptic connections was performed using JoSIM, where the threshold and input frequency were set to 4 SFQ pulses and 20 GHz, respectively. The simulation results are presented in the form of plots displaying the converted JJ phase data for better visualization. In these plots, the observed pulses from the JJs in JTLs connected to the neuron are evident. The color blue represents positive inputs, while negative inputs are displayed in red.
• Figure 5: The simulation result of the proposed neuron where negative pulses arrive earlier than the positive pulses for the initialization. The neuron threshold and input frequency were set to 4 SFQ pulses and 10 GHz, respectively. The JJ phase data is color-coded and converted for better visualization. (LP = 6.0pH, LX = 11.63pH, LN = 4.8pH, LOP = 0.2pH, JJP = $82\mu A$, RP = RN = 0.37 $\Omega$, RDP = 3.36 $\Omega$, RBP = 32.1 $\Omega$, KPX = 0.43, KNX = -0.59)