Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hybrid Synaptic Structure for Spiking Neural Network Realization

Sasan Razmkhah, Mustafa Altay Karamuftuoglu, Ali Bozbey

TL;DR

The paper tackles building ultra-fast, low-power neuromorphic hardware using superconducting SFQ circuits by introducing a compact JJ-Synapse with positive/negative inputs and pairing it with a JJ-Soma in a cryogenic-CMOS weight-control framework. The synapse implements a quantized-weight model with the equation $u = \sum_{k=1}^{n}(w_kP_k - w'_kN_k)$, with pulse accumulation, a Buffer/Quantizer, and programmable weight cells via SQUID arrays, enabling 3- to 4-bit weights. Simulations (JSIM) with particle swarm optimization show >$20%$ margins and successful 3-bit weight operation; experiments indicate feasibility of 10 GHz inputs (potentially 25 GHz) and threshold-driven soma firing within tight timing windows, with measurements validating weight control via SiGe cryo-CMOS. Fabrication in Nb-based processes and cryogenic CMOS compatibility pave the way for a high-speed, programmable SFQ-based SNN operating at tens of GHz with atttojoule-level energy per operation.

Abstract

Neural networks and neuromorphic computing play pivotal roles in deep learning and machine vision. Due to their dissipative nature and inherent limitations, traditional semiconductor-based circuits face challenges in realizing ultra-fast and low-power neural networks. However, the spiking behavior characteristic of single flux quantum (SFQ) circuits positions them as promising candidates for spiking neural networks (SNNs). Our previous work showcased a JJ-Soma design capable of operating at tens of gigahertz while consuming only a fraction of the power compared to traditional circuits, as documented in [1]. This paper introduces a compact SFQ-based synapse design that applies positive and negative weighted inputs to the JJ-Soma. Using an RSFQ synapse empowers us to replicate the functionality of a biological neuron, a crucial step in realizing a complete SNN. The JJ-Synapse can operate at ultra-high frequencies, exhibits orders of magnitude lower power consumption than CMOS counterparts, and can be conveniently fabricated using commercial Nb processes. Furthermore, the network's flexibility enables modifications by incorporating cryo-CMOS circuits for weight value adjustments. In our endeavor, we have successfully designed, fabricated, and partially tested the JJ-Synapse within our cryocooler system. Integration with the JJ-Soma further facilitates the realization of a high-speed inference SNN.

Hybrid Synaptic Structure for Spiking Neural Network Realization

TL;DR

The paper tackles building ultra-fast, low-power neuromorphic hardware using superconducting SFQ circuits by introducing a compact JJ-Synapse with positive/negative inputs and pairing it with a JJ-Soma in a cryogenic-CMOS weight-control framework. The synapse implements a quantized-weight model with the equation , with pulse accumulation, a Buffer/Quantizer, and programmable weight cells via SQUID arrays, enabling 3- to 4-bit weights. Simulations (JSIM) with particle swarm optimization show > margins and successful 3-bit weight operation; experiments indicate feasibility of 10 GHz inputs (potentially 25 GHz) and threshold-driven soma firing within tight timing windows, with measurements validating weight control via SiGe cryo-CMOS. Fabrication in Nb-based processes and cryogenic CMOS compatibility pave the way for a high-speed, programmable SFQ-based SNN operating at tens of GHz with atttojoule-level energy per operation.

Abstract

Neural networks and neuromorphic computing play pivotal roles in deep learning and machine vision. Due to their dissipative nature and inherent limitations, traditional semiconductor-based circuits face challenges in realizing ultra-fast and low-power neural networks. However, the spiking behavior characteristic of single flux quantum (SFQ) circuits positions them as promising candidates for spiking neural networks (SNNs). Our previous work showcased a JJ-Soma design capable of operating at tens of gigahertz while consuming only a fraction of the power compared to traditional circuits, as documented in [1]. This paper introduces a compact SFQ-based synapse design that applies positive and negative weighted inputs to the JJ-Soma. Using an RSFQ synapse empowers us to replicate the functionality of a biological neuron, a crucial step in realizing a complete SNN. The JJ-Synapse can operate at ultra-high frequencies, exhibits orders of magnitude lower power consumption than CMOS counterparts, and can be conveniently fabricated using commercial Nb processes. Furthermore, the network's flexibility enables modifications by incorporating cryo-CMOS circuits for weight value adjustments. In our endeavor, we have successfully designed, fabricated, and partially tested the JJ-Synapse within our cryocooler system. Integration with the JJ-Soma further facilitates the realization of a high-speed inference SNN.
Paper Structure (5 sections, 1 equation, 10 figures)

This paper contains 5 sections, 1 equation, 10 figures.

Table of Contents

  1. Introduction
  2. Model and Methodology
  3. Simulation Results
  4. Hardware Implementation
  5. Conclusion

Figures (10)

  • Figure 1: Block diagram of the JJ-Synapse with positive and negative inputs. Before applying this to JJ-Soma, inputs are summed in the Buffer/Quantizer (BQ) circuit.
  • Figure 2: Design of the pulse accumulator. To accumulate pulses without losing any operating speed, we use a series of SQUID. (a) Demonstration of a single cell. (b) The truth table for a single cell as it switches. (c) Two connected cells make a weight of two for a single input. (d) The truth table for two cells. (e) Two connected cells of size two for two inputs that can weigh two. (f) The truth table for 2$\times$2 unit.
  • Figure 3: The Buffer/Quantizer (BQ) circuit functions similarly to the digital SQUID. BQ will sum the negative and positive fluxes applied to its loop and then provide quantized SFQ pulses based on the change in the flux. For symmetry, we have connected a negative load to the negative part of the circuit.
  • Figure 4: Schematic of the JJ-Synapse with positive and negative inputs. The Buffer/Quantizer (BQ) circuit has a negative load attached for symmetry.
  • Figure 5: Simulation result of the JJ-Synapse with different weight values. The tested circuit is similar to the design shown in Figure. \ref{['fig:2']}(e) but with 3-bit weight values. Here, we see the weight of the circuit changes as we switch the SQUID cells on and off
  • ...and 5 more figures