Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Fully Hardware Implemented Accelerator Design in ReRAM Analog Computing without ADCs

Peng Dang, Huawei Li, Wei Wang

TL;DR

The paper addresses the energy and area overhead of traditional ReRAM-based CiM accelerators caused by DAC/ADC peripherals and explicit activation computations. It introduces RACA, a fully hardware-implemented analog accelerator that uses Nyquist noise from ReRAM to realize stochastic Sigmoid activations and a Winner-Takes-All SoftMax readout, removing the need for DACs/ADCs in hidden and output layers. The approach leverages stochastic binarization of activations and crossbar-based in-situ computation, achieving competitive inference accuracy while delivering substantial hardware efficiency gains. On MNIST with a fully connected network, RACA demonstrates notable energy and area reductions and robust performance under stochastic inference, highlighting a practical path toward low-power analog CiM neural networks.

Abstract

Emerging ReRAM-based accelerators process neural networks via analog Computing-in-Memory (CiM) for ultra-high energy efficiency. However, significant overhead in peripheral circuits and complex nonlinear activation modes constrain system energy efficiency improvements. This work explores the hardware implementation of the Sigmoid and SoftMax activation functions of neural networks with stochastically binarized neurons by utilizing sampled noise signals from ReRAM devices to achieve a stochastic effect. We propose a complete ReRAM-based Analog Computing Accelerator (RACA) that accelerates neural network computation by leveraging stochastically binarized neurons in combination with ReRAM crossbars. The novel circuit design removes significant sources of energy/area efficiency degradation, i.e., the Digital-to-Analog and Analog-to-Digital Converters (DACs and ADCs) as well as the components to explicitly calculate the activation functions. Experimental results show that our proposed design outperforms traditional architectures across all overall performance metrics without compromising inference accuracy.

A Fully Hardware Implemented Accelerator Design in ReRAM Analog Computing without ADCs

TL;DR

The paper addresses the energy and area overhead of traditional ReRAM-based CiM accelerators caused by DAC/ADC peripherals and explicit activation computations. It introduces RACA, a fully hardware-implemented analog accelerator that uses Nyquist noise from ReRAM to realize stochastic Sigmoid activations and a Winner-Takes-All SoftMax readout, removing the need for DACs/ADCs in hidden and output layers. The approach leverages stochastic binarization of activations and crossbar-based in-situ computation, achieving competitive inference accuracy while delivering substantial hardware efficiency gains. On MNIST with a fully connected network, RACA demonstrates notable energy and area reductions and robust performance under stochastic inference, highlighting a practical path toward low-power analog CiM neural networks.

Abstract

Emerging ReRAM-based accelerators process neural networks via analog Computing-in-Memory (CiM) for ultra-high energy efficiency. However, significant overhead in peripheral circuits and complex nonlinear activation modes constrain system energy efficiency improvements. This work explores the hardware implementation of the Sigmoid and SoftMax activation functions of neural networks with stochastically binarized neurons by utilizing sampled noise signals from ReRAM devices to achieve a stochastic effect. We propose a complete ReRAM-based Analog Computing Accelerator (RACA) that accelerates neural network computation by leveraging stochastically binarized neurons in combination with ReRAM crossbars. The novel circuit design removes significant sources of energy/area efficiency degradation, i.e., the Digital-to-Analog and Analog-to-Digital Converters (DACs and ADCs) as well as the components to explicitly calculate the activation functions. Experimental results show that our proposed design outperforms traditional architectures across all overall performance metrics without compromising inference accuracy.
Paper Structure (13 sections, 14 equations, 6 figures, 1 table)

This paper contains 13 sections, 14 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Hardware Signal-to-Noise Ratio
  4. Weight Mapping
  5. Methodology
  6. Binary Stochastic Sigmoid Neurons
  7. WTA Binary Stochastic SoftMax Neuron
  8. RACA Architecture
  9. Evaluation
  10. Sigmoid Experimental Results
  11. SoftMax Experimental Results
  12. RACA Accelerators Architecture Experimental Results
  13. Conclusion

Figures (6)

  • Figure 1: The conventional CiM architecture.
  • Figure 2: The implementation of the binary stochastic Sigmoid neurons by utilizing the Nyquist resistor noise.
  • Figure 3: The implementation of the binary stochastic SoftMax neurons by utilizing the resistor noise and WTA rules.
  • Figure 4: Simulation results of Sigmoid neurons.
  • Figure 5: Simulation results of SoftMax neurons.
  • ...and 1 more figures