Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Time-Series Forecasting and Sequence Learning Using Memristor-based Reservoir System

Abdullah M. Zyarah, Dhireesha Kudithipudi

TL;DR

A memristor-based echo state network accelerator that features efficient temporal data processing and in situ online learning and 247× reduction in energy consumption is achieved when compared to a custom CMOS digital design implemented at the same technology node.

Abstract

Pushing the frontiers of time-series information processing in the ever-growing domain of edge devices with stringent resources has been impeded by the systems' ability to process information and learn locally on the device. Local processing and learning of time-series information typically demand intensive computations and massive storage as the process involves retrieving information and tuning hundreds of parameters back in time. In this work, we developed a memristor-based echo state network accelerator that features efficient temporal data processing and in-situ online learning. The proposed design is benchmarked using various datasets involving real-world tasks, such as forecasting the load energy consumption and weather conditions. The experimental results illustrate that the hardware model experiences a marginal degradation in performance as compared to the software counterpart. This is mainly attributed to the limited precision and dynamic range of network parameters when emulated using memristor devices. The proposed system is evaluated for lifespan, robustness, and energy-delay product. It is observed that the system demonstrates reasonable robustness for device failure below 10%, which may occur due to stuck-at faults. Furthermore, 247X reduction in energy consumption is achieved when compared to a custom CMOS digital design implemented at the same technology node.

Time-Series Forecasting and Sequence Learning Using Memristor-based Reservoir System

TL;DR

A memristor-based echo state network accelerator that features efficient temporal data processing and in situ online learning and 247× reduction in energy consumption is achieved when compared to a custom CMOS digital design implemented at the same technology node.

Abstract

Pushing the frontiers of time-series information processing in the ever-growing domain of edge devices with stringent resources has been impeded by the systems' ability to process information and learn locally on the device. Local processing and learning of time-series information typically demand intensive computations and massive storage as the process involves retrieving information and tuning hundreds of parameters back in time. In this work, we developed a memristor-based echo state network accelerator that features efficient temporal data processing and in-situ online learning. The proposed design is benchmarked using various datasets involving real-world tasks, such as forecasting the load energy consumption and weather conditions. The experimental results illustrate that the hardware model experiences a marginal degradation in performance as compared to the software counterpart. This is mainly attributed to the limited precision and dynamic range of network parameters when emulated using memristor devices. The proposed system is evaluated for lifespan, robustness, and energy-delay product. It is observed that the system demonstrates reasonable robustness for device failure below 10%, which may occur due to stuck-at faults. Furthermore, 247X reduction in energy consumption is achieved when compared to a custom CMOS digital design implemented at the same technology node.
Paper Structure (16 sections, 11 equations, 6 figures, 3 tables, 1 algorithm)

This paper contains 16 sections, 11 equations, 6 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Overview of Echo state Network
  3. System Design and Implementation
  4. ESN Accelerator Training
  5. Methodology
  6. Design Space Exploration
  7. Device Non-idealities and Process Limitations
  8. Device Model
  9. Verification Benchmark
  10. Experimental Results and Discussion
  11. Time-series Forecasting
  12. Device Failure Effect
  13. Network Lifespan
  14. Latency
  15. Energy-delay product
  16. ...and 1 more sections

Figures (6)

  • Figure 1: A high-level diagram of the ESN network which consists of an input, reservoir, and readout layer. The input works as a buffer, whereas the reservoir and readout layers are dedicated to feature extraction and classification, respectively.
  • Figure 2: The schematic of the developed ESN accelerator, including input, reservoir, and readout layers. Each layer comprises sample-and-hold circuits to discretize and temporally hold the time-series data, memristor devices to emulate the synaptic weights, and neuron circuits (leaky-integrated discrete-time continuous-value neurons in the reservoir and point neurons in the output layer). In the output layer, an additional unit, the training circuitry, is employed to enable in-situ learning.
  • Figure 3: The designed circuit used to convert ($\Phi$) into a pulse signal of fixed amplitude and variable duration. The circuit encompasses an integrator and comparator with internal positive feedback.
  • Figure 4: Fitting of the used memristor device model to the physical device characteristics provided in yang2010high
  • Figure 5: (a) The wMAPE of the developed ESN models (SW and HW) is calculated while predicting the load energy consumption for 50 hours, computed every 250 samples. (b) and (c) show the impact of various levels of stuck-on and stuck-off faults in memristor devices on the network performance while performing time-series forecasting task when using 1M1R and 2M crossbar structures, respectively.
  • ...and 1 more figures