Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integration of Physics-Derived Memristor Models with Machine Learning Frameworks

Zhenming Yu, Stephan Menzel, John Paul Strachan, Emre Neftci

TL;DR

This work modified a physics-derived SPICE-level VCM model, integrated it with the aihwkit simulator and tested the performance with the MNIST dataset, showing that noise that disrupts the SET/RESET matching affects network performance the most.

Abstract

Simulation frameworks such MemTorch, DNN+NeuroSim, and aihwkit are commonly used to facilitate the end-to-end co-design of memristive machine learning (ML) accelerators. These simulators can take device nonidealities into account and are integrated with modern ML frameworks. However, memristors in these simulators are modeled with either lookup tables or simple analytic models with basic nonlinearities. These simple models are unable to capture certain performance-critical aspects of device nonidealities. For example, they ignore the physical cause of switching, which induces errors in switching timings and thus incorrect estimations of conductance states. This work aims at bringing physical dynamics into consideration to model nonidealities while being compatible with GPU accelerators. We focus on Valence Change Memory (VCM) cells, where the switching nonlinearity and SET/RESET asymmetry relate tightly with the thermal resistance, ion mobility, Schottky barrier height, parasitic resistance, and other effects. The resulting dynamics require solving an ODE that captures changes in oxygen vacancies. We modified a physics-derived SPICE-level VCM model, integrated it with the aihwkit simulator and tested the performance with the MNIST dataset. Results show that noise that disrupts the SET/RESET matching affects network performance the most. This work serves as a tool for evaluating how physical dynamics in memristive devices affect neural network accuracy and can be used to guide the development of future integrated devices.

Integration of Physics-Derived Memristor Models with Machine Learning Frameworks

TL;DR

This work modified a physics-derived SPICE-level VCM model, integrated it with the aihwkit simulator and tested the performance with the MNIST dataset, showing that noise that disrupts the SET/RESET matching affects network performance the most.

Abstract

Simulation frameworks such MemTorch, DNN+NeuroSim, and aihwkit are commonly used to facilitate the end-to-end co-design of memristive machine learning (ML) accelerators. These simulators can take device nonidealities into account and are integrated with modern ML frameworks. However, memristors in these simulators are modeled with either lookup tables or simple analytic models with basic nonlinearities. These simple models are unable to capture certain performance-critical aspects of device nonidealities. For example, they ignore the physical cause of switching, which induces errors in switching timings and thus incorrect estimations of conductance states. This work aims at bringing physical dynamics into consideration to model nonidealities while being compatible with GPU accelerators. We focus on Valence Change Memory (VCM) cells, where the switching nonlinearity and SET/RESET asymmetry relate tightly with the thermal resistance, ion mobility, Schottky barrier height, parasitic resistance, and other effects. The resulting dynamics require solving an ODE that captures changes in oxygen vacancies. We modified a physics-derived SPICE-level VCM model, integrated it with the aihwkit simulator and tested the performance with the MNIST dataset. Results show that noise that disrupts the SET/RESET matching affects network performance the most. This work serves as a tool for evaluating how physical dynamics in memristive devices affect neural network accuracy and can be used to guide the development of future integrated devices.
Paper Structure (8 sections, 23 equations, 3 figures)

This paper contains 8 sections, 23 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Device Modeling
  3. JART VCM Model
  4. Simplified Model
  5. Conductance Update
  6. Noise Implementation
  7. Results
  8. Conclusion

Figures (3)

  • Figure 1: Equivalent circuit diagram for the JART memristor model. Details of this device can be found inhardtdegen2018improved
  • Figure 2: Simulation results of the memristor model in Python.(a): Changing pulse length with fixed pulse amplitude. (b): Changing pulse amplitude with fixed pulse length for the SET direction(i.e. $V_M<0$). (c): Changing pulse amplitude with fixed pulse length for the RESET direction(i.e. $V_M>0$). (d): Hand-tuned SET and RESET matching.
  • Figure 3: MNIST simulation results of accuracy (a) and loss (b) for simulations with realistic mixed noise, without any noise, and with floating point weights. Accuracy of device-to-device noise at the same scale (c) and based on realistic estimation (d). Accuracy of cycle-to-cycle noise on $N_{d,max}$, $N_{d,min}$(e) and on $l_{d}$. Accuracy (f) and loss (g) for cycle-to-cycle noise on $r_{d}$.