Spike-based Neuromorphic Computing for Next-Generation Computer Vision

TL;DR

This paper surveys spike-based neuromorphic computing for computer vision, arguing that spike-driven, in-memory, asynchronous processing can overcome energy and memory bottlenecks in traditional von Neumann systems, especially for edge vision tasks. It surveys neuromorphic devices and circuits (MOSFET-based and memristive), state-of-the-art processors, and training architectures including ANN-to-SNN mapping, surrogate-gradient backpropagation, STDP, reservoir computing, and evolutionary methods. It then reviews vision applications on static images and event-based datasets, including a neuromorphic dancing pose-estimation system with new datasets. The discussion highlights key challenges—scalability of learning rules, device variability, dataset availability for DVS, and reliability and security—along with future directions to realize robust, energy-efficient brain-inspired vision systems.

Abstract

Neuromorphic Computing promises orders of magnitude improvement in energy efficiency compared to traditional von Neumann computing paradigm. The goal is to develop an adaptive, fault-tolerant, low-footprint, fast, low-energy intelligent system by learning and emulating brain functionality which can be realized through innovation in different abstraction layers including material, device, circuit, architecture and algorithm. As the energy consumption in complex vision tasks keep increasing exponentially due to larger data set and resource-constrained edge devices become increasingly ubiquitous, spike-based neuromorphic computing approaches can be viable alternative to deep convolutional neural network that is dominating the vision field today. In this book chapter, we introduce neuromorphic computing, outline a few representative examples from different layers of the design stack (devices, circuits and algorithms) and conclude with a few exciting applications and future research directions that seem promising for computer vision in the near future.

Figures (9)

• Figure 1: (a) Simplified neural communication schematic rose2021system (Reproduced from Rose et al., Neuromorphic Computing and Engineering 1.2 (2021). Copyright CC BY 4.0.), (b) Post-synaptic spike generation, (c) Chemical synapse Najem2018MemristiveMimics; reproduced with permission from Najem et al., ACS nano 12.5 (2018), Copyright 2018 American Chemical Society.
• Figure 2: SNN with LIF (Leaky integrate and fire) neuron, (a) construction and (b) dynamics. roy2019towards. Reproduced with permission from Roy et al., Nature 575.7784 (2019). Copyright 2019 Springer Nature.
• Figure 3: Concept of memristor and memristive behaviours of various complexity. (a) Illustrations of a biological neuron and synapses, along with the current–voltage characteristics of synaptic and neuronal electrical devices. (b,c) Examples of synaptic (panel b) and neuronal (panel c) behaviours that require different orders of complexity. (d) | Illustration of a memristor and its basic model, depicting with an example how state variables connect currents and voltages with a temporal history dependence. $\Delta t$, time delay; G, conductance; gnd, electrical ground; I, current; t, time; V, voltagekumar2022dynamical. Reproduced with permission from Kumar et al., Nature Reviews Materials 7.7 (2022). Copyright 2022 Springer Nature.
• Figure 4: Mapping procedures take a pre-trained ANN and map it into an SNN suitable for neuromorphic hardware.
• Figure 5: Spike-based quasi-backpropagation approaches adapt backpropagation to work directly on SNNs.