Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A dynamic vision sensor object recognition model based on trainable event-driven convolution and spiking attention mechanism

Peng Zheng, Qian Zhou

TL;DR

The trainable event-driven convolution is proposed in this paper to update its convolution kernel through gradient descent and can extract local features of the event stream more efficiently than traditional event-driven convolution.

Abstract

Spiking Neural Networks (SNNs) are well-suited for processing event streams from Dynamic Visual Sensors (DVSs) due to their use of sparse spike-based coding and asynchronous event-driven computation. To extract features from DVS objects, SNNs commonly use event-driven convolution with fixed kernel parameters. These filters respond strongly to features in specific orientations while disregarding others, leading to incomplete feature extraction. To improve the current event-driven convolution feature extraction capability of SNNs, we propose a DVS object recognition model that utilizes a trainable event-driven convolution and a spiking attention mechanism. The trainable event-driven convolution is proposed in this paper to update its convolution kernel through gradient descent. This method can extract local features of the event stream more efficiently than traditional event-driven convolution. Furthermore, the spiking attention mechanism is used to extract global dependence features. The classification performances of our model are better than the baseline methods on two neuromorphic datasets including MNIST-DVS and the more complex CIFAR10-DVS. Moreover, our model showed good classification ability for short event streams. It was shown that our model can improve the performance of event-driven convolutional SNNs for DVS objects.

A dynamic vision sensor object recognition model based on trainable event-driven convolution and spiking attention mechanism

TL;DR

The trainable event-driven convolution is proposed in this paper to update its convolution kernel through gradient descent and can extract local features of the event stream more efficiently than traditional event-driven convolution.

Abstract

Spiking Neural Networks (SNNs) are well-suited for processing event streams from Dynamic Visual Sensors (DVSs) due to their use of sparse spike-based coding and asynchronous event-driven computation. To extract features from DVS objects, SNNs commonly use event-driven convolution with fixed kernel parameters. These filters respond strongly to features in specific orientations while disregarding others, leading to incomplete feature extraction. To improve the current event-driven convolution feature extraction capability of SNNs, we propose a DVS object recognition model that utilizes a trainable event-driven convolution and a spiking attention mechanism. The trainable event-driven convolution is proposed in this paper to update its convolution kernel through gradient descent. This method can extract local features of the event stream more efficiently than traditional event-driven convolution. Furthermore, the spiking attention mechanism is used to extract global dependence features. The classification performances of our model are better than the baseline methods on two neuromorphic datasets including MNIST-DVS and the more complex CIFAR10-DVS. Moreover, our model showed good classification ability for short event streams. It was shown that our model can improve the performance of event-driven convolutional SNNs for DVS objects.
Paper Structure (19 sections, 4 equations, 2 figures, 6 tables)

This paper contains 19 sections, 4 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Event-driven convolution for DVS object recognition
  4. Attention mechanism for DVS object recognition
  5. Method
  6. Trainable event-driven convolution module
  7. Spiking attention module
  8. Patch split and flattened block
  9. Spikformer encoder block
  10. Fully connected classification head
  11. Experiment and Result Analysis
  12. Experimental setup
  13. Performance on neuromorphic datasets
  14. Ablation studies
  15. Trainable event-driven convolution module
  16. ...and 4 more sections

Figures (2)

  • Figure 1: The overall structure of the proposed model based on trainable event-driven convolution and spiking attention mechanism
  • Figure 2: Traditional event-driven convolution and trainable event-driven convolution. (a) The index of the convolution kernel and the response map. (b) Traditional event-driven convolution. (c) The convolution kernel and the events in the response map. (d) Trainable event-driven convolution