Towards Efficient Deployment of Hybrid SNNs on Neuromorphic and Edge AI Hardware

TL;DR

The paper addresses efficient processing of dynamic vision sensor data by integrating spiking and non-spiking neural networks on a heterogeneous hardware platform that combines Intel Loihi for SNN execution and a Jetson Nano for ANN inference. It introduces a unified backpropagation-based training framework for hybrid SNN-ANN models using PyTorch and the LAVA library, together with an accumulator bridge to transfer temporally encoded spikes into the ANN domain. Hardware deployment is demonstrated with performance profiling across accuracy, latency, power, and energy, highlighting that hybrid models can surpass baseline ANN performance in energy efficiency while maintaining competitive accuracy, though inter-device communication costs remain unmeasured. The results show that using a small number of spiking layers as feature extractors can yields meaningful energy and latency benefits, with accumulator overheads found to be negligible in the evaluated configurations.

Abstract

This paper explores the synergistic potential of neuromorphic and edge computing to create a versatile machine learning (ML) system tailored for processing data captured by dynamic vision sensors. We construct and train hybrid models, blending spiking neural networks (SNNs) and artificial neural networks (ANNs) using PyTorch and Lava frameworks. Our hybrid architecture integrates an SNN for temporal feature extraction and an ANN for classification. We delve into the challenges of deploying such hybrid structures on hardware. Specifically, we deploy individual components on Intel's Neuromorphic Processor Loihi (for SNN) and Jetson Nano (for ANN). We also propose an accumulator circuit to transfer data from the spiking to the non-spiking domain. Furthermore, we conduct comprehensive performance analyses of hybrid SNN-ANN models on a heterogeneous system of neuromorphic and edge AI hardware, evaluating accuracy, latency, power, and energy consumption. Our findings demonstrate that the hybrid spiking networks surpass the baseline ANN model across all metrics and outperform the baseline SNN model in accuracy and latency.

Figures (7)

• Figure 1: The architecture of the CNN models investigated herein to process the data captured by the DVS camera.
• Figure 2: An example of accumulate operation on a 2-dimensional data with 9 timesteps (T=9) and an accumulate interval of 3 (I=3).
• Figure 3: The proposed unified python framework for training the hybrid SNN-ANN models and the corresponding deployment on a heterogenous system of neuromorphic hardware and edge AI accelerators.
• Figure 4: Comparative analysis of accuracy for various ANN, SNN, and hybrid SNN-ANN models.
• Figure 5: Inference latency for various ANN, SNN, and hybrid SNN-ANN models.