Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sigma-Delta Neural Network Conversion on Loihi 2

Matthew Brehove, Sadia Anjum Tumpa, Espoir Kyubwa, Naresh Menon, Vijaykrishnan Narayanan

TL;DR

This work tackles the challenge of training efficient spiking networks by converting trained ANNs into Sigma-Delta Neuron Networks (SDNNs) that run on Intel Loihi 2, leveraging graded spikes to exploit temporal and spatial sparsity. The authors develop a quantization-compatible conversion pipeline, adjusting Loihi 2 microcode with fixed-point scaling to support 8-bit weights and activations, and they introduce a delta-based neuron paradigm deployed via Lava. They train a YOLO-KP detector on a drone-detection dataset through teacher-student distillation from MobileNetV2, quantify the accuracy-sparsity tradeoff by thresholding SDNN activity, and benchmark performance against an NVIDIA Jetson Xavier. Results show substantial sparsity gains and favorable energy metrics on Loihi, but latency and energy efficiency are highly sensitive to IO and inter-chip communication, highlighting the potential for better gains with event-based sensing or larger single-chip Loihi deployments. Overall, the paper proves the feasibility of ANN-to-SDNN conversion for Loihi 2 and provides a practical framework and benchmarks for neuromorphic deployment of vision tasks, with implications for energy-efficient edge AI.

Abstract

Neuromorphic computing aims to improve the efficiency of artificial neural networks by taking inspiration from biological neurons and leveraging temporal sparsity, spatial sparsity, and compute near/in memory. Although these approaches have shown efficiency gains, training these spiking neural networks (SNN) remains difficult. The original attempts at converting trained conventional analog neural networks (ANN) to SNNs used the rate of binary spikes to represent neuron activations. This required many simulation time steps per inference, which degraded efficiency. Intel's Loihi 2 is a neuromorphic platform that supports graded spikes which can be used to represent changes in neuron activation. In this work, we use Loihi 2's graded spikes to develop a method for converting ANN networks to spiking networks, which take advantage of temporal and spatial sparsity. We evaluated the performance of this network on Loihi 2 and compared it to NVIDIA's Jetson Xavier edge AI platform.

Sigma-Delta Neural Network Conversion on Loihi 2

TL;DR

This work tackles the challenge of training efficient spiking networks by converting trained ANNs into Sigma-Delta Neuron Networks (SDNNs) that run on Intel Loihi 2, leveraging graded spikes to exploit temporal and spatial sparsity. The authors develop a quantization-compatible conversion pipeline, adjusting Loihi 2 microcode with fixed-point scaling to support 8-bit weights and activations, and they introduce a delta-based neuron paradigm deployed via Lava. They train a YOLO-KP detector on a drone-detection dataset through teacher-student distillation from MobileNetV2, quantify the accuracy-sparsity tradeoff by thresholding SDNN activity, and benchmark performance against an NVIDIA Jetson Xavier. Results show substantial sparsity gains and favorable energy metrics on Loihi, but latency and energy efficiency are highly sensitive to IO and inter-chip communication, highlighting the potential for better gains with event-based sensing or larger single-chip Loihi deployments. Overall, the paper proves the feasibility of ANN-to-SDNN conversion for Loihi 2 and provides a practical framework and benchmarks for neuromorphic deployment of vision tasks, with implications for energy-efficient edge AI.

Abstract

Neuromorphic computing aims to improve the efficiency of artificial neural networks by taking inspiration from biological neurons and leveraging temporal sparsity, spatial sparsity, and compute near/in memory. Although these approaches have shown efficiency gains, training these spiking neural networks (SNN) remains difficult. The original attempts at converting trained conventional analog neural networks (ANN) to SNNs used the rate of binary spikes to represent neuron activations. This required many simulation time steps per inference, which degraded efficiency. Intel's Loihi 2 is a neuromorphic platform that supports graded spikes which can be used to represent changes in neuron activation. In this work, we use Loihi 2's graded spikes to develop a method for converting ANN networks to spiking networks, which take advantage of temporal and spatial sparsity. We evaluated the performance of this network on Loihi 2 and compared it to NVIDIA's Jetson Xavier edge AI platform.
Paper Structure (15 sections, 3 figures, 3 tables, 1 algorithm)

This paper contains 15 sections, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Conversion and Loihi Implementation
  5. Dataset
  6. Model and Training
  7. Accuracy-Sparsity Tradeoff Exploration
  8. Benchmarking Loihi
  9. Benchmarking NVIDIA Jetson
  10. Results
  11. Model Accuracy
  12. SLAYER Training Comparisons
  13. Power, Latency, and Throughput
  14. Comparison with Intel's published work
  15. Discussion

Figures (3)

  • Figure 1: In an ANN neuron (A) all of the activations from the previous layer are multiplied by their respective weights to calculate the preactivation ($y'$). The ReLU operation is applied to calculate the activation ($y$) which is sent to the next layer. For a quantized sigma-delta neuron running on Loihi (B), the changes in the previous layer activations are multiplied by quantized weights to calculate $\Delta y'$. This is then added to a running sum and passed through a ReLU operation to find $y$. $y$ is then rescaled by multiplication and right-shift. If this rescaled value has changed by more than a set threshold, a spike is generated with the value of that change. This approach allows for the neuron to only perform calculation when the input of the neuron has changed, but otherwise behaves exactly like a conventional ANN neuron. (C) Overall workflow for our experiments. The quantized and converted network runs as an SDNN on Loihi. Frames are converted into sparse delta spikes, passed to Loihi, and the output spikes added to a running sum and interpreted as bounding boxes.
  • Figure 2: Results from the model as run on the Jetson Xavier in full precision and as a SDNN on Loihi. Along the x-axis of each plot we vary the number of layers in the 10-layer network that have thresholds raised to 1 to increase sparsity. (A) Mean average precision. (B) Power usage. (C) Frames per second. (D) Time between input and corresponding output. (E) Energy-delay product (J-$\mu s$) (F) Ratio of the Loihi synaptic operations per frame and the number of multiply accumulate operations performed per frame in the ANN network on the Jetson Xavier. As expected, increasing the thresholds dropped the accuracy, increased the speed, and reduced the number of operations performed.
  • Figure 3: Results from the model as run on Loihi with and without off-board IO with a varying number of layers with thresholds set to 1 to adjust sparsity. Input spikes were loaded into Loihi memory so that the model could run without off-board communication.