Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast Exploration of the Impact of Precision Reduction on Spiking Neural Networks

Sepide Saeedi, Alessio Carpegna, Alessandro Savino, Stefano Di Carlo

TL;DR

This work introduces an Interval Arithmetic (IA) based design-space exploration framework to study precision reduction in Spiking Neural Networks (SNNs) aimed at edge hardware. By modeling data and parameter quantization errors with a $|ian| \simeq [v]-[\epsilon]$ representation and deploying watchpoints to monitor error propagation, the method enables efficient, fine-grained exploration of bit-width reductions without sacrificing accuracy. Applied to a MNIST-trained SNN, the approach achieves substantial parameter-size reductions and demonstrates that exploration time can be dramatically shorter than exhaustive search, while keeping the spike-counter outputs intact. The results validate the practicality of IA-guided exploration for AxC in SNNs and point to future multi-objective optimizations and broader applicability to other neural architectures and hardware platforms.

Abstract

Approximate Computing (AxC) techniques trade off the computation accuracy for performance, energy, and area reduction gains. The trade-off is particularly convenient when the applications are intrinsically tolerant to some accuracy loss, as in the Spiking Neural Networks (SNNs) case. SNNs are a practical choice when the target hardware reaches the edge of computing, but this requires some area minimization strategies. In this work, we employ an Interval Arithmetic (IA) model to develop an exploration methodology that takes advantage of the capability of such a model to propagate the approximation error to detect when the approximation exceeds tolerable limits by the application. Experimental results confirm the capability of reducing the exploration time significantly, providing the chance to reduce the network parameters' size further and with more fine-grained results.

Fast Exploration of the Impact of Precision Reduction on Spiking Neural Networks

TL;DR

This work introduces an Interval Arithmetic (IA) based design-space exploration framework to study precision reduction in Spiking Neural Networks (SNNs) aimed at edge hardware. By modeling data and parameter quantization errors with a representation and deploying watchpoints to monitor error propagation, the method enables efficient, fine-grained exploration of bit-width reductions without sacrificing accuracy. Applied to a MNIST-trained SNN, the approach achieves substantial parameter-size reductions and demonstrates that exploration time can be dramatically shorter than exhaustive search, while keeping the spike-counter outputs intact. The results validate the practicality of IA-guided exploration for AxC in SNNs and point to future multi-objective optimizations and broader applicability to other neural architectures and hardware platforms.

Abstract

Approximate Computing (AxC) techniques trade off the computation accuracy for performance, energy, and area reduction gains. The trade-off is particularly convenient when the applications are intrinsically tolerant to some accuracy loss, as in the Spiking Neural Networks (SNNs) case. SNNs are a practical choice when the target hardware reaches the edge of computing, but this requires some area minimization strategies. In this work, we employ an Interval Arithmetic (IA) model to develop an exploration methodology that takes advantage of the capability of such a model to propagate the approximation error to detect when the approximation exceeds tolerable limits by the application. Experimental results confirm the capability of reducing the exploration time significantly, providing the chance to reduce the network parameters' size further and with more fine-grained results.
Paper Structure (8 sections, 7 equations, 3 figures, 1 algorithm)

This paper contains 8 sections, 7 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methods
  3. Spike Neuron Model and Network Description
  4. Data Quantization and precision reduction
  5. The Interval Arithmetic Error Propagation Model
  6. Monitoring of Acceptable Error Range
  7. Experimental Results
  8. Conclusion

Figures (3)

  • Figure 1: SNN functional model, including a detailed description of one layer
  • Figure 2: Computation Flow of SNN layer including the watch points used by the exploration algorithm
  • Figure 3: Percentage of neurons involved in changing weights at each round of exploration for one image