Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantization in Spiking Neural Networks

Bernhard A. Moser, Michael Lunglmayr

TL;DR

This paper investigates how leaky-integrate-and-fire (LIF) neurons can be interpreted as quantizers of spike trains. It extends the Alexiewicz norm to leaky variants and proves a bound $||\text{LIF}_{\vartheta,\alpha}(\eta) - \eta||_{A,\alpha} < \vartheta$. It proposes a modulo-based reset-to-mod reinitialization that satisfies the quantization bound under general conditions, contrasting with standard reset modes. Evaluations reveal how reinitialization choice affects error behavior and indicate a concentration-of-measure effect with increasing spike counts. The work offers theoretical error bounds and a quasi-isometry viewpoint for LIF-based SNNs, with practical implications for neuromorphic design and analysis.

Abstract

In spiking neural networks (SNN), at each node, an incoming sequence of weighted Dirac pulses is converted into an output sequence of weighted Dirac pulses by a leaky-integrate-and-fire (LIF) neuron model based on spike aggregation and thresholding. We show that this mapping can be understood as a quantization operator and state a corresponding formula for the quantization error by means of the Alexiewicz norm. This analysis has implications for rethinking re-initialization in the LIF model, leading to the proposal of 'reset-to-mod' as a modulo-based reset variant.

Quantization in Spiking Neural Networks

TL;DR

This paper investigates how leaky-integrate-and-fire (LIF) neurons can be interpreted as quantizers of spike trains. It extends the Alexiewicz norm to leaky variants and proves a bound . It proposes a modulo-based reset-to-mod reinitialization that satisfies the quantization bound under general conditions, contrasting with standard reset modes. Evaluations reveal how reinitialization choice affects error behavior and indicate a concentration-of-measure effect with increasing spike counts. The work offers theoretical error bounds and a quasi-isometry viewpoint for LIF-based SNNs, with practical implications for neuromorphic design and analysis.

Abstract

In spiking neural networks (SNN), at each node, an incoming sequence of weighted Dirac pulses is converted into an output sequence of weighted Dirac pulses by a leaky-integrate-and-fire (LIF) neuron model based on spike aggregation and thresholding. We show that this mapping can be understood as a quantization operator and state a corresponding formula for the quantization error by means of the Alexiewicz norm. This analysis has implications for rethinking re-initialization in the LIF model, leading to the proposal of 'reset-to-mod' as a modulo-based reset variant.
Paper Structure (6 sections, 1 theorem, 11 equations, 5 figures)

This paper contains 6 sections, 1 theorem, 11 equations, 5 figures.

Table of Contents

  1. Introduction
  2. LIF Model and Re-Initialization Variants
  3. Alexiewicz Norm and Spike Train Quantization
  4. Proof.
  5. Evaluations
  6. Conclusion

Key Result

Theorem 1

Given a LIF neuron model with reset-to-mod, the threshold $\vartheta>0$, the leaky parameter $\alpha \in [0,\infty]$ and the spike train $\eta \in \mathbb{S}$ with amplitudes $a_i \in \mathbb{R}$. Then, $\hbox{LIF}_{\vartheta, \alpha}(\eta)$ is a $\vartheta$-quantization of $\eta$, i.e., the result

Figures (5)

  • Figure 1: Illustration of steps of information processing by a LIF model. The weighted superposition of incoming spike trains can result in spike amplitudes that can exceed the threshold many times.
  • Figure 2: Example of input spike train with different LIF outputs depending on the re-initialization variant.
  • Figure 3: Illustration of Eqn. (\ref{['eq:quantDeltaRecursion']}). The red arrows indicate reset by reset-to-mod.
  • Figure 4: Evaluation of (\ref{['eq:quantization']}) for reset-to-mod, reset-by-subtraction and reset-to-zero (1.st/2nd/3rd column), based on spike trains with spike amplitudes in $[-\vartheta, \vartheta]$ and $100$ runs. The 1st row refers to $\alpha = 1$ and the 2nd row to $\alpha = 0.1$.
  • Figure 5: The same as in Fig. \ref{['fig:QuantizationError1']} but with spike amplitudes in $[-3/2 \vartheta, 3/2 \vartheta]$.

Theorems & Definitions (1)

  • Theorem 1: reset-to-mod LIF as $\|.\|_{A,\alpha}$-Quantization