Random Spiking Neural Networks are Stable and Spectrally Simple

TL;DR

The paper addresses the stability and robustness of wide discrete-time LIF spiking neural networks by applying Boolean function analysis to study how input perturbations propagate to outputs. It develops a framework around noise sensitivity and Fourier–Walsh spectral concentration, introducing spectral simplicity as a measure of low-frequency dominance that links stability to generalization biases. The authors prove quantitative stability bounds for single neurons and multi-layer SNNs initialized at random, show that random LIF-SNNs bias toward spectrally simple functions, and validate these findings with numerical experiments demonstrating stability both before and after training. The work provides a theoretical foundation for the robustness of energy-efficient SNNs and offers a lens to understand generalization via spectral properties of spike-based classifiers, with practical implications for neuromorphic hardware and training dynamics.

Abstract

Spiking neural networks (SNNs) are a promising paradigm for energy-efficient computation, yet their theoretical foundations-especially regarding stability and robustness-remain limited compared to artificial neural networks. In this work, we study discrete-time leaky integrate-and-fire (LIF) SNNs through the lens of Boolean function analysis. We focus on noise sensitivity and stability in classification tasks, quantifying how input perturbations affect outputs. Our main result shows that wide LIF-SNN classifiers are stable on average, a property explained by the concentration of their Fourier spectrum on low-frequency components. Motivated by this, we introduce the notion of spectral simplicity, which formalizes simplicity in terms of Fourier spectrum concentration and connects our analysis to the simplicity bias observed in deep networks. Within this framework, we show that random LIF-SNNs are biased toward simple functions. Experiments on trained networks confirm that these stability properties persist in practice. Together, these results provide new insights into the stability and robustness properties of SNNs.

Figures (7)

• Figure 1: Noise sensitivity $\operatorname{\bf{ENS}}_{1/\sqrt{n}}$ for different input dimensions $n$ for sIF and IF neurons with $\theta = 0.5$ and $T=10$. (a) sIF neuron (log-scale x-axis); dashed line: scaled bound from Theorem \ref{['thm:singleneuronbound']}. (b) IF neuron (log-scale x-axis); dashed line: scaled bound from Theorem \ref{['thm:singleneuronbound']}.
• Figure 2: Noise sensitivity $\operatorname{\bf{ENS}}_{1/\sqrt{n}}$ for different input dimensions $n$ for 5-layers sIF and IF neural networks with $\theta = 0.5$ and $T=10$. (a) sIF neuron (log-scale x-axis); (b) IF neuron (log-scale x-axis).
• Figure 3: Sensitivity to input perturbations in sLIF-SNNs ($T=100$, $\theta=0.5$, $\beta=1$, $L=3$), shown at initialization and after training on (a) MNIST and (b) CIFAR-10.
• Figure 4: Noise sensitivity $\operatorname{\bf{ENS}}_{2/n}$and $\operatorname{\bf{ENS}}_{1/(\sqrt{n}\log n)}$ for different input dimensions $n$ for sIF and IF neurons with $\theta = 0.5$ and $T=10$. (a)$\operatorname{\bf{ENS}}_{2/n}$ for sIF neuron. (b)$\operatorname{\bf{ENS}}_{2/n}$ for IF neuron; (c)$\operatorname{\bf{ENS}}_{1/(\sqrt{n}\log n)}$ for sIF neuron. (d)$\operatorname{\bf{ENS}}_{1/(\sqrt{n}\log n)}$ for IF neuron.
• Figure 5: Noise sensitivity $\operatorname{\bf{ENS}}_{2/n}$and $\operatorname{\bf{ENS}}_{1/(\sqrt{n}\log n)}$ for different input dimensions $n$ for sLIF and LIF neurons with $\theta = 0.5, T=10$ and $\beta=0.5$. (a)$\operatorname{\bf{ENS}}_{2/n}$ for sLIF neuron. (b)$\operatorname{\bf{ENS}}_{2/n}$ for LIF neuron; (c)$\operatorname{\bf{ENS}}_{1/(\sqrt{n}\log n)}$ for sLIF neuron. (d)$\operatorname{\bf{ENS}}_{1/(\sqrt{n}\log n)}$ for LIF neuron.

Theorems & Definitions (22)

• Definition 1: sLIF neuron
• Definition 2: sLIF SNN
• Definition 3: Expected noise sensitivity
• Definition 4: Expected spectrum concentration
• Theorem 1
• proof : Proof sketch of Theorem \ref{['thm:singleneuronbound']}
• Theorem 2
• proof : Proof sketch of Theorem \ref{['thm:multneustab']}
• Corollary 1
• Lemma 1: Linear Combination of Gaussians