On the Universal Representation Property of Spiking Neural Networks

TL;DR

The paper analyzes the expressivity of spiking neural networks under a sequence-to-sequence spike-coding paradigm. It introduces a constructive universal representation framework for spike-train functions with memory constraints, proving single-input and multi-input theorems and showing that deep, compositional SNNs can efficiently represent complex temporal functions. The results come with near-optimal lower bounds and a rich set of explicit constructions (e.g., SKIP, CLOCK_m, CEIL_m, memory modules) that illustrate how SNNs can emulate Boolean operations, periodic functions, and hierarchical compositions. The work provides a rigorous foundation for neuromorphic computing, clarifying when SNNs are especially powerful (few inputs, low temporal complexity, compositional structure) and offering a blueprint for spike-train classification tasks.

Abstract

Inspired by biology, spiking neural networks (SNNs) process information via discrete spikes over time, offering an energy-efficient alternative to the classical computing paradigm and classical artificial neural networks (ANNs). In this work, we analyze the representational power of SNNs by viewing them as sequence-to-sequence processors of spikes, i.e., systems that transform a stream of input spikes into a stream of output spikes. We establish the universal representation property for a natural class of spike train functions. Our results are fully quantitative, constructive, and near-optimal in the number of required weights and neurons. The analysis reveals that SNNs are particularly well-suited to represent functions with few inputs, low temporal complexity, or compositions of such functions. The latter is of particular interest, as it indicates that deep SNNs can efficiently capture composite functions via a modular design. As an application of our results, we discuss spike train classification. Overall, these results contribute to a rigorous foundation for understanding the capabilities and limitations of spike-based neuromorphic systems.

Figures (8)

• Figure 1: Schematic depiction of a spiking neural network acting as a classifier. An image stream is captured by a device (here represented by an eye) and encoded into input spike trains, where higher input signals increase the probability of a spike in the corresponding spike train. These signals are processed by the SNN over a time grid. The resulting output spikes determine the class label, distinguishing between categories such as 'Dogs' and 'Cats'. In \ref{['sec:classification_spike_trains']}, we provide an explicit architecture for this.
• Figure 2: A neuron receiving signal from three input neurons and its potential. The spike trains of all neurons and the membrane potential of the output neuron are displayed for no memory ($h = 0$), finite positive memory ($h \in (0, \infty)$), and infinite memory ($h = \infty$). Each input neuron emits spikes that are weighted and stored inside the membrane potential at the output neuron. A postsynaptic spike occurs when the membrane potential strictly exceeds a threshold value after which the potential is reset to zero.
• Figure 3: SNN computational unit and equivalent recurrent ANN with state-space block. In both SNN and ANN, the yellow and blue neurons denote input and output neurons, respectively. For the SNN unit, the input consists of spike trains $\mathcal{I}_1, \dots, \mathcal{I}_d$ which are all dominated by $\mathcal{N}_{\delta}$. For the ANN, the input consists of an input stream of binary vectors, $(\mathbf{x}^{(t)})_{t \in \mathbb{N}}$ which encodes the input spike trains via $\mathbf{x}^{(t)} := (\mathds{1}(t\delta \in \mathcal{I}_j))_{j \in [d]}$. The state-space block of the ANN consists of two hidden green neurons, one hidden neuron has activation function $\mathds{1}(x>1)$ and generates the output spike train and the other hidden neuron has activation function $x \mathds{1}(0 \leq x \leq 1)$ and encodes the potential. The output of the latter neuron is delayed by the time unit $\delta$ (dashed connections).
• Figure 4: SNNs implementing periodic functions. Explicit construction via SNNs for $(a)$SKIP function, $(b)$ODD/EVEN function, and $(c)$CLOCK$_4$ function with corresponding input and output spike trains. Input, hidden, and output neurons are depicted in yellow, green, and blue, respectively.
• Figure 5: SNNs implementing Boolean operations. Explicit constructions via SNNs for the OR, AND, and MINUS function based on two inputs.

Theorems & Definitions (90)

• Definition 2.1
• Definition 2.2: Domination
• Proposition 2.3
• Definition 2.4: SNN computational unit
• Definition 2.5: SNN layer
• Remark 2.6: Excitatory and inhibitory neurons
• Definition 2.7: Feedforward SNN
• Corollary 2.8
• Definition 3.1
• Definition 3.2: Finite and periodic functions