Delay Neural Networks (DeNN) for exploiting temporal information in event-based datasets

TL;DR

This work introduces Delay Neural Networks (DeNN), a temporal neural framework where neuron activations are computed from input delays and spike times, enabling exact temporal information to flow through both forward and backward passes. By learning per-synapse delays with a differentiable kernel and incorporating event-based preprocessing, standardization, and short/long-term memory, DeNN achieves competitive accuracy on image, video, and audio benchmarks with far fewer parameters and reduced energy consumption. The approach leverages a continuous-time formulation ($t_j = \sum_i \operatorname{sign}(d_{ij}^s) [ \kappa(t_i+d_{ij}) - \kappa(t_i+1) ]$, $\kappa(x)=e^{-x}$) to sidestep non-differentiability in SNNs while preserving temporal precision, and demonstrates practical advantages for event-based data using online and offline latency benefits. Overall, DeNN advances temporal coding by presenting a scalable, energy-efficient, fully temporal network suitable for neuromorphic applications and temporally rich data.

Abstract

In Deep Neural Networks (DNN) and Spiking Neural Networks (SNN), the information of a neuron is computed based on the sum of the amplitudes (weights) of the electrical potentials received in input from other neurons. We propose here a new class of neural networks, namely Delay Neural Networks (DeNN), where the information of a neuron is computed based on the sum of its input synaptic delays and on the spike times of the electrical potentials received from other neurons. This way, DeNN are designed to explicitly use exact continuous temporal information of spikes in both forward and backward passes, without approximation. (Deep) DeNN are applied here to images and event-based (audio and visual) data sets. Good performances are obtained, especially for datasets where temporal information is important, with much less parameters and less energy than other models.

Figures (11)

• Figure 1: Slow (left) and fast (right) regimes of the DeNN. Each layer outputs spikes after an integration phase, which duration is calibrated by $q$. If $q=1$, then each layer has to wait until every neuron of the preceding layer has emitted a spike, which corresponds to the slow regime. To reduce the latency of the model, it is possible to decrease $q$ (fast regime), so that each layer can ignore the slowest neurons of the preceding layer.
• Figure 2: DeNN full pipeline: (1) Events arrive every $\Delta t$ timestep, where $\Delta t$ is the precision of the neuromorphic camera, and are used such as by SNN. (2) Using event2time algorithm, events are aggregated over $T\Delta t$ timesteps, for one sample, and fed to the network in an on-line manner. (3) After all events are computed, the sequence of $M$ images can be simulated much faster jumping from one image to the other without waiting for every $\Delta t$ timestep, giving rise to an off-line latency. (4) Feed-forward network used for each image.
• Figure 3: Graph of probabilities $p$ for each class of the dataset given the past at each timestep $t$, for a sound of the GSC dataset ("Off").
• Figure 4: Ablation studies for DVS-Gesture and GSC
• Figure 5: Top line: Input images $I_{2}$ to $I_{6}$ from a right-hand counter clockwise movement of the DVS-Gesture dataset, obtained after application of our preprocessing algorithm. Bottom line: Differences $\delta$ between neurons of the first convolutional layer's feature maps at images $I_{s}$ and $I_{s-1}$. Darker pixels indicate faster neurons between two timesteps. Note that neurons in the range $[-1, 1]$ are canceled for clarity of image, leaving us with neurons where the difference is significant enough.