Line-based Event Preprocessing: Towards Low-Energy Neuromorphic Computer Vision

TL;DR

This work introduces an end-to-end line-based preprocessing mechanism for event-based neuromorphic vision, aiming to reduce energy expenditure by extracting linear features before classification. By adapting a bio-inspired line-detection SNN into a preprocessing layer and evaluating multiple strategies across PokerDVS, NMNIST, and DVS128 Gesture, the authors show substantial reductions in synaptic events while preserving, and in some cases improving, classification accuracy. Across toy and benchmark datasets, theoretical energy efficiency can improve by factors up to about 7x, with notable gains in complex real-world tasks when combining strategies. The findings support the potential of event preprocessing to enable more frugal neuromorphic computer vision and motivate hardware-oriented validation on platforms like SpiNNaker or FPGAs.

Abstract

Neuromorphic vision made significant progress in recent years, thanks to the natural match between spiking neural networks and event data in terms of biological inspiration, energy savings, latency and memory use for dynamic visual data processing. However, optimising its energy requirements still remains a challenge within the community, especially for embedded applications. One solution may reside in preprocessing events to optimise data quantity thus lowering the energy cost on neuromorphic hardware, proportional to the number of synaptic operations. To this end, we extend an end-to-end neuromorphic line detection mechanism to introduce line-based event data preprocessing. Our results demonstrate on three benchmark event-based datasets that preprocessing leads to an advantageous trade-off between energy consumption and classification performance. Depending on the line-based preprocessing strategy and the complexity of the classification task, we show that one can maintain or increase the classification accuracy while significantly reducing the theoretical energy consumption. Our approach systematically leads to a significant improvement of the neuromorphic classification efficiency, thus laying the groundwork towards a more frugal neuromorphic computer vision thanks to event preprocessing.

Figures (10)

• Figure 1: Our protocol for the evaluation of the impact of added line-based preprocessing on neuromorphic classification. The parameters $\ell$ and $n_\mathcal{P}$ respectively stand for the size in pixels of the input sensor and the number of neurons involved in the preprocessing layer (further detailed in Tab. \ref{['tab:nb_neurons_preprocessing']}).
• Figure 2: Schematic explanation adapted from gruel_aicas_2024 of the line detection model used within this work. (a) Overall connectivity from the input sensor (pictured as a square of $8{\times}8$ input pixels) to the four detectors, i.e. the four neuronal populations allowing for preprocessing. (b) Synaptic patterns of activation and inhibition between the input sensor and the bottom detector. In this example, input neurons from the lower half of the sensor are connected to the neuron $idx$ in the bottom detector with a step of $k=4$ between the diagonal connections and with an activating strength $\omega$ (see details in Sec. \ref{['sec:hyperparam_finetune']}). This pattern is reproduced for all neurons in all four detectors. Winner-Takes-All is applied on each detector to ensure model selectivity.
• Figure 3: (a) Schematic representation of the different proposed strategies for line-based preprocessing, applied to an input N-MNIST sample as an example. (b) Input and output spikes obtained from the strategy "corner quarter - inner detectors" with split polarities averaged on N-MNIST samples labeled as 6. Green and blue dots correspond respectively to the positive and negative input events; red and black correspond respectively to the model outputs for positive and negative polarities. Events are averaged over 7,000 "train" and "test" samples. (c) Output spikes produced by the intermediate preprocessing layer in the same context as (b), aggregated in a 2D format.
• Figure 4: Model architecture depending on the sensor size. The model is characterised by the number of neurons (left plot) and synapses (center plot) involved, as well as by the number of incoming synapses per neuron (right). The number of synapses is computed for a step $k=30$. The bars for merged (no hatches) and split polarities (diagonal hatches) approaches overlap: the split polarities systematically leads to an equal or higher value than the merged polarities one.
• Figure 5: Trade-off between inference efficiency $\mathds{E}$ (y-axis) and inference accuracy $\mathds{A}$ (x-axis) for varying set of hyperparameters $(k,\omega)$ according to the strategies for different line-based preprocessing applied to PokerDVS. (a) and (b) correspond respectively to merged and split polarities, with a shared legend. The "no preprocessing" trade-off is indicated by a black cross. The variation of the hyperparameter $k$ are indicated by the colour variation, whereas the size of each point increases with the strength $\omega$. The dotted black line indicates the $\mathds{A}_\text{T}$ threshold.