Ultralight Polarity-Split Neuromorphic SNN for Event-Stream Super-Resolution

TL;DR

This work tackles the challenge of coarse spatial resolution in event cameras by proposing an ultra-lightweight, event-to-event super-resolution network based on Spiking Neural Networks. It introduces a Dual-Forward Polarity-Split Encoding strategy to separately process positive and negative events with a shared SNN, and a Learnable Spatio-temporal Polarity-aware Loss (LearnSTPLoss) to balance temporal, spatial, and polarity fidelity. The approach yields competitive SR performance while dramatically reducing model size and inference time, enabling real-time, on-device deployment and improved downstream tasks such as image reconstruction and object recognition. Extensive experiments across multiple datasets and real-world deployment validate the effectiveness and efficiency of the method, with notable parameter reductions and fast inference on resource-constrained hardware.

Abstract

Event cameras offer unparalleled advantages such as high temporal resolution, low latency, and high dynamic range. However, their limited spatial resolution poses challenges for fine-grained perception tasks. In this work, we propose an ultra-lightweight, stream-based event-to-event super-resolution method based on Spiking Neural Networks (SNNs), designed for real-time deployment on resource-constrained devices. To further reduce model size, we introduce a novel Dual-Forward Polarity-Split Event Encoding strategy that decouples positive and negative events into separate forward paths through a shared SNN. Furthermore, we propose a Learnable Spatio-temporal Polarity-aware Loss (LearnSTPLoss) that adaptively balances temporal, spatial, and polarity consistency using learnable uncertainty-based weights. Experimental results demonstrate that our method achieves competitive super-resolution performance on multiple datasets while significantly reducing model size and inference time. The lightweight design enables embedding the module into event cameras or using it as an efficient front-end preprocessing for downstream vision tasks.

Figures (16)

• Figure 1: An overview of event stream super-resolution with Ultralight Polarity-Split Neuromorphic SNN.
• Figure 2: Event-to-frame SR compresses the temporal dimension in exchange for spatial awareness and typically relies on heavier image SR models. Event-to-event SR requires specialized networks to directly generate asynchronous event streams, maintaining the nature of events.
• Figure 3: Architecture of key modules in our proposed event super-resolution method. Left: Main architecture of our method. Right: Composition of the Learnable Spatio-temporal Polarity-aware Loss function. (a) Dual-layer EventSR Network. (b) Dual-Forward Polarity-Split Event Encoding Strategy. (c) Ultra-lightweight Polarity-Split EventSR Network.
• Figure 4: SRM-based spiking neuron structure and spike triggering mechanism.
• Figure 5: Illustration of the main network path and the PSP-based residual upsampling bypass.