SAEN-BGS: Energy-Efficient Spiking AutoEncoder Network for Background Subtraction

TL;DR

SAEN-BGS introduces a spiking autoencoder architecture for background subtraction that leverages the temporal sensitivity and noise resilience of spiking neural networks to better separate foreground objects from dynamic backgrounds. It features a continuous spiking conv-and-dconv block in the decoder and a self-distillation spiking learning algorithm within an ANN-to-SNN framework to achieve energy-efficient inference while maintaining strong segmentation performance. Across CDnet-2014 and DAVIS-2016, SAEN-BGS demonstrates superior segmentation, especially in challenging scenes, and achieves substantial energy savings compared to non-spiking baselines. The approach advances practical BGS by combining spike-based computation with robust temporal modeling, enabling lower-power deployment in real-world video analysis tasks.

Abstract

Background subtraction (BGS) is utilized to detect moving objects in a video and is commonly employed at the onset of object tracking and human recognition processes. Nevertheless, existing BGS techniques utilizing deep learning still encounter challenges with various background noises in videos, including variations in lighting, shifts in camera angles, and disturbances like air turbulence or swaying trees. To address this problem, we design a spiking autoencoder network, termed SAEN-BGS, based on noise resilience and time-sequence sensitivity of spiking neural networks (SNNs) to enhance the separation of foreground and background. To eliminate unnecessary background noise and preserve the important foreground elements, we begin by creating the continuous spiking conv-and-dconv block, which serves as the fundamental building block for the decoder in SAEN-BGS. Moreover, in striving for enhanced energy efficiency, we introduce a novel self-distillation spiking supervised learning method grounded in ANN-to-SNN frameworks, resulting in decreased power consumption. In extensive experiments conducted on CDnet-2014 and DAVIS-2016 datasets, our approach demonstrates superior segmentation performance relative to other baseline methods, even when challenged by complex scenarios with dynamic backgrounds.

Figures (8)

• Figure 1: Brain-like leaky integrate-and-fire (LIF) model corresponds to its physical circuit. The soma constantly receives integrated input from dendrites. When the sum of total input exceeds a certain threshold $\vartheta$, an output spike is generated and then delivered to other neurons by the axon. After firing a spike, the membrane potential is reset to $V_{rest}$.
• Figure 2: Separation Performance of our method and two state-of-the-art baselines over five continuous frames in the badweather video with high background noise.
• Figure 3: Architecture of the proposed SAEN-BGS. Each spiking layer integrates a CNN-based module and a spike-based module with shared weights. The CNN-based module facilitates the spike-based module training but is inactive during inference. The spike-based module processes each frame in 10 time steps.
• Figure 4: Outputs of intermediate layers of SAEN-BGS model over a randomly selected frame.
• Figure 5: Frameworks of our proposed self-distillation spiking learning algorithm.