When Spiking neural networks meet temporal attention image decoding and adaptive spiking neuron

TL;DR

This paper addresses the underutilization of temporal information and fixed thresholds in Spiking Neural Networks (SNNs) for image tasks. It introduces Temporal Attention Image Decoding (TAID) to extract temporal-channel correlations from SNN outputs and Adaptive Leaky-Integrate-and-Fire (ALIF) neurons with learnable time constant $\tau$ and threshold $V_{th}$, enabling end-to-end training via surrogate gradients. The combined TAID-ALIF framework is evaluated on image generation with Fully Spiking Variational Autoencoders (FSVAE) and image classification, achieving state-of-the-art image-generation metrics (e.g., IS, FID, FAD) and competitive MNIST/CIFAR-10 accuracies, including $99.78\%$ on MNIST with only 8 time steps and $93.89\%$ on CIFAR-10. The work demonstrates the practical impact of adaptive thresholds and temporal attention in SNNs for high-quality image decoding and robust classification, with code publicly available. The proposed methods offer a scalable path to leverage temporal dynamics in neuromorphic computation for vision tasks.

Abstract

Spiking Neural Networks (SNNs) are capable of encoding and processing temporal information in a biologically plausible way. However, most existing SNN-based methods for image tasks do not fully exploit this feature. Moreover, they often overlook the role of adaptive threshold in spiking neurons, which can enhance their dynamic behavior and learning ability. To address these issues, we propose a novel method for image decoding based on temporal attention (TAID) and an adaptive Leaky-Integrate-and-Fire (ALIF) neuron model. Our method leverages the temporal information of SNN outputs to generate high-quality images that surpass the state-of-the-art (SOTA) in terms of Inception score, Fréchet Inception Distance, and Fréchet Autoencoder Distance. Furthermore, our ALIF neuron model achieves remarkable classification accuracy on MNIST (99.78\%) and CIFAR-10 (93.89\%) datasets, demonstrating the effectiveness of learning adaptive thresholds for spiking neurons. The code is available at https://github.com/bollossom/ICLR_TINY_SNN.

Figures (3)

• Figure 1: (a): the workflow of fully spiking variation autoencoder. (b): comparison between the original image decoding method and TAID.
• Figure 2: Generated images of CIFAR10
• Figure 3: Generated images of CelebA