Biologically Inspired Spiking Diffusion Model with Adaptive Lateral Selection Mechanism

TL;DR

The paper tackles generative modeling with spiking neural networks by embedding a biologically inspired lateral connectivity mechanism within a diffusion framework. It introduces a spiking diffusion model that couples a transformer-based denoiser with a learnable lateral aggregation Λ_agg and a spike-mapped substructure selection network, updated through a spiking inner loop and surrogate-gradient learning. The authors provide theoretical grounding showing that the lateral updates approximate STDP-like plasticity under a local objective, and they demonstrate through extensive experiments on MNIST, CelebA, CIFAR-10, and LSUN Bedroom that their method achieves superior FID scores compared to existing SNN-based generative models. The work highlights the practical potential of brain-inspired mechanisms for energy-efficient, adaptable generative modeling and offers a solid foundation for future neuromorphic diffusion systems.

Abstract

Lateral connection is a fundamental feature of biological neural circuits, facilitating local information processing and adaptive learning. In this work, we integrate lateral connections with a substructure selection network to develop a novel diffusion model based on spiking neural networks (SNNs). Unlike conventional artificial neural networks, SNNs employ an intrinsic spiking inner loop to process sequential binary spikes. We leverage this spiking inner loop alongside a lateral connection mechanism to iteratively refine the substructure selection network, enhancing model adaptability and expressivity. Specifically, we design a lateral connection framework comprising a learnable lateral matrix and a lateral mapping function, both implemented using spiking neurons, to dynamically update lateral connections. Through mathematical modeling, we establish that the proposed lateral update mechanism, under a well-defined local objective, aligns with biologically plausible synaptic plasticity principles. Extensive experiments validate the effectiveness of our approach, analyzing the role of substructure selection and lateral connection during training. Furthermore, quantitative comparisons demonstrate that our model consistently surpasses state-of-the-art SNN-based generative models across multiple benchmark datasets.

Figures (9)

• Figure 1: Comparison of the proposed method and other SNN-based generative models on CIFAR-10 in terms of FID and spiking time steps
• Figure 2: Overview of the proposed lateral connection spiking diffusion. $t_s = 1, 2 \dots, T_s$ denote the inner cycles of the spiking neurons. $f^{t_s}_{1:r}$ represents the portion of the entire model before the $r$-th substructure selection module at time step $t_s$, while $l^{t_s}_r$ and $\delta^{t_s}_r$ denote the initial logits to the $r$-th substructure selection module and the correction value at time step $t_s$, respectively. The lateral connections output $\delta^{t_s}_r$ and update the membrane potential $m^{t_s}_r$; then, $\delta^{t_s}_r$ is used to correct the logits $l^{t_s}_r$, resulting in the selected substructure network.
• Figure 3: The schematic diagram of the interactions between the states of the variables in the lateral connection update process in Equation \ref{['eq:lateral_1']}, \ref{['eq:lateral_2']} and \ref{['eq:lateral_3']}. This update process is similar to a model with recurrent units, where the membrane potential $m^{t_s}_r$ can be viewed as a recurrent hidden state.
• Figure 4: Denoising process of the proposed model on (A) LSUN Bedroom, (B) CelebA, (C) CIFAR-10, (D) Fashion MNIST, and (E) MNIST datasets.
• Figure 5: Visualization of generated images on CelebA, FashionMNIST, LSUN Bedroom, MNIST and CIFAR-10 datasets