Scalable Dendritic Modeling Advances Expressive and Robust Deep Spiking Neural Networks

TL;DR

The paper addresses the limited expressivity of point-neuron SNNs by introducing the dendritic spiking neuron (DendSN), which explicitly models dendritic morphology and nonlinear integration in a lightweight, scalable way. DendSNs are integrated into deep DendSNN architectures with GPU-accelerated dendritic computation and trained end-to-end with surrogate gradients, achieving higher task performance than conventional SNNs across static and neuromorphic datasets. A key contribution is the dendritic branch gating (DBG) mechanism for task incremental learning, which reduces inter-task interference by enforcing sparse, task-specific dendritic substructures. Additional results demonstrate enhanced robustness to noise and adversarial perturbations and improved few-shot learning, illustrating the practical advantages of dendritic computation for deep SNNs. The work also provides a detailed analysis of dendritic variants, scalability considerations, and future directions toward broader applicability and biological plausibility.

Abstract

Dendritic computation endows biological neurons with rich nonlinear integration and high representational capacity, yet it is largely missing in existing deep spiking neural networks (SNNs). Although detailed multi-compartment models can capture dendritic computations, their high computational cost and limited flexibility make them impractical for deep learning. To combine the advantages of dendritic computation and deep network architectures for a powerful, flexible and efficient computational model, we propose the dendritic spiking neuron (DendSN). DendSN explicitly models dendritic morphology and nonlinear integration in a streamlined design, leading to substantially higher expressivity than point neurons and wide compatibility with modern deep SNN architectures. Leveraging the efficient formulation and high-performance Triton kernels, dendritic SNNs (DendSNNs) can be efficiently trained and easily scaled to deeper networks. Experiments show that DendSNNs consistently outperform conventional SNNs on classification tasks. Furthermore, inspired by dendritic modulation and synaptic clustering, we introduce the dendritic branch gating (DBG) algorithm for task-incremental learning, which effectively reduces inter-task interference. Additional evaluations show that DendSNNs exhibit superior robustness to noise and adversarial attacks, along with improved generalization in few-shot learning scenarios. Our work firstly demonstrates the possibility of training deep SNNs with multiple nonlinear dendritic branches, and comprehensively analyzes the impact of dendrite computation on representation learning across various machine learning settings, thereby offering a fresh perspective on advancing SNN design.

Figures (8)

• Figure 1: A comparison between deep neural networks with point neurons and biophysical neural networks with multi-compartment neurons.(a) Deep neural networks typically adopt point neuron models. Both the perceptron in deep ANNs and the LIF model in deep SNNs belong to this category. (b) Multi-compartment models in neuroscience capture the detailed morphologies of biological neurons and have complex dendritic dynamics. By connecting these neurons, biophysical network models are established to simulate small-scale neural circuits in the brain. (c) Deep neural networks based on point neurons are computationally efficient and easy to train, but fall short on bio-plausibility and single-neuron expressivity (blue box). Biophysical networks with multi-compartment neurons have remarkable neuron-level expressivity, but cannot be easily scaled up to large networks (green box). There is a gap between these two types of models. In this work, we propose DendSN and DendSNN, combining dendritic computation with the design principles of deep learning to power up deep SNNs.
• Figure 2: DendSN and L5PC fitting task.(a) The proposed DendSN model with stateful dendrite and LIF soma. (b) The setting of the L5PC somatic potential prediction task. (c) The coefficient of determination ($R^2$) of different neuron models on L5PC somatic potential prediction. Higher is better. (d),(e) The somatic membrane potential of a detailed multi-compartment L5PC model (black, ground truth), a LIF model (blue), and a DendSN (orange and dashed, $P=4,B=2$). The trial is selected from the test set.
• Figure 3: Construction of DendSNNs and their evaluation on classification tasks.(a) A DendSN layer placed after a fully connected layer. (b) A DendSN layer placed after a 2D convolutional layer. (c) Deriving a DendSNN architecture from a PointSNN by replacing the neurons and adjusting the number of channels. (d) Accuracy comparison on Fashion-MNIST. (e) Stateful and stateless dendritic compartments. (f) Left: accuracy comparison between Stateful and Stateless Dendrites on CIFAR10-DVS. Right: CIFAR10-DVS accuracies under different $P$ and $B$ settings. (g) DendSN with dendritic residual connection. (h) Accuracy comparison on Tiny ImageNet. (i) Loss landscape visualizations of the models on Tiny ImageNet. (j) Comparison of different models' throughput and peak allocated memory when trained on Tiny ImageNet. (k) Evolution of training throughput and memory cost when $P$ (left), $B$ (mid) or $T$ (right) increases.
• Figure 4: Dendritic branch gating (DBG) for task-incremental learning (TIL).(a) Synaptic clustering on biological dendrites. (b) An illustration of DBG on a single DendSN. (c) DBG induces task-specific synaptic clusters in DendSNNs. (d) An illustration of the Permuted MNIST TIL benchmark. (e) Evolution of mean accuracy for different models on Permuted MNIST. (f) 50-task mean accuracies of different models on Permuted MNIST. (g) The effect of sparsity factor $\rho$ on TIL performance. (h) Accuracy heatmaps. The pixel on row $i$ and column $j$ represents the test accuracy of task $i$ after training on task $1$ to task $j$. Darker colors indicate higher accuracies. The wider the dark region, the less the model forgets across tasks. (i) Mean squared synaptic distance between the first-layer weights after training on task $q$ and those of the final network. dEWC and fEWC denote EWC applied to the decoder and full network, respectively. Unless otherwise stated, DendSNs use $P=4$, $B=2$, Stateful Dendrite, and Identity branch activation.
• Figure 5: Robustness of DendSNNs against noise and adversarial attacks.(a) Fashion-MNIST noise robustness experiment and its results. (b) CIFAR-100-C corruption robustness experiment and its results. rmCE: relative mean corruption error w.r.t. LIF-based SNN (lower is better). (c) White-box adversarial robustness experiment on Fashion-MNIST and its results. rmAE: relative mean adversarial error w.r.t. LIF-based SNN (lower is better). (d) Black-box adversarial robustness experiment on Fashion-MNIST and its results. DendSNs in (c) and (d) use Stateful Dendrite and Mexican hat branch activation. (e) Mean squared distance between layer-2 somatic potentials under clean and noisy inputs. (f) t-SNE visualization of layer-2 somatic potentials under clean and noisy ($\epsilon = 0.5$) inputs. (g) Distributions of decision boundary thickness (Equation \ref{['eq:methods-thickness']}) across different models.