Towards Robust Spiking Neural Networks:Mitigating Heterogeneous Training Vulnerability via Dominant Eigencomponent Projection

TL;DR

The paper identifies vulnerability in SNNs trained with direct encoding and BPTT under heterogeneous training, where a single perturbed batch can trigger collapse. It shows that repeated inputs and gradient accumulation cause an extremely large Hessian spectral radius $\rho(H)$, and proposes Dominant Eigencomponent Projection (DEP) to orthogonally project gradients, removing the dominant component and reducing curvature. Through extensive experiments on CIFAR-10/100, TinyImageNet, and ImageNet, DEP improves robustness under both homogeneous and heterogeneous training and outperforms several defenses with a hyperparameter-free, low-overhead implementation. This approach enables safer, more reliable deployment of energy-efficient SNNs in real-world, data-diverse environments.

Abstract

Spiking Neural Networks (SNNs) process information via discrete spikes, enabling them to operate at remarkably low energy levels. However, our experimental observations reveal a striking vulnerability when SNNs are trained using the mainstream method--direct encoding combined with backpropagation through time (BPTT): even a single backward pass on data drawn from a slightly different distribution can lead to catastrophic network collapse. Our theoretical analysis attributes this vulnerability to the repeated inputs inherent in direct encoding and the gradient accumulation characteristic of BPTT, which together produce an exceptional large Hessian spectral radius. To address this challenge, we develop a hyperparameter-free method called Dominant Eigencomponent Projection (DEP). By orthogonally projecting gradients to precisely remove their dominant components, DEP effectively reduces the Hessian spectral radius, thereby preventing SNNs from settling into sharp minima. Extensive experiments demonstrate that DEP not only mitigates the vulnerability of SNNs to heterogeneous data poisoning, but also significantly enhances overall robustness compared to key baselines, providing strong support for safer and more reliable SNN deployment.

Figures (4)

• Figure 1: The vulnerability of SNNs in heterogeneous training.
• Figure 2: SNN model collapse curves caused by hetero-training. In the "c/+p_10" setting, the model undergoes homogeneous training on a clean CIFAR-10 dataset for the first 9 epochs, after which hetero-training is initiated starting from epoch 10. During the heterogeneous phase, one batch per epoch is randomly selected and poisoned using perturbed data (FGSM with $\epsilon$ = 2). In the "p/+c" setting, homogeneous training is performed using perturbed data (FGSM with $\epsilon$ = 2), and during the heterogeneous phase, one random batch per epoch is poisoned with clean data.
• Figure 3: Performance comparison in different white box and black box attack.
• Figure :