Enhancing Adversarial Robustness in SNNs with Sparse Gradients

TL;DR

The paper addresses adversarial robustness in Spiking Neural Networks (SNNs) by tying the vulnerability gap between adversarial and random perturbations to the sparsity of the true-label input gradient. It introduces a sparsity-regularized loss (SR) that approximates the $\ell_1$ gradient norm via a finite-difference scheme and provides a theoretical bound $3 \le \frac{\rho_{adv}}{\rho_{rand}} \le 3 \| \nabla_{\mathbf{x}} f_y(\mathbf{x}) \|_0$. Empirically, SR and its adversarially trained variant SR* improve robustness across CIFAR-10/100 and dynamic vision sensor (DVS) data, outperforming state-of-the-art defenses while incurring some loss in clean accuracy. The work demonstrates that gradient sparsity is a viable and effective lever for enhancing SNN robustness, with practical implications for neuromorphic vision systems and energy-efficient AI.

Abstract

Spiking Neural Networks (SNNs) have attracted great attention for their energy-efficient operations and biologically inspired structures, offering potential advantages over Artificial Neural Networks (ANNs) in terms of energy efficiency and interpretability. Nonetheless, similar to ANNs, the robustness of SNNs remains a challenge, especially when facing adversarial attacks. Existing techniques, whether adapted from ANNs or specifically designed for SNNs, exhibit limitations in training SNNs or defending against strong attacks. In this paper, we propose a novel approach to enhance the robustness of SNNs through gradient sparsity regularization. We observe that SNNs exhibit greater resilience to random perturbations compared to adversarial perturbations, even at larger scales. Motivated by this, we aim to narrow the gap between SNNs under adversarial and random perturbations, thereby improving their overall robustness. To achieve this, we theoretically prove that this performance gap is upper bounded by the gradient sparsity of the probability associated with the true label concerning the input image, laying the groundwork for a practical strategy to train robust SNNs by regularizing the gradient sparsity. We validate the effectiveness of our approach through extensive experiments on both image-based and event-based datasets. The results demonstrate notable improvements in the robustness of SNNs. Our work highlights the importance of gradient sparsity in SNNs and its role in enhancing robustness.

Figures (6)

• Figure 1: Comparison of the random vulnerability and adversarial vulnerability of SNNs on CIFAR-10 and CIFAR-100.
• Figure 2: Illustration of (a) the proposed SR strategy, (b) gradient regularization and (c) adversarial training.
• Figure 3: The influence of the coefficient parameter $\lambda$ on classification accuracy and gradient sparsity. (a): Fluctuations in clean accuracy and adversarial accuracy under PGD attacks across different values of $\lambda$. (b): The $\ell_1$ and $\ell_2$ norms of the gradient with varying $\lambda$.
• Figure 4: The normalized distribution of $\nabla_{\bm{x}} f_y(\bm{x})$.
• Figure 5: Heatmaps of $\nabla_{\bm{x}} f_y$ where $f$ is a villain SNN (top) or an SR-SNN (down).

Theorems & Definitions (8)

• Definition 4.1
• Definition 4.2
• Theorem 4.3
• Proposition 4.4
• Theorem 1
• proof
• Proposition 1
• proof