Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MPD-SGR: Robust Spiking Neural Networks with Membrane Potential Distribution-Driven Surrogate Gradient Regularization

Runhao Jiang, Chengzhi Jiang, Rui Yan, Huajin Tang

TL;DR

This paper addresses adversarial vulnerability in deep Spiking Neural Networks trained with surrogate gradients by linking robustness to the gradient magnitude governed by the interaction between membrane potential distribution (MPD) and the SG function. It derives an overlap-based SG magnitude $\Omega = \int_{-\gamma}^{\gamma} p(x)dx = \Phi\left(\frac{\mu+\gamma}{\sigma}\right) - \Phi\left(\frac{\mu-\gamma}{\sigma}\right)$ and proposes MPD-SGR, which regularizes the MPD to reduce this overlap. The authors provide a theoretical framework connecting robustness error to SG magnitude, and an explicit MPD-SGR loss $\mathcal{L}_{MPD-SGR}$ added to the task loss with weight $\eta$. Extensive experiments on CIFAR-10/100 and Tiny-ImageNet across VGG11 and WRN16 show improved adversarial robustness against white-box, black-box, and non-gradient perturbations, with strong generalization across SG functions and spike codes and without large clean-accuracy losses.

Abstract

The surrogate gradient (SG) method has shown significant promise in enhancing the performance of deep spiking neural networks (SNNs), but it also introduces vulnerabilities to adversarial attacks. Although spike coding strategies and neural dynamics parameters have been extensively studied for their impact on robustness, the critical role of gradient magnitude, which reflects the model's sensitivity to input perturbations, remains underexplored. In SNNs, the gradient magnitude is primarily determined by the interaction between the membrane potential distribution (MPD) and the SG function. In this study, we investigate the relationship between the MPD and SG and their implications for improving the robustness of SNNs. Our theoretical analysis reveals that reducing the proportion of membrane potentials lying within the gradient-available range of the SG function effectively mitigates the sensitivity of SNNs to input perturbations. Building upon this insight, we propose a novel MPD-driven surrogate gradient regularization (MPD-SGR) method, which enhances robustness by explicitly regularizing the MPD based on its interaction with the SG function. Extensive experiments across multiple image classification benchmarks and diverse network architectures confirm that the MPD-SGR method significantly enhances the resilience of SNNs to adversarial perturbations and exhibits strong generalizability across diverse network configurations, SG functions, and spike encoding schemes.

MPD-SGR: Robust Spiking Neural Networks with Membrane Potential Distribution-Driven Surrogate Gradient Regularization

TL;DR

This paper addresses adversarial vulnerability in deep Spiking Neural Networks trained with surrogate gradients by linking robustness to the gradient magnitude governed by the interaction between membrane potential distribution (MPD) and the SG function. It derives an overlap-based SG magnitude and proposes MPD-SGR, which regularizes the MPD to reduce this overlap. The authors provide a theoretical framework connecting robustness error to SG magnitude, and an explicit MPD-SGR loss added to the task loss with weight . Extensive experiments on CIFAR-10/100 and Tiny-ImageNet across VGG11 and WRN16 show improved adversarial robustness against white-box, black-box, and non-gradient perturbations, with strong generalization across SG functions and spike codes and without large clean-accuracy losses.

Abstract

The surrogate gradient (SG) method has shown significant promise in enhancing the performance of deep spiking neural networks (SNNs), but it also introduces vulnerabilities to adversarial attacks. Although spike coding strategies and neural dynamics parameters have been extensively studied for their impact on robustness, the critical role of gradient magnitude, which reflects the model's sensitivity to input perturbations, remains underexplored. In SNNs, the gradient magnitude is primarily determined by the interaction between the membrane potential distribution (MPD) and the SG function. In this study, we investigate the relationship between the MPD and SG and their implications for improving the robustness of SNNs. Our theoretical analysis reveals that reducing the proportion of membrane potentials lying within the gradient-available range of the SG function effectively mitigates the sensitivity of SNNs to input perturbations. Building upon this insight, we propose a novel MPD-driven surrogate gradient regularization (MPD-SGR) method, which enhances robustness by explicitly regularizing the MPD based on its interaction with the SG function. Extensive experiments across multiple image classification benchmarks and diverse network architectures confirm that the MPD-SGR method significantly enhances the resilience of SNNs to adversarial perturbations and exhibits strong generalizability across diverse network configurations, SG functions, and spike encoding schemes.

Paper Structure

This paper contains 23 sections, 1 theorem, 13 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Surrogate Gradient Learning
  4. Adversarial Defense for SNN
  5. Preliminaries
  6. Spiking Neural Networks
  7. Adversarial Attacks for SNN
  8. Methods
  9. Analysis of robustness error
  10. Analysis of Membrane Potential Distribution
  11. The relationship between MPD and SG
  12. Experiments
  13. Experimental settings
  14. Performance for various attack types
  15. Comparison with state-of-the-art work.
  16. ...and 8 more sections

Key Result

Theorem 1

In an iterative LIF model with decay factor $\tau$ over $T$ timesteps, the tdBN-normalized postsynaptic input follows the distribution $\overline{I} \sim \mathcal{N}(\beta_{c}, (\lambda_{c}\alpha V_{th})^2)$. For $t=1, 2,3,...,T$, the membrane potentials is distributed as $\overline{U}_{c}^{l}(t) \s

Figures (3)

  • Figure 1: The overall framework of MPD-SGR. The MPD-SGR constrains membrane potential distribution (MPD) of SNNs (mean $\mu$ and standard deviation $\sigma$) to reduce the overlap $\Omega$ between MPD and the gradient-available range of the SG function (gray area). This regularization minimizes output error under adversarial perturbations.
  • Figure 2: Performance of the white-box PGD attack with increasing perturbation $\epsilon$ and iterative step $k$ = 4 (Top Panels), increasing iterative step $k$ and $\epsilon$ = 8/255 (Bottom Panels).
  • Figure 3: Performance of the proposed MPD-SGR method under different black-box attacks.

Theorems & Definitions (2)

  • Theorem 1
  • proof