Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Stable Spiking Neural Networks

Jianhao Ding, Zhiyu Pan, Yujia Liu, Zhaofei Yu, Tiejun Huang

TL;DR

This work tackles the robustness of spiking neural networks by reframing them as nonlinear dynamical systems and introducing membrane potential perturbation dynamics (MPPD) as a faithful perturbation indicator. It establishes $L_2$ input-output stability bounds and proposes a training framework that minimizes the mean square of membrane potential perturbations (MS-MPPD) using Dynamic LIF (DLIF) neurons. The approach yields improved adversarial and Gaussian robustness on CIFAR-10/100 compared to state-of-the-art SNN defenses, aided by a joint loss that enforces perturbation similarity between clean and perturbed inputs. The results demonstrate a viable path toward safe, robust neuromorphic computing for safety-critical applications, with theoretical guarantees and practical gains in robustness.

Abstract

Spiking neural networks (SNNs) are gaining popularity in deep learning due to their low energy budget on neuromorphic hardware. However, they still face challenges in lacking sufficient robustness to guard safety-critical applications such as autonomous driving. Many studies have been conducted to defend SNNs from the threat of adversarial attacks. This paper aims to uncover the robustness of SNN through the lens of the stability of nonlinear systems. We are inspired by the fact that searching for parameters altering the leaky integrate-and-fire dynamics can enhance their robustness. Thus, we dive into the dynamics of membrane potential perturbation and simplify the formulation of the dynamics. We present that membrane potential perturbation dynamics can reliably convey the intensity of perturbation. Our theoretical analyses imply that the simplified perturbation dynamics satisfy input-output stability. Thus, we propose a training framework with modified SNN neurons and to reduce the mean square of membrane potential perturbation aiming at enhancing the robustness of SNN. Finally, we experimentally verify the effectiveness of the framework in the setting of Gaussian noise training and adversarial training on the image classification task.

Robust Stable Spiking Neural Networks

TL;DR

This work tackles the robustness of spiking neural networks by reframing them as nonlinear dynamical systems and introducing membrane potential perturbation dynamics (MPPD) as a faithful perturbation indicator. It establishes input-output stability bounds and proposes a training framework that minimizes the mean square of membrane potential perturbations (MS-MPPD) using Dynamic LIF (DLIF) neurons. The approach yields improved adversarial and Gaussian robustness on CIFAR-10/100 compared to state-of-the-art SNN defenses, aided by a joint loss that enforces perturbation similarity between clean and perturbed inputs. The results demonstrate a viable path toward safe, robust neuromorphic computing for safety-critical applications, with theoretical guarantees and practical gains in robustness.

Abstract

Spiking neural networks (SNNs) are gaining popularity in deep learning due to their low energy budget on neuromorphic hardware. However, they still face challenges in lacking sufficient robustness to guard safety-critical applications such as autonomous driving. Many studies have been conducted to defend SNNs from the threat of adversarial attacks. This paper aims to uncover the robustness of SNN through the lens of the stability of nonlinear systems. We are inspired by the fact that searching for parameters altering the leaky integrate-and-fire dynamics can enhance their robustness. Thus, we dive into the dynamics of membrane potential perturbation and simplify the formulation of the dynamics. We present that membrane potential perturbation dynamics can reliably convey the intensity of perturbation. Our theoretical analyses imply that the simplified perturbation dynamics satisfy input-output stability. Thus, we propose a training framework with modified SNN neurons and to reduce the mean square of membrane potential perturbation aiming at enhancing the robustness of SNN. Finally, we experimentally verify the effectiveness of the framework in the setting of Gaussian noise training and adversarial training on the image classification task.
Paper Structure (19 sections, 1 theorem, 22 equations, 5 figures, 2 tables)

This paper contains 19 sections, 1 theorem, 22 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Spiking Neural Networks
  4. Adversarial Attacks
  5. Defensive Tools for SNN
  6. Input-Output Stability
  7. Stable Spiking Neural Networks
  8. Neuronal Dynamics for Input Perturbations
  9. Metric for Measuring Perturbation
  10. Determining the Stability
  11. Stabilizing Spiking Neural Networks
  12. Experiments
  13. Experimental Setup
  14. Results
  15. Effect of $\rho$
  16. ...and 4 more sections

Key Result

Theorem 3.1

Given the membrane potential perturbation dynamics of SNN inferring for $T$ time steps as $\boldsymbol{\varepsilon }^l\left[ t \right] =\lambda \boldsymbol{\varepsilon }^l\left[ t-1 \right] +\boldsymbol{W}^l\varDelta \boldsymbol{s}^{l-1}\left[ t \right]$ for layer $l$, where $\boldsymbol{W}^l$ is th where $\gamma^l=\sqrt{1/(1-\lambda)} \Vert \boldsymbol{W}^l \Vert$ and $\beta^l=0$. $\Vert \boldsym

Figures (5)

  • Figure 1: Illustration of the membrane potential perturbation (MPP) dynamics. The LIF neuron in all subfigures receives a constant input of 0.3$u_{th}$. In (a)(b), the perturbation is +0.1$u_{th}$. In (c), the perturbation is sampled from a Gaussian distribution $\mathcal{N}(0, (0.3u_{th})^2)$.
  • Figure 2: Training paradigm of our robust stable SNN.
  • Figure 3: Effect of the parameter $\rho$.
  • Figure 4: Visualization of parameters in DLIF after proper training.
  • Figure 5: Visualization of training process of WRN16.

Theorems & Definitions (3)

  • Definition 2.1
  • Theorem 3.1
  • proof