Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Knowing When to Stop: Delay-Adaptive Spiking Neural Network Classifiers with Reliability Guarantees

Jiechen Chen, Sangwoo Park, Osvaldo Simeone

TL;DR

SpikeCP introduces a delay-adaptive, set-valued inference framework for spiking neural networks with formal reliability guarantees via conformal prediction. By predefining checkpoints and applying a Bonferroni-corrected CP threshold, SpikeCP ensures the true label lies in the predicted set with probability at least $p_{\rm targ}$ at stopping times, independent of calibration data availability. A CP-aware training regime further reduces average delay by directly penalizing large predicted sets while preserving accuracy. Empirical results on MNIST-DVS, DVS128 Gesture, and CIFAR-10 demonstrate reliable latency-energy trade-offs and improved efficiency over prior adaptive SNN methods, with Simes-based heuristic alternatives offering potential latency gains in some regimes.

Abstract

Spiking neural networks (SNNs) process time-series data via internal event-driven neural dynamics. The energy consumption of an SNN depends on the number of spikes exchanged between neurons over the course of the input presentation. Typically, decisions are produced after the entire input sequence has been processed. This results in latency and energy consumption levels that are fairly uniform across inputs. However, as explored in recent work, SNNs can produce an early decision when the SNN model is sufficiently ``confident'', adapting delay and energy consumption to the difficulty of each example. Existing techniques are based on heuristic measures of confidence that do not provide reliability guarantees, potentially exiting too early. In this paper, we introduce a novel delay-adaptive SNN-based inference methodology that, wrapping around any pre-trained SNN classifier, provides guaranteed reliability for the decisions produced at input-dependent stopping times. The approach, dubbed SpikeCP, leverages tools from conformal prediction (CP). It entails minimal complexity increase as compared to the underlying SNN, requiring only additional thresholding and counting operations at run time. SpikeCP is also extended to integrate a CP-aware training phase that targets delay performance. Variants of CP based on alternative confidence correction schemes, from Bonferroni to Simes, are explored, and extensive experiments are described using the MNIST-DVS data set, DVS128 Gesture dataset, and CIFAR-10 dataset.

Knowing When to Stop: Delay-Adaptive Spiking Neural Network Classifiers with Reliability Guarantees

TL;DR

SpikeCP introduces a delay-adaptive, set-valued inference framework for spiking neural networks with formal reliability guarantees via conformal prediction. By predefining checkpoints and applying a Bonferroni-corrected CP threshold, SpikeCP ensures the true label lies in the predicted set with probability at least at stopping times, independent of calibration data availability. A CP-aware training regime further reduces average delay by directly penalizing large predicted sets while preserving accuracy. Empirical results on MNIST-DVS, DVS128 Gesture, and CIFAR-10 demonstrate reliable latency-energy trade-offs and improved efficiency over prior adaptive SNN methods, with Simes-based heuristic alternatives offering potential latency gains in some regimes.

Abstract

Spiking neural networks (SNNs) process time-series data via internal event-driven neural dynamics. The energy consumption of an SNN depends on the number of spikes exchanged between neurons over the course of the input presentation. Typically, decisions are produced after the entire input sequence has been processed. This results in latency and energy consumption levels that are fairly uniform across inputs. However, as explored in recent work, SNNs can produce an early decision when the SNN model is sufficiently ``confident'', adapting delay and energy consumption to the difficulty of each example. Existing techniques are based on heuristic measures of confidence that do not provide reliability guarantees, potentially exiting too early. In this paper, we introduce a novel delay-adaptive SNN-based inference methodology that, wrapping around any pre-trained SNN classifier, provides guaranteed reliability for the decisions produced at input-dependent stopping times. The approach, dubbed SpikeCP, leverages tools from conformal prediction (CP). It entails minimal complexity increase as compared to the underlying SNN, requiring only additional thresholding and counting operations at run time. SpikeCP is also extended to integrate a CP-aware training phase that targets delay performance. Variants of CP based on alternative confidence correction schemes, from Bonferroni to Simes, are explored, and extensive experiments are described using the MNIST-DVS data set, DVS128 Gesture dataset, and CIFAR-10 dataset.
Paper Structure (27 sections, 1 theorem, 33 equations, 12 figures, 1 table, 1 algorithm)

This paper contains 27 sections, 1 theorem, 33 equations, 12 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Motivation
  3. SpikeCP
  4. Related Work
  5. Main Contributions and Paper Organization
  6. Problem Definition
  7. Multi-Class Time Series Classification
  8. Taxonomy of SNN Classifiers
  9. Reliability, Latency, and Inference Energy
  10. Spiking Neural Network Model
  11. Adaptive Point Classification
  12. SpikeCP: Reliable Adaptive Set Classification
  13. Stopping Time
  14. Evaluation of the Threshold
  15. Reliability Guarantees of SpikeCP
  16. ...and 12 more sections

Key Result

Theorem 1

The adaptive decision $\Gamma(\hbox{\boldmath{$x$}})=\Gamma(\hbox{\boldmath{$x$}}^{T_s(\hbox{\boldmath{$x$}})})$ produced by SpikeCP, as described in Algorithm 1, satisfies the reliability condition (relia), and hence has a non-positive reliability gap, i.e., $\triangle R \leq 0$.

Figures (12)

  • Figure 1: (a) SNN $C$-class classification model: At time $t$, real-valued discrete-time time-series data ${\hbox{\boldmath{$x$}}}^t$ are fed to the input neurons of an SNN and processed by internal spiking neurons, whose spikes feed $C$ readout neurons. Each output neuron $c \in \{1,...,C\}$ evaluates the local spike count variable $r_c(\hbox{\boldmath{$x$}}^t)$ by accumulating the number of spikes it produces. The spike rates may be aggregated across all output neurons to produce the predictive probability vector$\{ p_c(\hbox{\boldmath{$x$}}^t)\}_{c=1}^C$. (b) Evolution of confidence and accuracy as a function of time $t$ for a conventional pre-trained SNN. As illustrated, SNN classifiers tend to be first under-confident and then over-confident with respect to the true accuracy, which may cause a positive reliability gap, i.e., a shortfall in accuracy, when the confidence level is used as an inference-stopping criterion. (c) Evolution of the (test-averaged) predicted set size (normalized by the number of classes $C=10$) and of the set accuracy as a function of time $t$ for the same pre-trained SNN when used in conjunction with the proposed SpikeCP method. The set accuracy is the probability that the true label lies inside the predicted set. It is observed that, irrespective of the stopping time, the set accuracy is always guaranteed to exceed the target accuracy level. Therefore, the inference-stopping criterion can be designed to control the trade-off between latency, and hence also energy consumption, and the size of the predicted set.
  • Figure 2: (a) A non-adaptive point classifier outputs a point decision $\hat{c}(\hbox{\boldmath{$x$}})$ after having observed the entire time series $\hbox{\boldmath{$x$}}$. (b) An adaptive point classifier stops when the confidence level of the classifier passes a given threshold $p_{\text{th}}$, producing a classification decision at an input-dependent time $T_s(\hbox{\boldmath{$x$}})$. (c) A non-adaptive set classifier produces a predicted set $\Gamma(\hbox{\boldmath{$x$}})$ consisting of a subset of the class labels after having observed the entire time series $\hbox{\boldmath{$x$}}$. (d) The adaptive set classifiers presented in this work stop at the earliest time $T_s(\hbox{\boldmath{$x$}})$ when the predicted set $\Gamma(\hbox{\boldmath{$x$}}^{T_s(\hbox{\boldmath{$x$}})})$ is sufficiently informative, in the sense that its cardinality is below a given threshold $I_\text{th}$ (in the figure we set $I_\text{th}=2$). The proposed SpikeCP method can guarantee that the predicted set $\Gamma(\hbox{\boldmath{$x$}})=\Gamma(\hbox{\boldmath{$x$}}^{T_s(\hbox{\boldmath{$x$}})})$ at the stopping time $T_s(\hbox{\boldmath{$x$}})$ includes the true label with probability no smaller than the target probability $p_{\text{targ}}$.
  • Figure 3: CP meets condition \ref{['eq:percp']} by choosing the threshold $s_{\rm th}^t$ in \ref{['cpsett']} as the $\lceil (1-\alpha)(|\mathcal{D}^\text{cal}|+1)\rceil$-th smallest value among the NC scores evaluated in the calibration set.
  • Figure 4: MNIST-DVS experiments: (a) Top-3 accuracy $\Pr(c \in \hat{\Gamma}(\hbox{\boldmath{$x$}}))$ and normalized latency $\mathbb{E}[T_s(\hbox{\boldmath{$x$}})]/T$ for the DC-SNN point classifier li2023unleashing; (b) Accuracy $\Pr(c \in \Gamma(\hbox{\boldmath{$x$}}))$ and normalized latency $\mathbb{E}[T_s(\hbox{\boldmath{$x$}})]/T$ for the proposed SpikeCP set predictor given the target set size $I_{\rm th}=3$. The shaded error bars correspond to intervals covering $95\%$ of the realized values, obtained from $50$ different draws of calibration data.
  • Figure 5: MNIST-DVS experiments: Normalized latency, inference energy, and set size (informativeness) as a function of target set size $I_{\rm th}$ for SpikeCP, assuming $p_{\rm targ}=0.9$ and $|\mathcal{D}^{\rm cal}|=200$ under the same conditions as Fig. 4.
  • ...and 7 more figures

Theorems & Definitions (2)

  • Theorem 1: Reliability of SpikeCP
  • proof