Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Curriculum Design Helps Spiking Neural Networks to Classify Time Series

Chenxi Sun, Hongyan Li, Moxian Song, Derun Can, Shenda Hong

TL;DR

The paper tackles time-series classification with Spiking Neural Networks (SNNs) and shows that Curriculum Learning (CL) can be tailored to spiking dynamics. It proposes CSNN, combining Active-to-Dormant training order (A2D) and Value-based Regional Encoding (RE) to guide learning order and input representation. Theoretical analysis links CL to a reshaped optimization landscape, and experiments across UCR time-series benchmarks and healthcare datasets show CSNN achieving state-of-the-art results and faster convergence, with about a 3% accuracy boost for SNNs. The results indicate CL is particularly beneficial for SNNs, enabling more accurate, sparse, and robust event-driven time-series models with improved resistance to noise.

Abstract

Spiking Neural Networks (SNNs) have a greater potential for modeling time series data than Artificial Neural Networks (ANNs), due to their inherent neuron dynamics and low energy consumption. However, it is difficult to demonstrate their superiority in classification accuracy, because current efforts mainly focus on designing better network structures. In this work, enlighten by brain-inspired science, we find that, not only the structure but also the learning process should be human-like. To achieve this, we investigate the power of Curriculum Learning (CL) on SNNs by designing a novel method named CSNN with two theoretically guaranteed mechanisms: The active-to-dormant training order makes the curriculum similar to that of human learning and suitable for spiking neurons; The value-based regional encoding makes the neuron activity to mimic the brain memory when learning sequential data. Experiments on multiple time series sources including simulated, sensor, motion, and healthcare demonstrate that CL has a more positive effect on SNNs than ANNs with about twice the accuracy change, and CSNN can increase about 3% SNNs' accuracy by improving network sparsity, neuron firing status, anti-noise ability, and convergence speed.

Curriculum Design Helps Spiking Neural Networks to Classify Time Series

TL;DR

The paper tackles time-series classification with Spiking Neural Networks (SNNs) and shows that Curriculum Learning (CL) can be tailored to spiking dynamics. It proposes CSNN, combining Active-to-Dormant training order (A2D) and Value-based Regional Encoding (RE) to guide learning order and input representation. Theoretical analysis links CL to a reshaped optimization landscape, and experiments across UCR time-series benchmarks and healthcare datasets show CSNN achieving state-of-the-art results and faster convergence, with about a 3% accuracy boost for SNNs. The results indicate CL is particularly beneficial for SNNs, enabling more accurate, sparse, and robust event-driven time-series models with improved resistance to noise.

Abstract

Spiking Neural Networks (SNNs) have a greater potential for modeling time series data than Artificial Neural Networks (ANNs), due to their inherent neuron dynamics and low energy consumption. However, it is difficult to demonstrate their superiority in classification accuracy, because current efforts mainly focus on designing better network structures. In this work, enlighten by brain-inspired science, we find that, not only the structure but also the learning process should be human-like. To achieve this, we investigate the power of Curriculum Learning (CL) on SNNs by designing a novel method named CSNN with two theoretically guaranteed mechanisms: The active-to-dormant training order makes the curriculum similar to that of human learning and suitable for spiking neurons; The value-based regional encoding makes the neuron activity to mimic the brain memory when learning sequential data. Experiments on multiple time series sources including simulated, sensor, motion, and healthcare demonstrate that CL has a more positive effect on SNNs than ANNs with about twice the accuracy change, and CSNN can increase about 3% SNNs' accuracy by improving network sparsity, neuron firing status, anti-noise ability, and convergence speed.
Paper Structure (15 sections, 12 theorems, 25 equations, 3 figures, 2 tables, 1 algorithm)

This paper contains 15 sections, 12 theorems, 25 equations, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Classification of Time Series (CTS)
  4. Spiking Neural Networks (SNNs)
  5. Curriculum Learning (CL)
  6. Methods
  7. Active-to-dormant Training Order (A2D)
  8. Value-based Regional Encoding (RE)
  9. Curriculum for SNNs (CSNN)
  10. Experiments
  11. Experimental Setup
  12. Classification Accuracy
  13. Curriculum Learning Performance
  14. Conclusion
  15. Theorems, Propositions, and Proofs

Key Result

Theorem 1

In SNN, different input orders of spike trains make the neuron output different spike trains.

Figures (3)

  • Figure 1: Curriculum of Time Series Data for Recurrent Spiking Neural Network
  • Figure 2: Changes in Classification Accuracy of Different ANNs and SNNs after Applying Curriculum Learning
  • Figure 3: Improvements in Model Sparsity, Firing Statutes, Convergence Process, and Anti-noise Performance after Using CSNN

Theorems & Definitions (24)

  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Proposition 1
  • proof
  • Theorem 4
  • proof
  • ...and 14 more