Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Unsupervised Learning Using Spike-Timing-Dependent Plasticity

Sen Lu, Abhronil Sengupta

TL;DR

This work investigates a Deep-STDP framework where a rate-based convolutional network is trained in tandem with pseudo-labels generated by the STDP clustering process on the network outputs to achieve higher accuracy and faster convergence speed at iso-accuracy on a 10-class subset of the Tiny ImageNet dataset.

Abstract

Spike-Timing-Dependent Plasticity (STDP) is an unsupervised learning mechanism for Spiking Neural Networks (SNNs) that has received significant attention from the neuromorphic hardware community. However, scaling such local learning techniques to deeper networks and large-scale tasks has remained elusive. In this work, we investigate a Deep-STDP framework where a rate-based convolutional network, that can be deployed in a neuromorphic setting, is trained in tandem with pseudo-labels generated by the STDP clustering process on the network outputs. We achieve $24.56\%$ higher accuracy and $3.5\times$ faster convergence speed at iso-accuracy on a 10-class subset of the Tiny ImageNet dataset in contrast to a $k$-means clustering approach.

Deep Unsupervised Learning Using Spike-Timing-Dependent Plasticity

TL;DR

This work investigates a Deep-STDP framework where a rate-based convolutional network is trained in tandem with pseudo-labels generated by the STDP clustering process on the network outputs to achieve higher accuracy and faster convergence speed at iso-accuracy on a 10-class subset of the Tiny ImageNet dataset.

Abstract

Spike-Timing-Dependent Plasticity (STDP) is an unsupervised learning mechanism for Spiking Neural Networks (SNNs) that has received significant attention from the neuromorphic hardware community. However, scaling such local learning techniques to deeper networks and large-scale tasks has remained elusive. In this work, we investigate a Deep-STDP framework where a rate-based convolutional network, that can be deployed in a neuromorphic setting, is trained in tandem with pseudo-labels generated by the STDP clustering process on the network outputs. We achieve higher accuracy and faster convergence speed at iso-accuracy on a 10-class subset of the Tiny ImageNet dataset in contrast to a -means clustering approach.
Paper Structure (19 sections, 18 equations, 5 figures, 2 tables, 2 algorithms)

This paper contains 19 sections, 18 equations, 5 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Deep Clustering with k-means Algorithm
  5. STDP Enabled Neuromorphic Clustering
  6. Methods
  7. Proposed Deep-STDP Framework
  8. STDP Enabled SNN for Clustering
  9. Experiments and Results
  10. Datasets and Implementation
  11. Evaluation Metrics
  12. Fisher Information
  13. Normalized Mutual Information
  14. Performance Evaluation
  15. Computational Cost Estimation
  16. ...and 4 more sections

Figures (5)

  • Figure 1: STDP learns generic features of input patterns (MNIST dataset) in the excitatory synapses of the Winner-Take-All network. Each neuron represents a cluster and its learnt weights represent the corresponding cluster centroid.
  • Figure 2: Overall structure of Deep-STDP: The ConvNet compresses the input images to a lower-dimensional feature vector which is mapped by STDP clustering to a pseudo-label. The ConvNet is subsequently trained through backpropagation using the pseudo-labels.
  • Figure 3: Average accuracy over 5 independent runs of Deep-STDP and DeepClustering frameworks for the 10-class subset of Tiny ImageNet dataset.
  • Figure 4: Fisher information metric comparison: Deep-STDP retains more information over epochs. Pale curves are the actual data while darker curves are the smoothed plots.
  • Figure 5: Deep-STDP filter activations of Gaussian random noise from layers 6, 12, 24, and 36. We have used the unit-level visualization method proposed in Ref. erhan2009visualizing.