Towards Chip-in-the-loop Spiking Neural Network Training via Metropolis-Hastings Sampling

TL;DR

The paper tackles training SNNs on hardware with unknown non-idealities by applying Metropolis-Hastings Bayesian inference to learn weights in a chip-in-the-loop setting. It defines the likelihood L(D|W) and prior P(W) and infers the posterior P(W|D), focusing on the posterior mean W*. Experiments on the Wisconsin Breast Cancer task show MH-based training yields up to 27% accuracy improvement under distortions and requires more than 10x less training data to achieve robust generalization. The approach provides a robust, model-free training strategy for analog subthreshold neuromorphic hardware where backprop-based surrogate gradients can fail.

Abstract

This paper studies the use of Metropolis-Hastings sampling for training Spiking Neural Network (SNN) hardware subject to strong unknown non-idealities, and compares the proposed approach to the common use of the backpropagation of error (backprop) algorithm and surrogate gradients, widely used to train SNNs in literature. Simulations are conducted within a chip-in-the-loop training context, where an SNN subject to unknown distortion must be trained to detect cancer from measurements, within a biomedical application context. Our results show that the proposed approach strongly outperforms the use of backprop by up to $27\%$ higher accuracy when subject to strong hardware non-idealities. Furthermore, our results also show that the proposed approach outperforms backprop in terms of SNN generalization, needing $>10 \times$ less training data for achieving effective accuracy. These findings make the proposed training approach well-suited for SNN implementations in analog subthreshold circuits and other emerging technologies where unknown hardware non-idealities can jeopardize backprop.

Figures (3)

• Figure 1: Chip-in-the-loop training of SNN hardware. This work studies Metropolis-Hastings sampling, where the SNN weights $W$ are learned (inferred) using Bayes' rule with $P(W|D), P(W)$ the posterior and prior densities, and $L(D|W)$ the likelihood density given the data $D$ (see Section \ref{['mhdescription']}). The proposed method is compared to conventional back-propagation of error, where the weights $W$ are learned via gradient descent for minimizing the error $E$.
• Figure 2: Test accuracy in function of distortion strength $\sigma$. While backprop accuracy strongly degrades when $\sigma \geq 3$, Metropolis-Hastings accuracy is not affected by the increase in LIF distortion.
• Figure 3: Test accuracy in function of the train set size.