Compressed Latent Replays for Lightweight Continual Learning on Spiking Neural Networks

TL;DR

The paper introduces a memory-efficient Latent Replay (LR) framework for continual learning in Spiking Neural Networks (SNNs) on edge devices. It splits the network into frozen and learning parts, storing latent activations at a mid-layer to replay during updates, and applies time-domain compression to LR to drastically reduce rehearsal memory with minimal accuracy loss. Empirical results on Heidelberg SHD show near-state-of-the-art Top-1 accuracy for Sample- and Class-Incremental tasks (≈92%+) and substantial memory savings (up to ≈140×), with modest forgetting in multi-class scenarios. The approach demonstrates viable on-device continual learning for keyword spotting-like tasks and supports scalable learning of many classes with low memory overhead.

Abstract

Rehearsal-based Continual Learning (CL) has been intensely investigated in Deep Neural Networks (DNNs). However, its application in Spiking Neural Networks (SNNs) has not been explored in depth. In this paper we introduce the first memory-efficient implementation of Latent Replay (LR)-based CL for SNNs, designed to seamlessly integrate with resource-constrained devices. LRs combine new samples with latent representations of previously learned data, to mitigate forgetting. Experiments on the Heidelberg SHD dataset with Sample and Class-Incremental tasks reach a Top-1 accuracy of 92.5% and 92%, respectively, without forgetting the previously learned information. Furthermore, we minimize the LRs' requirements by applying a time-domain compression, reducing by two orders of magnitude their memory requirement, with respect to a naive rehearsal setup, with a maximum accuracy drop of 4%. On a Multi-Class-Incremental task, our SNN learns 10 new classes from an initial set of 10, reaching a Top-1 accuracy of 78.4% on the full test set.

Figures (6)

• Figure 1: Visual depiction of a generic SNN with Recurrent Neurons and Latent Replays. In a), our Recurrent Neuron model; in b), the structure of a generic Fully-Connected model for Continual Learning; in c), an example of the data fed to the Latent Replay stage.
• Figure 2: Example of a lossy compression (1) to store LRs. Compression ratio is here set to 4:1. Shrinking LRs and activations reduces the memory by 4$\times$. A de-compression step (2) is required to respect the SNN time scale.
• Figure 3: Measurement of (a) forgetting and (b) Top-1 accuracy on the new scenario (Sample-Incremental CL). Latent Replays are applied to the 2$^{nd}$ to last layer of our model using 2,560 past samples for the replay.
• Figure 4: Measurement of (a) forgetting and (b) Top-1 accuracy on the new class (Class-Incremental CL). Latent Replays are applied to the 2$^{nd}$ to the last layer of our model.
• Figure 5: Top-1 accuracy on SHD's test set in Sample and Class-Incremental CL, in case of a variable number of LRs (a) with respect to the epochs, (b) with respect to the LR index.