SAFA-SNN: Sparsity-Aware On-Device Few-Shot Class-Incremental Learning with Fast-Adaptive Structure of Spiking Neural Network
Huijing Zhang, Muyang Cao, Linshan Jiang, Xin Du, Di Yu, Changze Lv, Shuiguang Deng
TL;DR
SAFA-SNN tackles on-device few-shot class-incremental learning by integrating sparsity-aware dynamic neuronal dynamics, zeroth-order optimization for non-differentiable spikes, and fast adaptive prototype subspace projection. It introduces a channel-wise mask $ extbf{M}$ with $m_c=m{1}_{c \,\le\, \lfloor \eta C \rfloor}$ to keep most neurons stable while a subset learns new classes, and updates the threshold via $\boldsymbol{\Theta}_{t+1} = \boldsymbol{\Theta}_{t} - \mathbf{A}(\bar r_n - \bar r_b)$ with $\mathbf{A} = \beta(\mathbf{I}-\mathbf{M}) + \gamma\mathbf{M}$. For gradient estimation, it employs multi-point zeroth-order optimization with $\hat{\nabla} f(x) = \frac{\phi(d)}{\mu}\sum_{i=1}^b [f(x+\delta z_i) - f(x-\delta z_i)]$, enabling learning despite spike non-differentiability. Prototypes are adaptively combined through subspace projection using base and new prototypes ($\tilde B$, $\tilde C$) and matrices $G$, yielding $\tilde P = (1-\alpha)\tilde C + \alpha\tilde P_{proj}$ with $\tilde P_{proj}=\tilde C G$. Across CIFAR100, Mini-ImageNet, and three neuromorphic datasets, SAFA-SNN achieves superior last-session accuracy and reduced energy consumption on edge hardware, underscoring the practicality of sparsity-aware SNNs for continual learning on resource-constrained devices.
Abstract
Continuous learning of novel classes is crucial for edge devices to preserve data privacy and maintain reliable performance in dynamic environments. However, the scenario becomes particularly challenging when data samples are insufficient, requiring on-device few-shot class-incremental learning (FSCIL) to maintain consistent model performance. Although existing work has explored parameter-efficient FSCIL frameworks based on artificial neural networks (ANNs), their deployment is still fundamentally constrained by limited device resources. Inspired by neural mechanisms, Spiking neural networks (SNNs) process spatiotemporal information efficiently, offering lower energy consumption, greater biological plausibility, and compatibility with neuromorphic hardware than ANNs. In this work, we present an SNN-based method for On-Device FSCIL, i.e., Sparsity-Aware and Fast Adaptive SNN (SAFA-SNN). We first propose sparsity-conditioned neuronal dynamics, in which most neurons remain stable while a subset stays active, thereby mitigating catastrophic forgetting. To further cope with spike non-differentiability in gradient estimation, we employ zeroth-order optimization. Moreover, during incremental learning sessions, we enhance the discriminability of new classes through subspace projection, which alleviates overfitting to novel classes. Extensive experiments conducted on two standard benchmark datasets (CIFAR100 and Mini-ImageNet) and three neuromorphic datasets (CIFAR-10-DVS, DVS128gesture, and N-Caltech101) demonstrate that SAFA-SNN outperforms baseline methods, specifically achieving at least 4.01% improvement at the last incremental session on Mini-ImageNet and 20% lower energy cost over baseline methods with practical implementation.