Memory-Augmented Spiking Networks: Synergistic Integration of Complementary Mechanisms for Neuromorphic Vision

TL;DR

A five-model ablation study integrating Leaky Integrate-and-Fire neurons, Supervised Contrastive Learning, Supervised Contrastive Learning, Hopfield networks, and Hierarchical Gated Recurrent Networks on the N-MNIST dataset indicates that optimal performance emerges from architectural balance rather than isolated optimization.

Abstract

Spiking Neural Networks (SNNs) provide biological plausibility and energy efficiency, yet systematic investigations of memory augmentation strategies remain limited. We conduct a five-model ablation study integrating Leaky Integrate-and-Fire neurons, Supervised Contrastive Learning (SCL), Hopfield networks, and Hierarchical Gated Recurrent Networks (HGRN) on the N-MNIST dataset. Baseline SNNs exhibit organized neuronal groupings, or structured assemblies, characterized by a silhouette score of $0.687 \pm 0.012$. Individual augmentations introduce trade-offs: SCL improves accuracy by $0.28\%$ but reduces clustering (silhouette score $0.637 \pm 0.015$), while HGRN yields consistent gains in both accuracy ($+1.01\%$) and computational efficiency ($170.6\times$). Full integration achieves a balanced improvement across metrics, reaching a silhouette score of $0.715 \pm 0.008$, classification accuracy of $97.49 \pm 0.10\%$, energy consumption of $1.85 \pm 0.06\,μ\mathrm{J}$, and sparsity of $97.0\%$. These results indicate that optimal performance emerges from architectural balance rather than isolated optimization, establishing design principles for memory-augmented neuromorphic systems.

Figures (9)

• Figure 1: Complete Memory-Augmented SNN Architecture. Three-stage pipeline integrating: Stage 1 (SNN Encoder, blue) processes dynamic vision sensor(DVS) spikes through Conv1+LIF, Conv2+LIF, and FC(512)+LIF feature extraction; Stage 2 (Memory Augmentation, green) applies Supervised Contrastive Learning for engram formation and Hopfield networks for associative memory; Stage 3 (HGRN Temporal Gating, red) performs selective information flow with forget/update/cell gates over T=25 steps.
• Figure 2: LIF Neuron Temporal Dynamics and Circuit Model. Top: Membrane potential $U(t)$ exhibits leaky integration with decay $\beta=0.9$, generating spikes when crossing threshold $\theta=1.0$. Bottom: Circuit model showing synaptic inputs converging at soma with membrane capacitance and leak, implementing biological integrate-and-fire dynamics.
• Figure 3: Hopfield Network Module for Associative Memory. Top flowchart: Six-stage process from Input Pattern (blue, $h^0$ from SNN, 512-D with dot pattern visualization) through Weight Matrix (green, $\mathbf{W} = \sum \boldsymbol{\xi}^p (\boldsymbol{\xi}^p)^T$, outer-product learning), Energy (red, $E = -\frac{1}{2}\boldsymbol{h}^T\mathbf{W}\boldsymbol{h}$, minimization to attractors), State Update (purple, $h^{t+1} = \text{sign}(\mathbf{W}h^t)$, recurrent iterations t=0,1,2), Convergence (yellow, check $||h^{t+1}-h^t|| < \epsilon$, max $T_{max}=5$), to Retrieved Pattern (blue, $h^*$, 512-D). Purple dashed arrow shows recurrent loop. Bottom visualizations: Left shows energy landscape with global attractor (red star), local minima (blue triangles), and convergence paths (purple arrows) in 2D state space; Center shows stored pattern assemblies for digits 0-9 as colored clusters with overlap enabling generalization; Right shows convergence dynamics with pattern similarity (blue, 0$\rightarrow$1.0) and energy (red, 12$\rightarrow$1.2) over 5 iterations, demonstrating rapid convergence and energy minimization.
• Figure 4: Information Flow Control Mechanism in HGRN. Vertical flowchart showing how Previous State $C_{t-1}$ (green) influences current processing through two pathways: Forget Gate $f_t$ (orange, left) determines what to selectively discard, and Update Gate $u_t$ (orange, right) determines what to selectively integrate. Both gates converge to Current State $c_t$ (yellow) through element-wise operations. Formula box (pink) shows the cell state update equation $c_t = f_t \odot c_{t-1} + u_t \odot \tilde{c}_t$ combining forget and update operations. Arrow flows to Next State $c_{t+1}$ (purple), completing the temporal recurrence.
• Figure 5: HGRN Temporal Gating Mechanism - Detailed Module. Top: Information flow from Memory State (green, 512-D from Hopfield) through Forget Gate $f_t$ (pink, controls what to discard) and Update Gate $u_t$ (pink, controls what to integrate) to Cell State Update (yellow, gated information flow), Hidden State (blue, goal-directed representation), Hierarchical Coordination (gold, multi-level context integration), and Output (purple, to classifier). Purple dashed line shows temporal recurrence $h_{t-1} \rightarrow h_t$ maintaining memory across time. Bottom panels: Left shows complete gate equations with sigmoid and element-wise operations; Center shows HGRN performance benefits including +1.01% accuracy improvement, 0.698 silhouette, 1.85 µ J energy (170.6$\times$ vs ANNs), 97.0% sparsity, differentiable gradients, and temporal alignment with spike dynamics; Right explains why HGRN works through temporal domain operation, smooth gradients (unlike Hopfield sign function), adaptive temporal filtering, and consistent improvements across all metrics.