Exploiting Spiking Dynamics with Spatial-temporal Feature Normalization in Graph Learning

TL;DR

This work introduces Graph SNNs to address non-Euclidean graph data by embedding graph convolution and attention within spiking dynamics, complemented by spatial-temporal feature normalization (STFN). It provides two instantiations, GC-SNN and GA-SNN, and demonstrates competitive semi-supervised node classification on Cora, Citeseer, and PubMed with significantly lower computation costs. STFN improves convergence and training stability, while the event-driven, sparse firing yields substantial efficiency gains suitable for neuromorphic hardware. Overall, the framework enables energy-efficient, scalable graph processing on neuromorphic platforms with practical implications for graph-based AI tasks.

Abstract

Biological spiking neurons with intrinsic dynamics underlie the powerful representation and learning capabilities of the brain for processing multimodal information in complex environments. Despite recent tremendous progress in spiking neural networks (SNNs) for handling Euclidean-space tasks, it still remains challenging to exploit SNNs in processing non-Euclidean-space data represented by graph data, mainly due to the lack of effective modeling framework and useful training techniques. Here we present a general spike-based modeling framework that enables the direct training of SNNs for graph learning. Through spatial-temporal unfolding for spiking data flows of node features, we incorporate graph convolution filters into spiking dynamics and formalize a synergistic learning paradigm. Considering the unique features of spike representation and spiking dynamics, we propose a spatial-temporal feature normalization (STFN) technique suitable for SNN to accelerate convergence. We instantiate our methods into two spiking graph models, including graph convolution SNNs and graph attention SNNs, and validate their performance on three node-classification benchmarks, including Cora, Citeseer, and Pubmed. Our model can achieve comparable performance with the state-of-the-art graph neural network (GNN) models with much lower computation costs, demonstrating great benefits for the execution on neuromorphic hardware and prompting neuromorphic applications in graphical scenarios.

Figures (4)

• Figure 1: The Graph SNN framework can support spiking message propagation and feature affine transformation, reconciling the graph convolution operation and spiking communication mechanism in a unified paradigm. The proposed STFN normalizes the membrane potentials along both spatial and temporal dimension, which coordinates the data distribution with threshold but also facilitates the network convergence.
• Figure 2: The schematic diagram of spatial-temporal feature normalization (STFN), where the high-dimensional pre-synapse inputs will be normalized along the feature dimension and temporal dimension.
• Figure 3: (a) The pre-activated membrane potential distribution before STFN (Left) and after STFN (Right). (b) Test accuracy comparison during training on Citeseer between Graph SNN models with STFN and without STFN. (c) Test loss comparison during training on Citeseer between Graph SNN models with STFN and without STFN.
• Figure 4: (a) Accuracy variation with respect to time window length (error bar denotes standard deviation from 10 runs). (b) Averaged firing rate variation during training process. (c) Operation cost comparison of feature transformation between GNNs and SNNs.