Unveiling the Potential of Spiking Dynamics in Graph Representation Learning through Spatial-Temporal Normalization and Coding Strategies

TL;DR

This paper investigates how spiking dynamics can enhance graph representation learning on non-Euclidean data. It introduces a general Graph SNN framework that combines iterative spiking message passing with a novel Spatial-Temporal Feature Normalization (STFN), enabling stable, efficient training. The authors instantiate GC-SNN and GA-SNN, analyze rate versus rank-order coding, and demonstrate competitive performance against state-of-the-art GNNs on standard node and graph tasks, while achieving substantial computational savings and energy efficiency suitable for neuromorphic hardware. Overall, the work provides a practical pathway to leverage neuromorphic computation for complex graph problems, addressing oversmoothing and convergence challenges through STFN and spiking dynamics.

Abstract

In recent years, spiking neural networks (SNNs) have attracted substantial interest due to their potential to replicate the energy-efficient and event-driven processing of biological neurons. Despite this, the application of SNNs in graph representation learning, particularly for non-Euclidean data, remains underexplored, and the influence of spiking dynamics on graph learning is not yet fully understood. This work seeks to address these gaps by examining the unique properties and benefits of spiking dynamics in enhancing graph representation learning. We propose a spike-based graph neural network model that incorporates spiking dynamics, enhanced by a novel spatial-temporal feature normalization (STFN) technique, to improve training efficiency and model stability. Our detailed analysis explores the impact of rate coding and temporal coding on SNN performance, offering new insights into their advantages for deep graph networks and addressing challenges such as the oversmoothing problem. Experimental results demonstrate that our SNN models can achieve competitive performance with state-of-the-art graph neural networks (GNNs) while considerably reducing computational costs, highlighting the potential of SNNs for efficient neuromorphic computing applications in complex graph-based scenarios.

Figures (10)

• Figure 1: The Graph SNN framework can support spiking message propagation and feature affine transformation, reconciling the graph convolution operation and spiking communication mechanism in a unified paradigm. The proposed STFN normalizes the membrane potentials along both spatial and temporal dimension, which coordinates the data distribution with threshold but also facilitates the network convergence.
• Figure 2: The schematic diagram of spatial-temporal feature normalization (STFN), where the high-dimensional pre-synapse inputs will be normalized along the feature dimension and temporal dimension.
• Figure 3: (a)(b)(c)Visualization of the last hidden layer features of the GCN on the Cora, Citeseer, and Pubmed datasets produced by t-SNE. (d)(e)(f)Visualization of the last hidden layer features of the GC-SNN on the Cora, Citeseer, and Pubmed datasets produced by t-SNE.
• Figure 4: (a) Validation accuracy variations during the training of Graph SNN across Multiple datasets. (b) Loss variations during the training of Graph SNN across Multiple datasets. (c) Comparative visualization of weight distributions learned by Graph SNN across multiple datasets.
• Figure 5: (a) Accuracy comparison; (b) Inference time steps comparison; (c) Time cost per training epoch; (d) Firing rate comparison; (e) Impact of time window length on rate encoding.