A Graph is Worth 1-bit Spikes: When Graph Contrastive Learning Meets Spiking Neural Networks

TL;DR

SpikeGCL tackles the memory and energy bottlenecks of graph contrastive learning by learning binarized 1-bit graph representations with spiking neural networks. It introduces a time-extended encoding scheme that partitions node features into $T$ groups, uses shared GNN encoders, and trains via blockwise surrogate gradients with a margin-ranking objective. Theoretical guarantees show firing-rate SNNs can approximate full-precision GNNs with error decreasing as $T$ grows, and experiments across nine graph benchmarks demonstrate competitive accuracy with substantial storage and energy efficiency gains. This work offers a practical path toward scalable, hardware-friendly graph self-supervised learning, especially for edge and neuromorphic contexts.

Abstract

While contrastive self-supervised learning has become the de-facto learning paradigm for graph neural networks, the pursuit of higher task accuracy requires a larger hidden dimensionality to learn informative and discriminative full-precision representations, raising concerns about computation, memory footprint, and energy consumption burden (largely overlooked) for real-world applications. This work explores a promising direction for graph contrastive learning (GCL) with spiking neural networks (SNNs), which leverage sparse and binary characteristics to learn more biologically plausible and compact representations. We propose SpikeGCL, a novel GCL framework to learn binarized 1-bit representations for graphs, making balanced trade-offs between efficiency and performance. We provide theoretical guarantees to demonstrate that SpikeGCL has comparable expressiveness with its full-precision counterparts. Experimental results demonstrate that, with nearly 32x representation storage compression, SpikeGCL is either comparable to or outperforms many fancy state-of-the-art supervised and self-supervised methods across several graph benchmarks.

Figures (10)

• Figure 1: Comparison between conventional GCL and SpikeGCL. Instead of using full-precision (32-bit) representations, SpikeGCL produces sparse and compact 1-bit representations, making them more memory-friendly and computationally efficient for cheap devices with limited resources.
• Figure 2: Overview of SpikeGCL framework. (a)SpikeGCL follows the standard GCL philosophy, which learns binary representations by contrasting positive and negative samples with a margin ranking loss. (b)SpikeGCL first partitions node features into $T$ non-overlapping groups, each of which is then fed to an encoder whereby spiking neurons represent nodes of graph as 1-bit spikes.
• Figure 3: A comparison between two backpropagation learning paradigms. The backpropagation path during blockwise training is limited to a single block of networks to avoid a large memory footprint and vanishing gradient problem.
• Figure 4: Comparison of SpikeGCL and other graph SNNs in terms of accuracy (%), training time (s), memory usage (GB), and energy consumption (mJ), respectively.
• Figure 5: Average gradient norms of IF and LIF neurons, with and without the blockwise learning strategy.

Theorems & Definitions (8)

• Theorem 1: Informal
• Remark 1
• Theorem 2
• Lemma 1
• proof : Proof of Lemma \ref{['lem: reduction']}
• proof : Proof of theorem \ref{['thm: approx']}
• Remark 2
• Remark 3