Transductive Spiking Graph Neural Networks for Loihi

TL;DR

The study targets efficient graph learning on sparse data using neuromorphic spiking networks implemented on Loihi 2. It introduces a fixed-precision spiking Graph Neural Network with a novel LIF Long Reset neuron and transductive graph learning on citation graphs. The architecture is integrated with Lava Bayesian Optimization to tune hyperparameters in an event-driven Lava framework, and is evaluated against floating-point baselines on the Cora dataset, achieving a final accuracy of $62.86%$ after optimization versus a $63.57%$ floating-point baseline. The results demonstrate that integer-precision neuromorphic graph learning can approach floating-point performance and be deployed on neuromorphic hardware, with future work focused on real Loihi 2 deployment and broader datasets.

Abstract

Graph neural networks have emerged as a specialized branch of deep learning, designed to address problems where pairwise relations between objects are crucial. Recent advancements utilize graph convolutional neural networks to extract features within graph structures. Despite promising results, these methods face challenges in real-world applications due to sparse features, resulting in inefficient resource utilization. Recent studies draw inspiration from the mammalian brain and employ spiking neural networks to model and learn graph structures. However, these approaches are limited to traditional Von Neumann-based computing systems, which still face hardware inefficiencies. In this study, we present a fully neuromorphic implementation of spiking graph neural networks designed for Loihi 2. We optimize network parameters using Lava Bayesian Optimization, a novel hyperparameter optimization system compatible with neuromorphic computing architectures. We showcase the performance benefits of combining neuromorphic Bayesian optimization with our approach for citation graph classification using fixed-precision spiking neurons. Our results demonstrate the capability of integer-precision, Loihi 2 compatible spiking neural networks in performing citation graph classification with comparable accuracy to existing floating point implementations.

Figures (4)

• Figure 1: A simplified graph demonstrating the architecture of our spiking neural network for citation graph classification. Red dotted arrows represent the learnable synapses.
• Figure 2: The inter-neuron dynamics of our LIF Long Reset neuron model compared to a traditional LIF neuron through two neurons recurrently connected with fully-connected layers. (A) Input spikes from the ring buffer (B) Synaptic current (C) Post synaptic spike rasters from both neurons. Each neuron was initialized with a current decay of 0.5. Each LIF Long Reset neuron variant was configured with a reset interval of 10 time steps with a reset period of 4 time steps.
• Figure 3: The multi-process and subprocess model architecture representing our Spiking GNN approach for citation network classification in Lava. For simplicity, connections within the validation and testing neurons clusters along with the entire testing cluster have been omitted. Ref and Var Ports allow for read and write operations between interconnected processes.
• Figure 4: The runtime control flow provided by the Spiking GNN Runtime process whenever it receives new parameters from LavaBO 10.1145/3589737.3605998. For more details on the specifics of the Spiking GNN, see Figure \ref{['fig:process-architecture']}.